2024-06-13T06:25:32.503 INFO:root:teuthology version: 0.0.1.dev316+gc1b06c2 2024-06-13T06:25:32.503 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/orig.config.yaml 2024-06-13T06:25:32.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-13T06:25:32.679 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7752742' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 1373963c202323287783ccb099bbae1ec8f14a5d owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8815 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 250/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 1373963c202323287783ccb099bbae1ec8f14a5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIyS/bu3ntc2GwWSwg3AT7UDGyWC0EHzBn3SgaFxHwMoPQcbUjPW6FLC97eOpFyp+RwA8mrwVCbBUrb52l0GW+0= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZEXves2DvAv+czzlUTbNN8DAbrAyRlrGTSnrahc5219K06/sdBCS0Go/Yq2maCW3fSYv1MlKqJ4bAb3fOwDSQ= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGQkMAfcWdq1qWThI/+ijitRJjewN4ILC/U/InPHZh4BhkJ4XNmxGlmIWetlZOBvALuDmvzzQ83A7hIkOWyxTC4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: c1b06c2b5d3a02a3583265a115f09e052855bcf2 timestamp: 2024-06-12_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-13T06:25:32.679 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa; will attempt to use it 2024-06-13T06:25:32.680 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks 2024-06-13T06:25:32.680 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-13T06:25:32.681 INFO:teuthology.task.internal:Checking packages... 2024-06-13T06:25:32.702 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-13T06:25:32.702 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-13T06:25:32.702 INFO:teuthology.packaging:ref: None 2024-06-13T06:25:32.702 INFO:teuthology.packaging:tag: None 2024-06-13T06:25:32.702 INFO:teuthology.packaging:branch: squid 2024-06-13T06:25:32.702 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:25:32.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-13T06:25:33.012 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2998.g1373963c 2024-06-13T06:25:33.013 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-13T06:25:33.020 INFO:teuthology.task.internal:no buildpackages task found 2024-06-13T06:25:33.020 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-13T06:25:33.049 INFO:teuthology.task.internal:Saving configuration 2024-06-13T06:25:33.068 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-13T06:25:33.082 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-13T06:25:33.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:20:31.324315', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIyS/bu3ntc2GwWSwg3AT7UDGyWC0EHzBn3SgaFxHwMoPQcbUjPW6FLC97eOpFyp+RwA8mrwVCbBUrb52l0GW+0='} 2024-06-13T06:25:33.122 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:20:31.325487', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZEXves2DvAv+czzlUTbNN8DAbrAyRlrGTSnrahc5219K06/sdBCS0Go/Yq2maCW3fSYv1MlKqJ4bAb3fOwDSQ='} 2024-06-13T06:25:33.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-13 06:20:31.323003', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGQkMAfcWdq1qWThI/+ijitRJjewN4ILC/U/InPHZh4BhkJ4XNmxGlmIWetlZOBvALuDmvzzQ83A7hIkOWyxTC4='} 2024-06-13T06:25:33.145 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-13T06:25:33.151 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-13T06:25:33.151 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-13T06:25:33.151 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-13T06:25:33.151 INFO:teuthology.run_tasks:Running task console_log... 2024-06-13T06:25:33.257 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea6775fbe0>, signals=[15]) 2024-06-13T06:25:33.257 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-13T06:25:33.262 INFO:teuthology.task.internal:Opening connections... 2024-06-13T06:25:33.262 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:25:33.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:25:33.337 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-06-13T06:25:33.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:25:33.412 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-06-13T06:25:33.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:25:33.487 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-13T06:25:33.494 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-06-13T06:25:33.512 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-06-13T06:25:33.512 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-06-13T06:25:33.569 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-06-13T06:25:33.569 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:25:33.570 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:25:33.571 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-06-13T06:25:33.589 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-06-13T06:25:33.607 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-06-13T06:25:33.607 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-06-13T06:25:33.663 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-06-13T06:25:33.663 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-06-13T06:25:33.663 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:25:33.664 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:25:33.665 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-06-13T06:25:33.689 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-06-13T06:25:33.706 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-06-13T06:25:33.706 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-06-13T06:25:33.763 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-06-13T06:25:33.763 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-13T06:25:33.764 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-13T06:25:33.765 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-06-13T06:25:33.789 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-13T06:25:33.799 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-13T06:25:33.850 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-13T06:25:33.850 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:25:33.853 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:25:33.855 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-06-13T06:25:33.871 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-13T06:25:33.899 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-13T06:25:33.900 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:25:33.910 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:25:33.912 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-06-13T06:25:33.963 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-13T06:25:34.025 INFO:teuthology.run_tasks:Running task kernel... 2024-06-13T06:25:34.039 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-13T06:25:34.039 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-13T06:25:34.039 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-13T06:25:34.039 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:25:34.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:25:34.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-13T06:25:34.217 DEBUG:teuthology.task.kernel:sha1 for client.0 is ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:25:34.217 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "ff88c41504f3e4c4268de18904a6f347f1b3f2d6"... 2024-06-13T06:25:34.217 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-06-13T06:25:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-437.el9.x86_64 2024-06-13T06:25:34.235 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:25:34.235 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-13T06:25:34.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2024-06-13T06:25:39.357 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.1 MB 00:03 2024-06-13T06:25:42.875 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 20 MB 00:02 2024-06-13T06:25:48.557 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.3 MB 00:01 2024-06-13T06:25:50.366 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 17 kB 00:00 2024-06-13T06:25:51.644 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-06-13T06:25:57.041 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-06-13T06:25:59.247 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-06-13T06:25:59.259 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 1 Package 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 380 M 2024-06-13T06:25:59.260 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-06-13T06:25:59.419 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] linux-firmware-20240603-144.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/linux-firmware-20240603-144.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:27:44.679 INFO:teuthology.orchestra.run.smithi007.stdout:linux-firmware-20240603-144.el9.noarch.rpm 3.6 MB/s | 380 MB 01:45 2024-06-13T06:27:44.680 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:27:44.680 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.6 MB/s | 380 MB 01:45 2024-06-13T06:27:47.886 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-06-13T06:27:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-06-13T06:27:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-06-13T06:27:51.305 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-06-13T06:27:51.306 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-06-13T06:28:00.485 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-06-13T06:28:01.239 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-06-13T06:28:01.318 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:28:01.837 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:28:01.838 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-06-13T06:28:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-06-13T06:28:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:28:02.536 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-06-13T06:28:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: linux-firmware-20240603-144.el9.noarch 2024-06-13T06:28:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:28:02.537 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-06-13T06:28:02.828 INFO:teuthology.task.kernel:Downloading kernel ff88c41504f3e4c4268de18904a6f347f1b3f2d6 on client.0... 2024-06-13T06:28:02.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ff88c41504f3e4c4268de18904a6f347f1b3f2d6 2024-06-13T06:28:02.970 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/ 2024-06-13T06:28:02.970 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gff88c41504f3-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-13T06:28:02.973 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2024-06-13T06:28:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13 06:28:15 URL:https://3.chacra.ceph.com/r/kernel/testing/ff88c41504f3e4c4268de18904a6f347f1b3f2d6/centos/9/flavors/default/x86_64/kernel-6.9.0_gff88c41504f3-1.x86_64.rpm [356881057/356881057] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-13T06:28:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2024-06-13 06:28:15-- 2024-06-13T06:28:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 12s 2024-06-13T06:28:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 340M in 12s (28.2 MB/s) 2024-06-13T06:28:15.404 INFO:teuthology.task.kernel:Installing kernel ff88c41504f3e4c4268de18904a6f347f1b3f2d6 on client.0... 2024-06-13T06:28:15.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-13T06:28:17.542 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2024-06-13T06:28:17.589 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2024-06-13T06:28:17.643 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2024-06-13T06:28:25.902 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.9.0_gff88c41504f3-1 ######################################## 2024-06-13T06:29:06.592 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-13T06:29:06.593 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-13T06:29:06.671 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.9.0-gff88c41504f3 2024-06-13T06:29:06.671 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.9.0-gff88c41504f3 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.9.0-gff88c41504f3 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/System.map 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/config 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-13T06:29:06.672 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-13T06:29:06.673 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu/mce 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/arch/x86/kvm/kvm.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/ansi_cprng.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/anubis.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/arc4.ko 2024-06-13T06:29:06.674 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_memcpy.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_pq.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_tx.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/async_xor.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/async_tx/raid6test.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/authenc.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/authencesn.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blake2b_generic.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blowfish_common.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/blowfish_generic.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/camellia_generic.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast5_generic.ko 2024-06-13T06:29:06.675 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast6_generic.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cast_common.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/ccm.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cmac.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crc32_generic.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crypto_engine.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/crypto_user.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/cts.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/des_generic.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/dh_generic.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/echainiv.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/essiv.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/fcrypt.ko 2024-06-13T06:29:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/khazad.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/lz4.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/lz4hc.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/md4.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/michael_mic.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/pcbc.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/pcrypt.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/rmd160.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/seed.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/serpent_generic.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/tcrypt.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/tea.ko 2024-06-13T06:29:06.677 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/twofish_common.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/twofish_generic.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/vmac.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/wp512.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xcbc.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xor.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/crypto/xxhash_generic.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/acpi_pad.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/ec_sys.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/platform_profile.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/sbs.ko 2024-06-13T06:29:06.678 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/sbshc.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/acpi/video.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/acard-ahci.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/ahci_platform.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/ata_generic.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/libahci_platform.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_acpi.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ali.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_amd.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_artop.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_atiixp.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_atp867x.ko 2024-06-13T06:29:06.679 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cmd640.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cmd64x.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_cypress.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_efar.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt366.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt37x.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_it8213.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_it821x.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_jmicron.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_marvell.ko 2024-06-13T06:29:06.680 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_mpiix.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_netcell.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ninja32.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ns87410.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_ns87415.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_oldpiix.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_opti.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_optidma.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pcmcia.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_piccolo.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_rdc.ko 2024-06-13T06:29:06.681 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sch.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_serverworks.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sil680.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sis.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_sl82c105.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_triflex.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pata_via.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/pdc_adma.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_inic162x.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_mv.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_nv.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_promise.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_qstor.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sil.ko 2024-06-13T06:29:06.682 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sil24.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sis.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_svw.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_sx4.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_uli.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_via.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ata/sata_vsc.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/auxdisplay 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/auxdisplay/ks0108.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/bcma 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/bcma/bcma.ko 2024-06-13T06:29:06.683 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/aoe 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/aoe/aoe.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/brd.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/drbd 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/drbd/drbd.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/floppy.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/loop.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/mtip32xx 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/nbd.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/null_blk 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/null_blk/null_blk.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/pktcdvd.ko 2024-06-13T06:29:06.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/rbd.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/virtio_blk.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkback 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/xen-blkfront.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/zram 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/block/zram/zram.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hangcheck-timer.ko 2024-06-13T06:29:06.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/via-rng.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-13T06:29:06.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/lp.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/mwave 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/mwave/mwave.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/ppdev.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tlclk.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-13T06:29:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/uv_mmtimer.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/char/virtio_console.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/ccp/ccp.ko 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel 2024-06-13T06:29:06.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_common 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/padlock-aes.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/padlock-sha.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/virtio 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dca 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dca/dca.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/devfreq 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/dw 2024-06-13T06:29:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/ioat 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/amd64_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/e752x_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/edac_mce_amd.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i3000_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i3200_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i5100_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i5400_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i7300_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i7core_edac.ko 2024-06-13T06:29:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/i82975x_edac.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/ie31200_edac.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/sb_edac.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/edac/x38_edac.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware/edd.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio/gpio-ich.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-appleir.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-aureal.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-axff.ko 2024-06-13T06:29:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-dr.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-elecom.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-elo.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-emsff.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gaff.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gt683r.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-gyration.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-holtekff.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-hyperv.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-icade.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-keytouch.ko 2024-06-13T06:29:06.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-kye.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-lcpower.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-led.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-lenovo.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-multitouch.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-ortek.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-penmount.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-petalynx.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-picolcd.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-pl.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-plantronics.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-primax.ko 2024-06-13T06:29:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-rmi.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-common.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-13T06:29:06.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-roccat.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-saitek.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-samsung.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sjoy.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sony.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-speedlink.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-steelseries.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-sunplus.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-tivo.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-tmff.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-topseed.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-twinhan.ko 2024-06-13T06:29:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-uclogic.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-waltop.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-wiimote.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-xinmo.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-zpff.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/hid-zydacron.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/uhid.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hid/wacom.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_balloon.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_utils.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hv/hv_vmbus.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/abituguru.ko 2024-06-13T06:29:06.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/abituguru3.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ad7414.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ad7418.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adc128d818.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1025.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1026.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1029.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm1031.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adm9240.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ads7828.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7410.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7411.ko 2024-06-13T06:29:06.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7462.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7470.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7475.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/adt7x10.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/amc6821.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/applesmc.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asb100.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asc7621.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/atxp1.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/coretemp.ko 2024-06-13T06:29:06.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/dme1737.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ds1621.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ds620.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/emc1403.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/emc6w201.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f71805f.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f71882fg.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/f75375s.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/fam15h_power.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/fschmd.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/g760a.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/g762.ko 2024-06-13T06:29:06.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/gl518sm.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/gl520sm.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/i5500_temp.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/i5k_amb.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ibmaem.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ibmpex.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ina209.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ina2xx.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/it87.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/k10temp.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/k8temp.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lineage-pem.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm63.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm73.ko 2024-06-13T06:29:06.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm75.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm77.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm78.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm80.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm83.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm85.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm87.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm90.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm92.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm93.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95234.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95241.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/lm95245.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc2945.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4151.ko 2024-06-13T06:29:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4215.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4222.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4245.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4260.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/ltc4261.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max16065.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max1619.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max1668.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max197.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6639.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6650.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/max6697.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/mcp3021.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6683.ko 2024-06-13T06:29:06.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6775-core.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct6775.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/nct7802.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pc87360.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pc87427.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pcf8591.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-13T06:29:06.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/powr1220.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch5627.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch5636.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sht15.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sht21.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/shtc1.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/sis5595.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47b397.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47m1.ko 2024-06-13T06:29:06.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/smsc47m192.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/thmc50.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp102.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp103.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp401.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/tmp421.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/via-cputemp.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/via686a.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/vt1211.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/vt8231.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83627ehf.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83627hf.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83781d.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83791d.ko 2024-06-13T06:29:06.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83792d.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83793.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83795.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83l785ts.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/hwmon/w83l786ng.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-13T06:29:06.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-13T06:29:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-dev.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-mux.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-smbus.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/i2c/i2c-stub.ko 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core 2024-06-13T06:29:06.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_core.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/cxgb4 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx4 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx5 2024-06-13T06:29:06.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mthca 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/ipoib 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/iser 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/isert 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srp 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srpt 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-13T06:29:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/gameport/gameport.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/joydev.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/keyboard 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/appletouch.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/bcm5974.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/cyapatp.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/sermouse.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-13T06:29:06.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/rmi4 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/altera_ps2.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/arc_ps2.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/serio/serio_raw.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/input/sparse-keymap.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-blinkm.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-lm3530.ko 2024-06-13T06:29:06.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-lp3944.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/leds-ss4200.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md 2024-06-13T06:29:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/bcache 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/bcache/bcache.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-bio-prison.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-cache-smq.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-cache.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-crypt.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-delay.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-flakey.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-log-userspace.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-multipath.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-queue-length.ko 2024-06-13T06:29:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-raid.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-round-robin.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-service-time.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-switch.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-thin-pool.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/dm-verity.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/persistent-data 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid0.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid1.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid10.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/md/raid456.ko 2024-06-13T06:29:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptbase.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptctl.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptfc.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptlan.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptsas.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptscsih.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/message/fusion/mptspi.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/lpc_ich.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/lpc_sch.ko 2024-06-13T06:29:06.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/mfd-core.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/sm501.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/viperboard.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/vx855.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/mfd/wl1273-core.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/altera-stapl 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/apds9802als.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/apds990x.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/bh1770glc.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/cb710 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/cb710/cb710.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/at24.ko 2024-06-13T06:29:06.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/eeprom/max6875.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/enclosure.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/hpilo.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/isl29003.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/isl29020.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei-me.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei-txe.ko 2024-06-13T06:29:06.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/mei/mei.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-gru 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tifm_7xx1.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tifm_core.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/tsl2550.ko 2024-06-13T06:29:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_balloon.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_vmci 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/bonding 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/bonding/bonding.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/b53 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6xxx 2024-06-13T06:29:06.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/dummy.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/eql.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/adaptec 2024-06-13T06:29:06.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/agere 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/alteon 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/altera 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/alx 2024-06-13T06:29:06.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-13T06:29:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/genet 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade/bna 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-13T06:29:06.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco/enic 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink 2024-06-13T06:29:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/dnet.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex/benet 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ethoc.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/fealnx.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000e 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/fm10k 2024-06-13T06:29:06.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/i40e 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/iavf 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igb 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igbvf 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/jme.ko 2024-06-13T06:29:06.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/micrel 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom 2024-06-13T06:29:06.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/neterion 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/nvidia 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-13T06:29:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rdc 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rocker 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sfc 2024-06-13T06:29:06.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/silan 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-13T06:29:06.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/tehuti 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ti 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-13T06:29:06.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/xircom 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/geneve.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/hyperv 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ieee802154 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ifb.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ipvlan 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/macvlan.ko 2024-06-13T06:29:06.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/macvtap.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/mii.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/net_failover.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/netconsole.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/nlmon.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy 2024-06-13T06:29:06.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/amd.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/bcm87xx.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/broadcom.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/cicada.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/davicom.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/dp83640.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/et1011c.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/icplus.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/lxt.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/marvell.ko 2024-06-13T06:29:06.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/micrel.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/national.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/phylink.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/qsemi.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/realtek.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/smsc.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/ste10Xp.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/phy/vitesse.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_async.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-13T06:29:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pppoe.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pppox.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/ppp/pptp.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip/slhc.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/slip/slip.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/sungem_phy.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/tap.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_random.ko 2024-06-13T06:29:06.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/tun.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/veth.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/virtio_net.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vmxnet3 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vxlan 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/vxlan/vxlan.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netback 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/net/xen-netfront.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ntb 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ntb/ntb.ko 2024-06-13T06:29:06.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host/nvme-core.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/nvme/host/nvme.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_cs.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_pc.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/parport/parport_serial.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/hotplug 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/pcie 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-13T06:29:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pci/xen-pcifront.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/i82092.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/pd6729.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl/intel 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/acerhdf.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-13T06:29:06.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/hdaps.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/intel_ips.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-13T06:29:06.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/platform/x86/wmi.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-13T06:29:06.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/clients/pps_parport.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/pps/pps_core.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp/ptp.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ptp/ptp_kvm.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/atl 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ras/amd/fmpm.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-13T06:29:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-em3027.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-13T06:29:06.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-max6900.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-13T06:29:06.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/rtc/rtc-x1205.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-9xxx.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-sas.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/3w-xxxx.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/BusLogic.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/a100u2w.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aacraid 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/advansys.ko 2024-06-13T06:29:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/am53c974.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/arcmsr 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/atp870u.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/be2iscsi 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bfa 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bfa/bfa.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2fc 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2i 2024-06-13T06:29:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ch.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/csiostor 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/dc395x.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-13T06:29:06.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/dmx3191d.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esas2r 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/esp_scsi.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fnic 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/fnic/fnic.ko 2024-06-13T06:29:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hpsa.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hptiop.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/hv_storvsc.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/initio.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ipr.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ips.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/isci 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/isci/isci.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libfc 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libfc/libfc.ko 2024-06-13T06:29:06.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libiscsi.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libsas 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/libsas/libsas.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/lpfc 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-13T06:29:06.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mpt3sas 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvsas 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/mvumi.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pm8001 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/pmcraid.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla1280.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-13T06:29:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla4xxx 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/raid_class.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_debug.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/ses.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/st.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/stex.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/sym53c8xx_2 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/virtio_scsi.ko 2024-06-13T06:29:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/scsi/wd719x.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ssb 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ssb/ssb.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/iscsi 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/loopback 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_file.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_iblock.ko 2024-06-13T06:29:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_mod.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_pscsi.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/target_core_user.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/tcm_fc 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-13T06:29:06.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-13T06:29:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/ipwireless 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/n_gsm.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/n_hdlc.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/nozomi.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/8250 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/arc_uart.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/jsm 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs 2024-06-13T06:29:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/core 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/host 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_aec.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_cif.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_pci_generic.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/uio/uio_sercos3.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/cdc-acm.ko 2024-06-13T06:29:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/usblp.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/class/usbtmc.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/host 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image/mdc800.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/image/microtek.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/adutux.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/appledisplay.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/emi26.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/emi62.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/ezusb.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/idmouse.ko 2024-06-13T06:29:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/iowarrior.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/ldusb.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/legousbtower.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/sisusbvga 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usb3503.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usblcd.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/uss720.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/misc/yurex.ko 2024-06-13T06:29:06.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/phy 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/phy/phy-generic.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/aircable.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ark3116.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ch341.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cp210x.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cyberjack.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-13T06:29:06.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/empeg.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/io_ti.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ipaq.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ipw.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ir-usb.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/keyspan.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-13T06:29:06.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mct_u232.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mos7720.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/mos7840.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/navman.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/omninet.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/opticon.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/option.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/oti6858.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/pl2303.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/qcaux.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/qcserial.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/quatech2.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/safe_serial.ko 2024-06-13T06:29:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/sierra.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ssu100.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/symbolserial.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb_debug.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/visor.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/whiteheat.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/uas.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-13T06:29:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-karma.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/storage/usb-storage.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-13T06:29:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/vfio.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_net.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/vhost/vhost_scsi.ko 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video 2024-06-13T06:29:06.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/apple_bl.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/lcd.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_balloon.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_mmio.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci.ko 2024-06-13T06:29:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/virtio/virtio_ring.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters/ds2482.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/masters/ds2490.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-13T06:29:06.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/w1/wire.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/hpwdt.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/i6300esb.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-13T06:29:06.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ib700wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ibmasr.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/it87_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/machzwd.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/nv_tco.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/softdog.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/via_wdt.ko 2024-06-13T06:29:06.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/wdt_pci.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/watchdog/xen_wdt.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-evtchn.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-gntalloc.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-gntdev.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-pciback 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xen-privcmd.ko 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xenfs 2024-06-13T06:29:06.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/9p 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/9p/9p.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/affs 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/affs/affs.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/befs 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/befs/befs.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/binfmt_misc.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/btrfs 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/btrfs/btrfs.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cachefiles 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cachefiles/cachefiles.ko 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ceph 2024-06-13T06:29:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ceph/ceph.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/coda 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/coda/coda.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cramfs 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/cramfs/cramfs.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/dlm 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/dlm/dlm.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ecryptfs 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/f2fs 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/f2fs/f2fs.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/fat.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/msdos.ko 2024-06-13T06:29:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fat/vfat.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse/cuse.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/fuse/fuse.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/gfs2 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/gfs2/gfs2.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfs 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfs/hfs.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfsplus 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/hfsplus/hfsplus.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/isofs 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/isofs/isofs.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/jfs 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/jfs/jfs.ko 2024-06-13T06:29:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/lockd 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/lockd/lockd.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/minix 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/minix/minix.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/netfs 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/netfs/netfs.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/blocklayout 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/filelayout 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/flexfilelayout 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfs.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfsv3.ko 2024-06-13T06:29:06.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs/nfsv4.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common/grace.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfs_common/nfs_acl.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfsd 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nfsd/nfsd.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nilfs2 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nilfs2/nilfs2.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-celtic.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-centeuro.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-croatian.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-cyrillic.ko 2024-06-13T06:29:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-gaelic.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-greek.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-iceland.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-inuit.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-roman.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-romanian.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/mac-turkish.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1250.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1251.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp1255.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp737.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp775.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp850.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp852.ko 2024-06-13T06:29:06.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp855.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp857.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp860.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp861.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp862.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp863.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp864.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp865.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp866.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp869.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp874.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp932.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp936.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp949.ko 2024-06-13T06:29:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_cp950.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_euc-jp.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-1.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-13.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-14.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-15.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-2.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-3.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-4.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-5.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-6.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-7.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_iso8859-9.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-r.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-ru.ko 2024-06-13T06:29:06.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_koi8-u.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/nls/nls_utf8.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/cluster 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlm 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlmfs 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-13T06:29:06.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/overlayfs 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/overlayfs/overlay.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/pstore 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/pstore/ramoops.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/reiserfs 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/reiserfs/reiserfs.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/romfs 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/romfs/romfs.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/client 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/client/cifs.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common/cifs_arc4.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/smb/common/cifs_md4.ko 2024-06-13T06:29:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/squashfs 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/squashfs/squashfs.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/sysv 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/sysv/sysv.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/udf 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/udf/udf.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ufs 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/ufs/ufs.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/xfs 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/fs/xfs/xfs.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/rcu 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/rcu/rcutorture.ko 2024-06-13T06:29:06.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/torture.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/trace 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/asn1_encoder.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crc-itu-t.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crc8.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto/libarc4.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/crypto/libdes.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/libcrc32c.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lru_cache.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4/lz4_compress.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/lz4/lz4hc_compress.ko 2024-06-13T06:29:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/math 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/math/cordic.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/raid6 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/raid6/raid6_pq.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/reed_solomon 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_bm.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_fsm.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/ts_kmp.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/zstd 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/lib/zstd/zstd_compress.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/mm 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/mm/hwpoison-inject.ko 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net 2024-06-13T06:29:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/6lowpan.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_dest.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_fragment.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_hop.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_mobility.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_routing.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/6lowpan/nhc_udp.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/garp.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/mrp.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/p8022.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/psnap.ko 2024-06-13T06:29:06.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/802/stp.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/8021q 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/8021q/8021q.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_fd.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_rdma.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/9p/9pnet_virtio.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/batman-adv 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/batman-adv/batman-adv.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/br_netfilter.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/bridge.ko 2024-06-13T06:29:06.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-13T06:29:06.787 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/ebtables.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ceph 2024-06-13T06:29:06.788 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ceph/libceph.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core/failover.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/core/pktgen.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_diag.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_ipv4.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dccp/dccp_ipv6.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dns_resolver 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dns_resolver/dns_resolver.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/dsa_core.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_brcm.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_dsa.ko 2024-06-13T06:29:06.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_none.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/dsa/tag_trailer.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/6lowpan 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/ieee802154.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ah4.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/esp4.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/fou.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/gre.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/inet_diag.ko 2024-06-13T06:29:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_gre.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_tunnel.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ip_vti.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ipcomp.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/ipip.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-13T06:29:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_bic.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_dctcp.ko 2024-06-13T06:29:06.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_diag.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_highspeed.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_htcp.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_hybla.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_illinois.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_lp.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_scalable.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_vegas.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_veno.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_westwood.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tcp_yeah.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/tunnel4.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/udp_diag.ko 2024-06-13T06:29:06.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/udp_tunnel.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ah6.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/esp6.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/fou6.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_tunnel.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ip6_vti.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/ipcomp6.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-13T06:29:06.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-13T06:29:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/sit.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/tunnel6.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/key 2024-06-13T06:29:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/key/af_key.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_core.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_eth.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ip.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ip6.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_netlink.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/l2tp/l2tp_ppp.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/llc 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/llc/llc.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mac802154 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mac802154/mac802154.ko 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mpls 2024-06-13T06:29:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/mpls/mpls_gso.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-13T06:29:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-13T06:29:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conncount.ko 2024-06-13T06:29:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_flow_table.ko 2024-06-13T06:29:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_log_syslog.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_irc.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_sip.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nf_tables.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_log.ko 2024-06-13T06:29:06.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_chain_nat.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_compat.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_ct.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_fib.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_hash.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_limit.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_log.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_masq.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_nat.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_numgen.ko 2024-06-13T06:29:06.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_queue.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_quota.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_redir.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_reject.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_reject_inet.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_socket.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/nft_tunnel.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_AUDIT.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_CT.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_DSCP.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_HL.ko 2024-06-13T06:29:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_HMARK.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_LED.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_LOG.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NETMAP.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NFLOG.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_RATEEST.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_SECMARK.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TEE.ko 2024-06-13T06:29:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TPROXY.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_TRACE.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_addrtype.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_bpf.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cgroup.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cluster.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_comment.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connbytes.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connlabel.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connlimit.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_connmark.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_conntrack.ko 2024-06-13T06:29:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_cpu.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_dccp.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_devgroup.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_dscp.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ecn.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_esp.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_hashlimit.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_helper.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_hl.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ipcomp.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_iprange.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_ipvs.ko 2024-06-13T06:29:06.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_l2tp.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_length.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_limit.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_mac.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_mark.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_multiport.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_nat.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_nfacct.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_osf.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_owner.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_physdev.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_pkttype.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_policy.ko 2024-06-13T06:29:06.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_quota.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_rateest.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_realm.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_recent.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_sctp.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_set.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_socket.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_state.ko 2024-06-13T06:29:06.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_statistic.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_string.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_tcpmss.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_time.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netfilter/xt_u32.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netlink 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/netlink/netlink_diag.ko 2024-06-13T06:29:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/nsh 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/nsh/nsh.ko 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/openvswitch.ko 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-geneve.ko 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-gre.ko 2024-06-13T06:29:06.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/openvswitch/vport-vxlan.ko 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/packet 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/packet/af_packet_diag.ko 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds.ko 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds_rdma.ko 2024-06-13T06:29:06.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/rds/rds_tcp.ko 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_csum.ko 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_gact.ko 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_mirred.ko 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_nat.ko 2024-06-13T06:29:06.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_pedit.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_police.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_simple.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_skbedit.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/act_vlan.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_basic.ko 2024-06-13T06:29:06.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_bpf.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_flow.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_fw.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_route.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/cls_u32.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_cmp.ko 2024-06-13T06:29:06.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_ipset.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_meta.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_nbyte.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_text.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/em_u32.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_choke.ko 2024-06-13T06:29:06.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_codel.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_drr.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_fq.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_gred.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_hfsc.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_hhf.ko 2024-06-13T06:29:06.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_htb.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_ingress.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_mqprio.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_mqprio_lib.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_multiq.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_netem.ko 2024-06-13T06:29:06.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_pie.ko 2024-06-13T06:29:06.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_plug.ko 2024-06-13T06:29:06.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_prio.ko 2024-06-13T06:29:06.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_qfq.ko 2024-06-13T06:29:06.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_red.ko 2024-06-13T06:29:06.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_sfb.ko 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_sfq.ko 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_tbf.ko 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sched/sch_teql.ko 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp/sctp.ko 2024-06-13T06:29:06.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sctp/sctp_diag.ko 2024-06-13T06:29:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc 2024-06-13T06:29:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss 2024-06-13T06:29:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-13T06:29:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-13T06:29:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/sunrpc.ko 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/xprtrdma 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc/tipc.ko 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/tipc/tipc_diag.ko 2024-06-13T06:29:06.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/unix 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/unix/unix_diag.ko 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock.ko 2024-06-13T06:29:06.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/xfrm 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys 2024-06-13T06:29:06.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/encrypted-keys 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/trusted-keys 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/kernel/security/keys/trusted-keys/trusted.ko 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.alias 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.alias.bin 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin 2024-06-13T06:29:06.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.alias.bin 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.bin 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.builtin.modinfo 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.dep 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.dep.bin 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.devname 2024-06-13T06:29:06.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.order 2024-06-13T06:29:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.softdep 2024-06-13T06:29:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.symbols 2024-06-13T06:29:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/modules.symbols.bin 2024-06-13T06:29:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-gff88c41504f3/vmlinuz 2024-06-13T06:29:06.833 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gff88c41504f3 2024-06-13T06:29:06.834 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/bin/kernel-install add 6.9.0-gff88c41504f3 /lib/modules/6.9.0-gff88c41504f3/vmlinuz 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout:for file in vmlinuz System.map config; do 2024-06-13T06:29:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: if ! cmp --silent "/lib/modules/6.9.0-gff88c41504f3/${file}" "/boot/${file}-6.9.0-gff88c41504f3"; then 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: cp "/lib/modules/6.9.0-gff88c41504f3/${file}" "/boot/${file}-6.9.0-gff88c41504f3" 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: fi 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -d "/lib/modules/6.9.0-gff88c41504f3/dtb" ] && \ 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: ! diff -rq "/lib/modules/6.9.0-gff88c41504f3/dtb" "/boot/dtb-6.9.0-gff88c41504f3" >/dev/null 2>&1; then 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: rm -rf "/boot/dtb-6.9.0-gff88c41504f3" 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: cp -r "/lib/modules/6.9.0-gff88c41504f3/dtb" "/boot/dtb-6.9.0-gff88c41504f3" 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:if [ ! -e "/lib/modules/6.9.0-gff88c41504f3/modules.dep" ]; then 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/sbin/depmod 6.9.0-gff88c41504f3 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.9.0-gff88c41504f3 2024-06-13T06:29:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-06-13T06:29:06.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-06-13T06:29:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:29:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-06-13T06:29:06.936 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-13T06:29:06.936 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.9.0-gff88c41504f3 2024-06-13T06:29:06.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-13T06:29:07.492 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-06-13T06:29:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-13T06:29:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-06-13T06:29:08.876 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-06-13T06:29:08.897 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.L2fyH57cWR 2024-06-13T06:29:08.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.L2fyH57cWR 2024-06-13T06:29:08.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.L2fyH57cWR 2024-06-13T06:29:09.107 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.L2fyH57cWR is 6KB 2024-06-13T06:29:09.156 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.L2fyH57cWR 2024-06-13T06:29:09.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-06-13T06:29:09.232 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-13T06:29:09.232 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-13T06:29:09.232 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-13T06:29:09.233 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gff88c41504f3.conf 2024-06-13T06:29:09.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gff88c41504f3 2024-06-13T06:29:09.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-06-13T06:29:39.389 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-13T06:29:39.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:29:39.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:29:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-06-13T06:30:06.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:30:06.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:31:06.923 DEBUG:teuthology.orchestra.remote:timed out 2024-06-13T06:31:18.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:31:18.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:31:19.270 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-06-13T06:31:19.888 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:31:19.888 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gff88c41504f3"... 2024-06-13T06:31:19.888 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-06-13T06:31:19.939 INFO:teuthology.orchestra.run.smithi007.stdout:6.9.0-gff88c41504f3 2024-06-13T06:31:19.939 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gff88c41504f3 vs 6.9.0-gff88c41504f3 2024-06-13T06:31:19.939 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-13T06:31:19.940 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-13T06:31:20.940 DEBUG:teuthology.parallel:result is None 2024-06-13T06:31:20.941 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-13T06:31:20.950 INFO:teuthology.task.internal:Creating test directory... 2024-06-13T06:31:20.950 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:31:20.958 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:31:20.961 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-13T06:31:21.008 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-13T06:31:21.014 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-13T06:31:21.053 INFO:teuthology.task.internal:Creating archive directory... 2024-06-13T06:31:21.054 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:31:21.061 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:31:21.064 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-13T06:31:21.144 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-13T06:31:21.151 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-13T06:31:21.151 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:31:21.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:21.168 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-13T06:31:21.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:21.185 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:31:21.192 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:31:21.211 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-13T06:31:21.241 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.260 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.271 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.294 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.320 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.384 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-13T06:31:21.389 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-13T06:31:21.396 INFO:teuthology.task.internal:Configuring sudo... 2024-06-13T06:31:21.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:31:21.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:31:21.404 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-13T06:31:21.486 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-13T06:31:21.495 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-13T06:31:21.496 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:31:21.501 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:31:21.503 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-13T06:31:21.552 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:21.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:21.737 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:21.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:21.913 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:31:21.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:31:21.998 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:22.042 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:22.093 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:22.178 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:22.217 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:31:22.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:31:22.282 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:22.320 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-13T06:31:22.386 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:22.470 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-13T06:31:22.511 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:31:22.511 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-13T06:31:22.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-06-13T06:31:22.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-06-13T06:31:22.586 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-06-13T06:31:22.623 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:31:22.650 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:31:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T06:31:23.123 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-13T06:31:23.132 INFO:teuthology.task.internal:Starting timer... 2024-06-13T06:31:23.132 INFO:teuthology.run_tasks:Running task pcp... 2024-06-13T06:31:23.174 INFO:teuthology.run_tasks:Running task selinux... 2024-06-13T06:31:23.212 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-13T06:31:23.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-06-13T06:31:23.317 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-06-13T06:31:23.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-06-13T06:31:23.383 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-06-13T06:31:23.385 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-06-13T06:31:23.455 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-06-13T06:31:23.457 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-13T06:31:23.457 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-06-13T06:31:23.504 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-06-13T06:31:23.505 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-06-13T06:31:23.531 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-06-13T06:31:23.531 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-06-13T06:31:23.555 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-06-13T06:31:23.555 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-06-13T06:31:23.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:31:23.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:23.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:31:23.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:23.672 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T06:31:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T06:31:23.700 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-13T06:31:23.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:31:23.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:31:23.805 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-06-13T06:31:23.830 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-13T06:31:23.840 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-13T06:31:23.843 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-13T06:31:23.981 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-13T06:31:23.992 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-13T06:31:23.993 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi175.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-13T06:37:38.410 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-06-13T06:37:38.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:37:38.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:37:38.527 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-06-13T06:37:38.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-06-13T06:37:38.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-06-13T06:37:38.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:37:38.775 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-06-13T06:37:38.864 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-06-13T06:37:38.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-06-13T06:37:38.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-13T06:37:38.947 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-06-13T06:37:39.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-06-13T06:37:39.033 INFO:teuthology.run_tasks:Running task clock... 2024-06-13T06:37:39.046 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-13T06:37:39.046 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:37:39.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:37:39.054 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:37:39.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:37:39.057 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-13T06:37:39.057 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T06:37:39.087 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:37:39.101 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:37:39.103 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:37:39.117 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:37:39.126 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-06-13T06:37:39.137 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-06-13T06:37:39.142 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-06-13T06:37:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-13T06:37:39.151 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:37:39.153 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-06-13T06:37:39.164 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:37:39.166 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:37:39.180 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:37:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-13T06:37:39.220 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.223 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.237 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.241 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.269 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-06-13T06:37:39.299 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-06-13T06:37:39.342 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-13T06:37:39.387 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-13T06:37:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.534 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T06:37:39.535 INFO:teuthology.run_tasks:Running task install... 2024-06-13T06:37:39.544 DEBUG:teuthology.task.install:project ceph 2024-06-13T06:37:39.544 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-13T06:37:39.544 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-13T06:37:39.544 INFO:teuthology.task.install:Using flavor: default 2024-06-13T06:37:39.549 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-13T06:37:39.550 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-13T06:37:39.550 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:37:39.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:37:39.552 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:37:39.553 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:37:39.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-13T06:37:39.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:37:39.699 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:37:39.699 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:37:39.784 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:37:39.784 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:37:39.788 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-13T06:37:39.788 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-13T06:37:39.817 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:37:39.817 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:37:39.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:37:39.845 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:37:39.845 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:37:39.896 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:37:39.896 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:37:39.896 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:37:39.912 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:37:39.922 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:37:39.922 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:37:39.958 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-13T06:37:39.958 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:37:39.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-13T06:37:39.989 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:37:39.990 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:37:40.054 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-06-13T06:37:40.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-06-13T06:37:40.057 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-13T06:37:40.058 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-13T06:37:40.063 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:37:40.127 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-06-13T06:37:40.128 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-06-13T06:37:40.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-13T06:37:40.281 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-06-13T06:37:40.304 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-13T06:37:40.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:37:40.352 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-06-13T06:37:40.379 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:37:40.390 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-06-13T06:37:40.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-06-13T06:37:40.723 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-06-13T06:37:40.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-13T06:37:41.022 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 218 kB/s | 83 kB 00:00 2024-06-13T06:37:41.035 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-06-13T06:37:41.381 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-06-13T06:37:41.402 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-06-13T06:37:41.568 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 232 kB/s | 83 kB 00:00 2024-06-13T06:37:41.690 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-06-13T06:37:41.811 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-06-13T06:37:42.052 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-13T06:37:42.628 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-06-13T06:37:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.1 MB 00:01 2024-06-13T06:37:43.710 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-06-13T06:37:45.522 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-06-13T06:37:47.165 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 20 MB 00:02 2024-06-13T06:37:51.890 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 20 MB 00:05 2024-06-13T06:37:54.207 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.3 MB 00:03 2024-06-13T06:37:55.839 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 17 kB 00:00 2024-06-13T06:37:56.984 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-06-13T06:37:58.773 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.3 MB 00:02 2024-06-13T06:38:00.434 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 20 MB 00:15 2024-06-13T06:38:00.923 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 17 kB 00:00 2024-06-13T06:38:02.028 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-06-13T06:38:02.189 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-06-13T06:38:04.483 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:04.484 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:04.495 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:38:04.496 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:38:04.497 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:38:04.498 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:38:04.499 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:38:04.502 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:38:04.502 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:38:04.565 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-06-13T06:38:04.579 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-13T06:38:04.579 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-06-13T06:38:04.579 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:38:04.580 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:38:04.581 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:38:04.582 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:38:04.583 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:38:04.584 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:38:04.585 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:38:04.586 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:38:04.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:38:04.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:38:04.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:38:04.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:38:04.591 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 12 Packages 2024-06-13T06:38:04.592 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:04.593 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 226 M 2024-06-13T06:38:04.593 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-06-13T06:38:04.992 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.3 MB 00:00 2024-06-13T06:38:05.407 INFO:teuthology.orchestra.run.smithi175.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-06-13T06:38:05.749 INFO:teuthology.orchestra.run.smithi175.stdout:(2/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 9.3 MB/s | 5.4 MB 00:00 2024-06-13T06:38:05.807 INFO:teuthology.orchestra.run.smithi175.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 2.4 MB/s | 145 kB 00:00 2024-06-13T06:38:06.024 INFO:teuthology.orchestra.run.smithi175.stdout:(4/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 2.4 MB 00:00 2024-06-13T06:38:06.100 INFO:teuthology.orchestra.run.smithi175.stdout:(5/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 890 kB 00:00 2024-06-13T06:38:06.167 INFO:teuthology.orchestra.run.smithi175.stdout:(6/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 1.5 MB 00:00 2024-06-13T06:38:06.750 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 17 kB 00:00 2024-06-13T06:38:07.434 INFO:teuthology.orchestra.run.smithi175.stdout:(7/160): ceph-common-19.0.0-2946.g3d89accf.el9. 9.3 MB/s | 21 MB 00:02 2024-06-13T06:38:07.650 INFO:teuthology.orchestra.run.smithi175.stdout:(8/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 17 MB 00:01 2024-06-13T06:38:07.726 INFO:teuthology.orchestra.run.smithi175.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 337 kB/s | 25 kB 00:00 2024-06-13T06:38:07.959 INFO:teuthology.orchestra.run.smithi175.stdout:(10/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x8 2.5 MB/s | 4.7 MB 00:01 2024-06-13T06:38:07.962 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-13T06:38:08.068 INFO:teuthology.orchestra.run.smithi175.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.0 MB/s | 221 kB 00:00 2024-06-13T06:38:08.176 INFO:teuthology.orchestra.run.smithi175.stdout:(12/160): cephfs-shell-19.0.0-2946.g3d89accf.el 250 kB/s | 27 kB 00:00 2024-06-13T06:38:08.277 INFO:teuthology.orchestra.run.smithi175.stdout:(13/160): libcephfs-devel-19.0.0-2946.g3d89accf 318 kB/s | 32 kB 00:00 2024-06-13T06:38:08.385 INFO:teuthology.orchestra.run.smithi175.stdout:(14/160): ceph-radosgw-19.0.0-2946.g3d89accf.el 11 MB/s | 10 MB 00:00 2024-06-13T06:38:08.461 INFO:teuthology.orchestra.run.smithi175.stdout:(15/160): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-13T06:38:08.494 INFO:teuthology.orchestra.run.smithi175.stdout:(16/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.4 MB/s | 747 kB 00:00 2024-06-13T06:38:08.513 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-06-13T06:38:08.544 INFO:teuthology.orchestra.run.smithi175.stdout:(17/160): librados-devel-19.0.0-2946.g3d89accf. 1.5 MB/s | 127 kB 00:00 2024-06-13T06:38:08.670 INFO:teuthology.orchestra.run.smithi175.stdout:(18/160): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-13T06:38:08.770 INFO:teuthology.orchestra.run.smithi175.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 451 kB/s | 45 kB 00:00 2024-06-13T06:38:08.887 INFO:teuthology.orchestra.run.smithi175.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.1 MB/s | 135 kB 00:00 2024-06-13T06:38:09.004 INFO:teuthology.orchestra.run.smithi175.stdout:(21/160): python3-cephfs-19.0.0-2946.g3d89accf. 1.4 MB/s | 162 kB 00:00 2024-06-13T06:38:09.104 INFO:teuthology.orchestra.run.smithi175.stdout:(22/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 9.5 MB/s | 5.3 MB 00:00 2024-06-13T06:38:09.129 INFO:teuthology.orchestra.run.smithi175.stdout:(23/160): python3-rados-19.0.0-2946.g3d89accf.e 2.5 MB/s | 321 kB 00:00 2024-06-13T06:38:09.188 INFO:teuthology.orchestra.run.smithi175.stdout:(24/160): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-13T06:38:09.221 INFO:teuthology.orchestra.run.smithi175.stdout:(25/160): python3-rgw-19.0.0-2946.g3d89accf.el9 1.1 MB/s | 99 kB 00:00 2024-06-13T06:38:09.272 INFO:teuthology.orchestra.run.smithi175.stdout:(26/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-13T06:38:09.374 INFO:teuthology.orchestra.run.smithi175.stdout:(27/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 171 kB 00:00 2024-06-13T06:38:09.464 INFO:teuthology.orchestra.run.smithi175.stdout:(28/160): ceph-grafana-dashboards-19.0.0-2946.g 346 kB/s | 32 kB 00:00 2024-06-13T06:38:09.564 INFO:teuthology.orchestra.run.smithi175.stdout:(29/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.4 MB/s | 142 kB 00:00 2024-06-13T06:38:09.839 INFO:teuthology.orchestra.run.smithi175.stdout:(30/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.0 MB/s | 3.1 MB 00:00 2024-06-13T06:38:09.998 INFO:teuthology.orchestra.run.smithi175.stdout:(31/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 8.1 MB/s | 3.5 MB 00:00 2024-06-13T06:38:10.107 INFO:teuthology.orchestra.run.smithi175.stdout:(32/160): ceph-mgr-modules-core-19.0.0-2946.g3d 2.2 MB/s | 249 kB 00:00 2024-06-13T06:38:10.207 INFO:teuthology.orchestra.run.smithi175.stdout:(33/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 495 kB/s | 49 kB 00:00 2024-06-13T06:38:10.283 INFO:teuthology.orchestra.run.smithi175.stdout:(34/160): ceph-prometheus-alerts-19.0.0-2946.g3 219 kB/s | 16 kB 00:00 2024-06-13T06:38:10.700 INFO:teuthology.orchestra.run.smithi175.stdout:(35/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 16 MB/s | 48 MB 00:02 2024-06-13T06:38:10.724 INFO:teuthology.orchestra.run.smithi175.stdout:(36/160): ceph-volume-19.0.0-2946.g3d89accf.el9 597 kB/s | 264 kB 00:00 2024-06-13T06:38:10.775 INFO:teuthology.orchestra.run.smithi175.stdout:(37/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 10 MB/s | 763 kB 00:00 2024-06-13T06:38:10.800 INFO:teuthology.orchestra.run.smithi175.stdout:(38/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 242 kB/s | 18 kB 00:00 2024-06-13T06:38:10.864 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:10.872 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:11.109 INFO:teuthology.orchestra.run.smithi175.stdout:(39/160): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-06-13T06:38:11.110 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:11.212 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:11.213 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:11.231 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:38:11.232 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:38:11.234 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:38:11.235 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:38:11.237 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:38:11.240 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:38:11.241 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:38:11.251 INFO:teuthology.orchestra.run.smithi175.stdout:(40/160): fuse-2.9.9-15.el9.x86_64.rpm 169 kB/s | 80 kB 00:00 2024-06-13T06:38:11.276 INFO:teuthology.orchestra.run.smithi175.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-06-13T06:38:11.319 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:38:11.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:38:11.343 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:38:11.344 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:38:11.345 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:38:11.346 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:38:11.347 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:38:11.348 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:38:11.349 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:38:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:38:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:38:11.352 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:38:11.353 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:38:11.355 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 12 Packages 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:38:11.356 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 226 M 2024-06-13T06:38:11.357 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-06-13T06:38:11.368 INFO:teuthology.orchestra.run.smithi175.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-06-13T06:38:11.468 INFO:teuthology.orchestra.run.smithi175.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-06-13T06:38:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 672 kB/s | 106 kB 00:00 2024-06-13T06:38:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-06-13T06:38:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 703 kB/s | 135 kB 00:00 2024-06-13T06:38:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:38:11.703 INFO:teuthology.orchestra.run.smithi175.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 711 kB/s | 125 kB 00:00 2024-06-13T06:38:11.703 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:11.745 INFO:teuthology.orchestra.run.smithi175.stdout:(48/160): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-06-13T06:38:11.774 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:12.023 INFO:teuthology.orchestra.run.smithi007.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-06-13T06:38:12.278 INFO:teuthology.orchestra.run.smithi175.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 211 kB/s | 107 kB 00:00 2024-06-13T06:38:12.323 INFO:teuthology.orchestra.run.smithi007.stdout:(2/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 890 kB 00:00 2024-06-13T06:38:12.407 INFO:teuthology.orchestra.run.smithi007.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 1.7 MB/s | 145 kB 00:00 2024-06-13T06:38:12.437 INFO:teuthology.orchestra.run.smithi175.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-06-13T06:38:12.454 INFO:teuthology.orchestra.run.smithi175.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 124 kB/s | 22 kB 00:00 2024-06-13T06:38:12.487 INFO:teuthology.orchestra.run.smithi175.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-06-13T06:38:12.562 INFO:teuthology.orchestra.run.smithi175.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-06-13T06:38:12.579 INFO:teuthology.orchestra.run.smithi175.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-06-13T06:38:12.674 INFO:teuthology.orchestra.run.smithi007.stdout:(4/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-06-13T06:38:12.757 INFO:teuthology.orchestra.run.smithi007.stdout:(5/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 6.8 MB/s | 2.4 MB 00:00 2024-06-13T06:38:12.796 INFO:teuthology.orchestra.run.smithi175.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-06-13T06:38:12.813 INFO:teuthology.orchestra.run.smithi175.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 73 kB/s | 18 kB 00:00 2024-06-13T06:38:12.830 INFO:teuthology.orchestra.run.smithi175.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 44 kB/s | 11 kB 00:00 2024-06-13T06:38:12.857 INFO:teuthology.orchestra.run.smithi007.stdout:(6/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-06-13T06:38:12.897 INFO:teuthology.orchestra.run.smithi175.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-06-13T06:38:12.922 INFO:teuthology.orchestra.run.smithi175.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-06-13T06:38:12.948 INFO:teuthology.orchestra.run.smithi175.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-06-13T06:38:13.015 INFO:teuthology.orchestra.run.smithi175.stdout:(61/160): libtool-2.4.6-45.el9.x86_64.rpm 4.8 MB/s | 582 kB 00:00 2024-06-13T06:38:13.032 INFO:teuthology.orchestra.run.smithi175.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-06-13T06:38:13.049 INFO:teuthology.orchestra.run.smithi175.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-06-13T06:38:13.132 INFO:teuthology.orchestra.run.smithi175.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-06-13T06:38:13.157 INFO:teuthology.orchestra.run.smithi175.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-06-13T06:38:13.224 INFO:teuthology.orchestra.run.smithi175.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 201 kB/s | 35 kB 00:00 2024-06-13T06:38:13.284 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-06-13T06:38:13.366 INFO:teuthology.orchestra.run.smithi007.stdout:(7/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 7.7 MB/s | 4.7 MB 00:00 2024-06-13T06:38:13.383 INFO:teuthology.orchestra.run.smithi175.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-06-13T06:38:13.525 INFO:teuthology.orchestra.run.smithi175.stdout:(68/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-06-13T06:38:13.567 INFO:teuthology.orchestra.run.smithi175.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-06-13T06:38:13.592 INFO:teuthology.orchestra.run.smithi175.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-06-13T06:38:13.625 INFO:teuthology.orchestra.run.smithi175.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-06-13T06:38:13.851 INFO:teuthology.orchestra.run.smithi175.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-06-13T06:38:13.876 INFO:teuthology.orchestra.run.smithi175.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 349 kB/s | 87 kB 00:00 2024-06-13T06:38:13.909 INFO:teuthology.orchestra.run.smithi175.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-06-13T06:38:14.043 INFO:teuthology.orchestra.run.smithi175.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 370 kB/s | 61 kB 00:00 2024-06-13T06:38:14.118 INFO:teuthology.orchestra.run.smithi175.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-06-13T06:38:14.144 INFO:teuthology.orchestra.run.smithi175.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-06-13T06:38:14.169 INFO:teuthology.orchestra.run.smithi175.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-06-13T06:38:14.219 INFO:teuthology.orchestra.run.smithi175.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-06-13T06:38:14.244 INFO:teuthology.orchestra.run.smithi175.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-06-13T06:38:14.270 INFO:teuthology.orchestra.run.smithi175.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-06-13T06:38:14.320 INFO:teuthology.orchestra.run.smithi175.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-06-13T06:38:14.458 INFO:teuthology.orchestra.run.smithi007.stdout:(8/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 10 MB/s | 17 MB 00:01 2024-06-13T06:38:14.495 INFO:teuthology.orchestra.run.smithi175.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-06-13T06:38:14.520 INFO:teuthology.orchestra.run.smithi175.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-06-13T06:38:14.525 INFO:teuthology.orchestra.run.smithi007.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 380 kB/s | 25 kB 00:00 2024-06-13T06:38:14.545 INFO:teuthology.orchestra.run.smithi175.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 987 kB/s | 222 kB 00:00 2024-06-13T06:38:14.596 INFO:teuthology.orchestra.run.smithi175.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 774 kB/s | 77 kB 00:00 2024-06-13T06:38:14.646 INFO:teuthology.orchestra.run.smithi175.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-06-13T06:38:14.671 INFO:teuthology.orchestra.run.smithi175.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-06-13T06:38:14.695 INFO:teuthology.orchestra.run.smithi007.stdout:(10/160): ceph-radosgw-19.0.0-2946.g3d89accf.el 7.7 MB/s | 10 MB 00:01 2024-06-13T06:38:14.696 INFO:teuthology.orchestra.run.smithi175.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-06-13T06:38:14.747 INFO:teuthology.orchestra.run.smithi175.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-06-13T06:38:14.767 INFO:teuthology.orchestra.run.smithi007.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.9 MB/s | 221 kB 00:00 2024-06-13T06:38:14.772 INFO:teuthology.orchestra.run.smithi175.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-06-13T06:38:14.797 INFO:teuthology.orchestra.run.smithi175.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-06-13T06:38:14.939 INFO:teuthology.orchestra.run.smithi175.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 250 kB/s | 42 kB 00:00 2024-06-13T06:38:14.973 INFO:teuthology.orchestra.run.smithi175.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 251 kB/s | 44 kB 00:00 2024-06-13T06:38:15.043 INFO:teuthology.orchestra.run.smithi007.stdout:(12/160): ceph-common-19.0.0-2946.g3d89accf.el9 6.3 MB/s | 21 MB 00:03 2024-06-13T06:38:15.068 INFO:teuthology.orchestra.run.smithi007.stdout:(13/160): cephfs-shell-19.0.0-2946.g3d89accf.el 90 kB/s | 27 kB 00:00 2024-06-13T06:38:15.101 INFO:teuthology.orchestra.run.smithi007.stdout:(14/160): libcephfs-devel-19.0.0-2946.g3d89accf 553 kB/s | 32 kB 00:00 2024-06-13T06:38:15.140 INFO:teuthology.orchestra.run.smithi175.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 218 kB/s | 36 kB 00:00 2024-06-13T06:38:15.185 INFO:teuthology.orchestra.run.smithi007.stdout:(15/160): libcephsqlite-19.0.0-2946.g3d89accf.e 1.9 MB/s | 163 kB 00:00 2024-06-13T06:38:15.219 INFO:teuthology.orchestra.run.smithi007.stdout:(16/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.9 MB/s | 747 kB 00:00 2024-06-13T06:38:15.282 INFO:teuthology.orchestra.run.smithi175.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 645 kB/s | 221 kB 00:00 2024-06-13T06:38:15.286 INFO:teuthology.orchestra.run.smithi007.stdout:(17/160): librados-devel-19.0.0-2946.g3d89accf. 1.2 MB/s | 127 kB 00:00 2024-06-13T06:38:15.299 INFO:teuthology.orchestra.run.smithi175.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 303 kB/s | 47 kB 00:00 2024-06-13T06:38:15.387 INFO:teuthology.orchestra.run.smithi007.stdout:(18/160): libradosstriper1-19.0.0-2946.g3d89acc 2.9 MB/s | 501 kB 00:00 2024-06-13T06:38:15.458 INFO:teuthology.orchestra.run.smithi175.stdout:(98/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-06-13T06:38:15.478 INFO:teuthology.orchestra.run.smithi007.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 498 kB/s | 45 kB 00:00 2024-06-13T06:38:15.483 INFO:teuthology.orchestra.run.smithi175.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-06-13T06:38:15.499 INFO:teuthology.orchestra.run.smithi175.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-06-13T06:38:15.558 INFO:teuthology.orchestra.run.smithi175.stdout:(101/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rp 950 kB/s | 95 kB 00:00 2024-06-13T06:38:15.562 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:15.570 INFO:teuthology.orchestra.run.smithi007.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.5 MB/s | 135 kB 00:00 2024-06-13T06:38:15.571 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:15.572 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-13T06:38:15.583 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-13T06:38:15.583 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-13T06:38:15.584 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-13T06:38:15.585 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-13T06:38:15.587 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-13T06:38:15.589 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-13T06:38:15.590 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-13T06:38:15.600 INFO:teuthology.orchestra.run.smithi175.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-06-13T06:38:15.617 INFO:teuthology.orchestra.run.smithi175.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-06-13T06:38:15.617 INFO:teuthology.orchestra.run.smithi175.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:15.650 INFO:teuthology.orchestra.run.smithi175.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-13T06:38:15.653 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-06-13T06:38:15.671 INFO:teuthology.orchestra.run.smithi007.stdout:(21/160): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-13T06:38:15.683 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-06-13T06:38:15.683 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-06-13T06:38:15.683 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-06-13T06:38:15.683 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-13T06:38:15.684 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.2.2-1.el9 appstream 4.4 M 2024-06-13T06:38:15.685 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.4.0-1.el9 appstream 95 k 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-06-13T06:38:15.686 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.2.2-1.el9 baseos 1.4 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.2.2-1.el9 baseos 2.4 M 2024-06-13T06:38:15.687 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs x86_64 6.4.0-1.el9 baseos 1.1 M 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-13T06:38:15.688 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-13T06:38:15.689 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-13T06:38:15.690 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-13T06:38:15.691 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-13T06:38:15.692 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-13T06:38:15.693 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-13T06:38:15.694 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-13T06:38:15.695 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-13T06:38:15.696 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:Install 148 Packages 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 12 Packages 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 226 M 2024-06-13T06:38:15.697 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-06-13T06:38:15.709 INFO:teuthology.orchestra.run.smithi175.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-06-13T06:38:15.726 INFO:teuthology.orchestra.run.smithi175.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-06-13T06:38:15.779 INFO:teuthology.orchestra.run.smithi007.stdout:(22/160): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-13T06:38:15.921 INFO:teuthology.orchestra.run.smithi007.stdout:(23/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 8.4 MB/s | 5.3 MB 00:00 2024-06-13T06:38:15.964 INFO:teuthology.orchestra.run.smithi007.stdout:(24/160): python3-rbd-19.0.0-2946.g3d89accf.el9 1.6 MB/s | 300 kB 00:00 2024-06-13T06:38:15.997 INFO:teuthology.orchestra.run.smithi007.stdout:(25/160): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-13T06:38:16.039 INFO:teuthology.orchestra.run.smithi007.stdout:(26/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-13T06:38:16.043 INFO:teuthology.orchestra.run.smithi175.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-06-13T06:38:16.068 INFO:teuthology.orchestra.run.smithi175.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-06-13T06:38:16.085 INFO:teuthology.orchestra.run.smithi175.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-06-13T06:38:16.102 INFO:teuthology.orchestra.run.smithi175.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-06-13T06:38:16.123 INFO:teuthology.orchestra.run.smithi007.stdout:(27/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-13T06:38:16.135 INFO:teuthology.orchestra.run.smithi175.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-06-13T06:38:16.215 INFO:teuthology.orchestra.run.smithi007.stdout:(28/160): ceph-grafana-dashboards-19.0.0-2946.g 349 kB/s | 32 kB 00:00 2024-06-13T06:38:16.219 INFO:teuthology.orchestra.run.smithi175.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-06-13T06:38:16.277 INFO:teuthology.orchestra.run.smithi189.stdout:(1/160): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-06-13T06:38:16.303 INFO:teuthology.orchestra.run.smithi175.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-06-13T06:38:16.320 INFO:teuthology.orchestra.run.smithi175.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-06-13T06:38:16.337 INFO:teuthology.orchestra.run.smithi175.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-06-13T06:38:16.353 INFO:teuthology.orchestra.run.smithi175.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 906 kB/s | 15 kB 00:00 2024-06-13T06:38:16.365 INFO:teuthology.orchestra.run.smithi007.stdout:(29/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 8.4 MB/s | 3.1 MB 00:00 2024-06-13T06:38:16.371 INFO:teuthology.orchestra.run.smithi175.stdout:(117/160): lua-devel-5.4.4-4.el9.x86_64.rpm 27 kB/s | 22 kB 00:00 2024-06-13T06:38:16.390 INFO:teuthology.orchestra.run.smithi007.stdout:(30/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 816 kB/s | 142 kB 00:00 2024-06-13T06:38:16.396 INFO:teuthology.orchestra.run.smithi175.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-06-13T06:38:16.455 INFO:teuthology.orchestra.run.smithi175.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.8 MB/s | 303 kB 00:00 2024-06-13T06:38:16.497 INFO:teuthology.orchestra.run.smithi175.stdout:(120/160): python3-google-auth-2.29.0-2.el9.noa 4.9 MB/s | 205 kB 00:00 2024-06-13T06:38:16.530 INFO:teuthology.orchestra.run.smithi175.stdout:(121/160): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-06-13T06:38:16.564 INFO:teuthology.orchestra.run.smithi175.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-06-13T06:38:16.580 INFO:teuthology.orchestra.run.smithi175.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-06-13T06:38:16.606 INFO:teuthology.orchestra.run.smithi175.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 700 kB/s | 18 kB 00:00 2024-06-13T06:38:16.631 INFO:teuthology.orchestra.run.smithi175.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-06-13T06:38:16.657 INFO:teuthology.orchestra.run.smithi175.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-06-13T06:38:16.682 INFO:teuthology.orchestra.run.smithi175.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-06-13T06:38:16.741 INFO:teuthology.orchestra.run.smithi175.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-06-13T06:38:16.774 INFO:teuthology.orchestra.run.smithi175.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-06-13T06:38:16.833 INFO:teuthology.orchestra.run.smithi175.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-06-13T06:38:16.858 INFO:teuthology.orchestra.run.smithi175.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-13T06:38:16.883 INFO:teuthology.orchestra.run.smithi175.stdout:(132/160): protobuf-compiler-3.14.0-13.el9.x86_ 673 kB/s | 863 kB 00:01 2024-06-13T06:38:16.917 INFO:teuthology.orchestra.run.smithi175.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-06-13T06:38:16.924 INFO:teuthology.orchestra.run.smithi007.stdout:(31/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 6.6 MB/s | 3.5 MB 00:00 2024-06-13T06:38:16.968 INFO:teuthology.orchestra.run.smithi175.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-06-13T06:38:16.977 INFO:teuthology.orchestra.run.smithi189.stdout:(2/160): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 890 kB 00:00 2024-06-13T06:38:16.992 INFO:teuthology.orchestra.run.smithi175.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-13T06:38:17.018 INFO:teuthology.orchestra.run.smithi175.stdout:(136/160): python3-portend-3.1.0-2.el9.noarch.r 122 kB/s | 16 kB 00:00 2024-06-13T06:38:17.024 INFO:teuthology.orchestra.run.smithi007.stdout:(32/160): ceph-mgr-modules-core-19.0.0-2946.g3d 2.5 MB/s | 249 kB 00:00 2024-06-13T06:38:17.035 INFO:teuthology.orchestra.run.smithi175.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 435 kB/s | 28 kB 00:00 2024-06-13T06:38:17.052 INFO:teuthology.orchestra.run.smithi175.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-06-13T06:38:17.068 INFO:teuthology.orchestra.run.smithi175.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-06-13T06:38:17.085 INFO:teuthology.orchestra.run.smithi175.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-06-13T06:38:17.094 INFO:teuthology.orchestra.run.smithi189.stdout:(3/160): ceph-immutable-object-cache-19.0.0-294 1.2 MB/s | 145 kB 00:00 2024-06-13T06:38:17.102 INFO:teuthology.orchestra.run.smithi175.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-13T06:38:17.116 INFO:teuthology.orchestra.run.smithi007.stdout:(33/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 544 kB/s | 49 kB 00:00 2024-06-13T06:38:17.128 INFO:teuthology.orchestra.run.smithi175.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-06-13T06:38:17.153 INFO:teuthology.orchestra.run.smithi175.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-13T06:38:17.178 INFO:teuthology.orchestra.run.smithi175.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-13T06:38:17.195 INFO:teuthology.orchestra.run.smithi175.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-06-13T06:38:17.200 INFO:teuthology.orchestra.run.smithi007.stdout:(34/160): ceph-prometheus-alerts-19.0.0-2946.g3 199 kB/s | 16 kB 00:00 2024-06-13T06:38:17.221 INFO:teuthology.orchestra.run.smithi175.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-13T06:38:17.247 INFO:teuthology.orchestra.run.smithi175.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-06-13T06:38:17.296 INFO:teuthology.orchestra.run.smithi175.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-13T06:38:17.300 INFO:teuthology.orchestra.run.smithi007.stdout:(35/160): ceph-volume-19.0.0-2946.g3d89accf.el9 2.6 MB/s | 264 kB 00:00 2024-06-13T06:38:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:(36/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.1 MB/s | 763 kB 00:00 2024-06-13T06:38:17.576 INFO:teuthology.orchestra.run.smithi007.stdout:(37/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 200 kB/s | 18 kB 00:00 2024-06-13T06:38:17.638 INFO:teuthology.orchestra.run.smithi175.stdout:(149/160): NetworkManager-1.47.91-1.el9.x86_64. 6.8 MB/s | 2.3 MB 00:00 2024-06-13T06:38:17.655 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:17.901 INFO:teuthology.orchestra.run.smithi007.stdout:(38/160): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-06-13T06:38:17.909 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:17.930 INFO:teuthology.orchestra.run.smithi175.stdout:(150/160): NetworkManager-libnm-1.47.91-1.el9.x 6.2 MB/s | 1.8 MB 00:00 2024-06-13T06:38:17.980 INFO:teuthology.orchestra.run.smithi175.stdout:(151/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-06-13T06:38:18.089 INFO:teuthology.orchestra.run.smithi175.stdout:(152/160): NetworkManager-team-1.47.91-1.el9.x8 261 kB/s | 41 kB 00:00 2024-06-13T06:38:18.103 INFO:teuthology.orchestra.run.smithi189.stdout:(4/160): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-06-13T06:38:18.189 INFO:teuthology.orchestra.run.smithi175.stdout:(153/160): NetworkManager-tui-1.47.91-1.el9.x86 1.2 MB/s | 248 kB 00:00 2024-06-13T06:38:18.206 INFO:teuthology.orchestra.run.smithi175.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 313 kB/s | 37 kB 00:00 2024-06-13T06:38:18.235 INFO:teuthology.orchestra.run.smithi007.stdout:(39/160): fuse-2.9.9-15.el9.x86_64.rpm 122 kB/s | 80 kB 00:00 2024-06-13T06:38:18.238 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:18.329 INFO:teuthology.orchestra.run.smithi175.stdout:(155/160): librados2-19.0.0-2946.g3d89accf.el9. 3.1 MB/s | 3.4 MB 00:01 2024-06-13T06:38:18.395 INFO:teuthology.orchestra.run.smithi189.stdout:(5/160): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-06-13T06:38:18.437 INFO:teuthology.orchestra.run.smithi189.stdout:(6/160): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-06-13T06:38:18.440 INFO:teuthology.orchestra.run.smithi175.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 825 kB/s | 207 kB 00:00 2024-06-13T06:38:18.465 INFO:teuthology.orchestra.run.smithi175.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.0 MB/s | 260 kB 00:00 2024-06-13T06:38:18.632 INFO:teuthology.orchestra.run.smithi175.stdout:(158/160): openssl-3.2.2-1.el9.x86_64.rpm 4.5 MB/s | 1.4 MB 00:00 2024-06-13T06:38:18.752 INFO:teuthology.orchestra.run.smithi007.stdout:(40/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 11 MB/s | 48 MB 00:04 2024-06-13T06:38:18.757 INFO:teuthology.orchestra.run.smithi175.stdout:(159/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 3.7 MB/s | 1.1 MB 00:00 2024-06-13T06:38:18.816 INFO:teuthology.orchestra.run.smithi175.stdout:(160/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 6.4 MB/s | 2.4 MB 00:00 2024-06-13T06:38:18.825 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:38:18.825 INFO:teuthology.orchestra.run.smithi175.stdout:Total 16 MB/s | 226 MB 00:14 2024-06-13T06:38:18.885 INFO:teuthology.orchestra.run.smithi007.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 821 kB/s | 803 kB 00:00 2024-06-13T06:38:18.919 INFO:teuthology.orchestra.run.smithi007.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 286 kB/s | 195 kB 00:00 2024-06-13T06:38:18.969 INFO:teuthology.orchestra.run.smithi007.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-06-13T06:38:19.220 INFO:teuthology.orchestra.run.smithi007.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 3.7 MB/s | 1.2 MB 00:00 2024-06-13T06:38:19.253 INFO:teuthology.orchestra.run.smithi007.stdout:(45/160): python3-ply-3.11-14.el9.noarch.rpm 319 kB/s | 106 kB 00:00 2024-06-13T06:38:19.279 INFO:teuthology.orchestra.run.smithi007.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 438 kB/s | 135 kB 00:00 2024-06-13T06:38:19.281 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:38:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-06-13T06:38:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:19.404 INFO:teuthology.orchestra.run.smithi007.stdout:(48/160): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-06-13T06:38:19.409 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:19.420 INFO:teuthology.orchestra.run.smithi189.stdout:(7/160): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-06-13T06:38:19.505 INFO:teuthology.orchestra.run.smithi007.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-06-13T06:38:19.630 INFO:teuthology.orchestra.run.smithi007.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-06-13T06:38:19.637 INFO:teuthology.orchestra.run.smithi189.stdout:(8/160): ceph-common-19.0.0-2946.g3d89accf.el9. 5.8 MB/s | 21 MB 00:03 2024-06-13T06:38:19.655 INFO:teuthology.orchestra.run.smithi007.stdout:(51/160): boost-program-options-1.75.0-8.el9.x8 431 kB/s | 107 kB 00:00 2024-06-13T06:38:19.672 INFO:teuthology.orchestra.run.smithi007.stdout:(52/160): elfutils-libelf-devel-0.191-4.el9.x86 133 kB/s | 22 kB 00:00 2024-06-13T06:38:19.679 INFO:teuthology.orchestra.run.smithi189.stdout:(9/160): ceph-selinux-19.0.0-2946.g3d89accf.el9 608 kB/s | 25 kB 00:00 2024-06-13T06:38:19.773 INFO:teuthology.orchestra.run.smithi007.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-06-13T06:38:19.798 INFO:teuthology.orchestra.run.smithi007.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 33 kB 00:00 2024-06-13T06:38:20.071 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-06-13T06:38:20.173 INFO:teuthology.orchestra.run.smithi007.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-06-13T06:38:20.198 INFO:teuthology.orchestra.run.smithi007.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 43 kB/s | 18 kB 00:00 2024-06-13T06:38:20.223 INFO:teuthology.orchestra.run.smithi007.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 26 kB/s | 11 kB 00:00 2024-06-13T06:38:20.260 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-06-13T06:38:20.260 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-06-13T06:38:20.290 INFO:teuthology.orchestra.run.smithi007.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-06-13T06:38:20.324 INFO:teuthology.orchestra.run.smithi007.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 455 kB/s | 45 kB 00:00 2024-06-13T06:38:20.391 INFO:teuthology.orchestra.run.smithi007.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-06-13T06:38:20.450 INFO:teuthology.orchestra.run.smithi007.stdout:(61/160): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-06-13T06:38:20.475 INFO:teuthology.orchestra.run.smithi007.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 173 kB/s | 25 kB 00:00 2024-06-13T06:38:20.500 INFO:teuthology.orchestra.run.smithi007.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 454 kB/s | 47 kB 00:00 2024-06-13T06:38:20.576 INFO:teuthology.orchestra.run.smithi007.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-06-13T06:38:20.609 INFO:teuthology.orchestra.run.smithi007.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-06-13T06:38:20.635 INFO:teuthology.orchestra.run.smithi007.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 264 kB/s | 35 kB 00:00 2024-06-13T06:38:21.154 INFO:teuthology.orchestra.run.smithi189.stdout:(10/160): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 6.2 MB/s | 17 MB 00:02 2024-06-13T06:38:21.168 INFO:teuthology.orchestra.run.smithi007.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-06-13T06:38:21.280 INFO:teuthology.orchestra.run.smithi189.stdout:(11/160): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.7 MB/s | 221 kB 00:00 2024-06-13T06:38:21.347 INFO:teuthology.orchestra.run.smithi189.stdout:(12/160): cephfs-shell-19.0.0-2946.g3d89accf.el 408 kB/s | 27 kB 00:00 2024-06-13T06:38:21.413 INFO:teuthology.orchestra.run.smithi189.stdout:(13/160): libcephfs-devel-19.0.0-2946.g3d89accf 476 kB/s | 32 kB 00:00 2024-06-13T06:38:21.452 INFO:teuthology.orchestra.run.smithi007.stdout:(68/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-06-13T06:38:21.519 INFO:teuthology.orchestra.run.smithi007.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-06-13T06:38:21.522 INFO:teuthology.orchestra.run.smithi189.stdout:(14/160): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.9 MB/s | 10 MB 00:02 2024-06-13T06:38:21.561 INFO:teuthology.orchestra.run.smithi007.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 2.3 MB/s | 911 kB 00:00 2024-06-13T06:38:21.581 INFO:teuthology.orchestra.run.smithi189.stdout:(15/160): libcephsqlite-19.0.0-2946.g3d89accf.e 2.7 MB/s | 163 kB 00:00 2024-06-13T06:38:21.586 INFO:teuthology.orchestra.run.smithi007.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-06-13T06:38:21.606 INFO:teuthology.orchestra.run.smithi189.stdout:(16/160): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.8 MB/s | 747 kB 00:00 2024-06-13T06:38:21.640 INFO:teuthology.orchestra.run.smithi189.stdout:(17/160): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-13T06:38:21.723 INFO:teuthology.orchestra.run.smithi189.stdout:(18/160): libradosstriper1-19.0.0-2946.g3d89acc 4.2 MB/s | 501 kB 00:00 2024-06-13T06:38:21.799 INFO:teuthology.orchestra.run.smithi189.stdout:(19/160): python3-ceph-argparse-19.0.0-2946.g3d 601 kB/s | 45 kB 00:00 2024-06-13T06:38:21.874 INFO:teuthology.orchestra.run.smithi189.stdout:(20/160): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-13T06:38:21.903 INFO:teuthology.orchestra.run.smithi007.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 276 kB/s | 87 kB 00:00 2024-06-13T06:38:21.949 INFO:teuthology.orchestra.run.smithi189.stdout:(21/160): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-13T06:38:22.021 INFO:teuthology.orchestra.run.smithi007.stdout:(73/160): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-06-13T06:38:22.083 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-06-13T06:38:22.084 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-06-13T06:38:22.087 INFO:teuthology.orchestra.run.smithi007.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-06-13T06:38:22.091 INFO:teuthology.orchestra.run.smithi189.stdout:(22/160): python3-rados-19.0.0-2946.g3d89accf.e 2.2 MB/s | 321 kB 00:00 2024-06-13T06:38:22.158 INFO:teuthology.orchestra.run.smithi189.stdout:(23/160): python3-rbd-19.0.0-2946.g3d89accf.el9 4.4 MB/s | 300 kB 00:00 2024-06-13T06:38:22.205 INFO:teuthology.orchestra.run.smithi007.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 376 kB/s | 61 kB 00:00 2024-06-13T06:38:22.234 INFO:teuthology.orchestra.run.smithi189.stdout:(24/160): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-13T06:38:22.326 INFO:teuthology.orchestra.run.smithi189.stdout:(25/160): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 927 kB/s | 85 kB 00:00 2024-06-13T06:38:22.330 INFO:teuthology.orchestra.run.smithi007.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-06-13T06:38:22.372 INFO:teuthology.orchestra.run.smithi007.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 899 kB/s | 244 kB 00:00 2024-06-13T06:38:22.397 INFO:teuthology.orchestra.run.smithi007.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-06-13T06:38:22.430 INFO:teuthology.orchestra.run.smithi007.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 270 kB/s | 26 kB 00:00 2024-06-13T06:38:22.514 INFO:teuthology.orchestra.run.smithi007.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 165 kB/s | 19 kB 00:00 2024-06-13T06:38:22.548 INFO:teuthology.orchestra.run.smithi007.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-13T06:38:22.564 INFO:teuthology.orchestra.run.smithi007.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-06-13T06:38:22.743 INFO:teuthology.orchestra.run.smithi189.stdout:(26/160): librgw2-19.0.0-2946.g3d89accf.el9.x86 4.8 MB/s | 5.3 MB 00:01 2024-06-13T06:38:22.810 INFO:teuthology.orchestra.run.smithi189.stdout:(27/160): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 171 kB 00:00 2024-06-13T06:38:22.848 INFO:teuthology.orchestra.run.smithi007.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-06-13T06:38:22.868 INFO:teuthology.orchestra.run.smithi189.stdout:(28/160): ceph-grafana-dashboards-19.0.0-2946.g 546 kB/s | 32 kB 00:00 2024-06-13T06:38:22.882 INFO:teuthology.orchestra.run.smithi007.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 793 kB/s | 250 kB 00:00 2024-06-13T06:38:22.907 INFO:teuthology.orchestra.run.smithi007.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 652 kB/s | 222 kB 00:00 2024-06-13T06:38:22.918 INFO:teuthology.orchestra.run.smithi189.stdout:(29/160): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.2 MB/s | 3.1 MB 00:00 2024-06-13T06:38:22.944 INFO:teuthology.orchestra.run.smithi189.stdout:(30/160): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.8 MB/s | 142 kB 00:00 2024-06-13T06:38:22.957 INFO:teuthology.orchestra.run.smithi007.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 740 kB/s | 77 kB 00:00 2024-06-13T06:38:23.008 INFO:teuthology.orchestra.run.smithi007.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 425 kB/s | 42 kB 00:00 2024-06-13T06:38:23.041 INFO:teuthology.orchestra.run.smithi007.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-06-13T06:38:23.066 INFO:teuthology.orchestra.run.smithi007.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-06-13T06:38:23.117 INFO:teuthology.orchestra.run.smithi007.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.6 MB/s | 279 kB 00:00 2024-06-13T06:38:23.142 INFO:teuthology.orchestra.run.smithi007.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 518 kB/s | 51 kB 00:00 2024-06-13T06:38:23.167 INFO:teuthology.orchestra.run.smithi007.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-06-13T06:38:23.326 INFO:teuthology.orchestra.run.smithi007.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 227 kB/s | 42 kB 00:00 2024-06-13T06:38:23.351 INFO:teuthology.orchestra.run.smithi007.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 241 kB/s | 44 kB 00:00 2024-06-13T06:38:23.419 INFO:teuthology.orchestra.run.smithi189.stdout:(31/160): ceph-test-19.0.0-2946.g3d89accf.el9.x 13 MB/s | 48 MB 00:03 2024-06-13T06:38:23.469 INFO:teuthology.orchestra.run.smithi189.stdout:(32/160): ceph-mgr-modules-core-19.0.0-2946.g3d 4.9 MB/s | 249 kB 00:00 2024-06-13T06:38:23.519 INFO:teuthology.orchestra.run.smithi189.stdout:(33/160): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 993 kB/s | 49 kB 00:00 2024-06-13T06:38:23.543 INFO:teuthology.orchestra.run.smithi007.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 203 kB/s | 36 kB 00:00 2024-06-13T06:38:23.570 INFO:teuthology.orchestra.run.smithi189.stdout:(34/160): ceph-prometheus-alerts-19.0.0-2946.g3 329 kB/s | 16 kB 00:00 2024-06-13T06:38:23.620 INFO:teuthology.orchestra.run.smithi189.stdout:(35/160): ceph-volume-19.0.0-2946.g3d89accf.el9 5.2 MB/s | 264 kB 00:00 2024-06-13T06:38:23.652 INFO:teuthology.orchestra.run.smithi007.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 680 kB/s | 221 kB 00:00 2024-06-13T06:38:23.679 INFO:teuthology.orchestra.run.smithi189.stdout:(36/160): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.6 MB/s | 3.5 MB 00:00 2024-06-13T06:38:23.702 INFO:teuthology.orchestra.run.smithi007.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 324 kB/s | 47 kB 00:00 2024-06-13T06:38:23.746 INFO:teuthology.orchestra.run.smithi189.stdout:(37/160): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-06-13T06:38:23.762 INFO:teuthology.orchestra.run.smithi189.stdout:(38/160): cephfs-top-19.0.0-2946.g3d89accf.el9. 218 kB/s | 18 kB 00:00 2024-06-13T06:38:23.820 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:23.826 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:23.911 INFO:teuthology.orchestra.run.smithi007.stdout:(98/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-13T06:38:23.945 INFO:teuthology.orchestra.run.smithi007.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-06-13T06:38:23.987 INFO:teuthology.orchestra.run.smithi007.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 205 kB/s | 58 kB 00:00 2024-06-13T06:38:24.020 INFO:teuthology.orchestra.run.smithi007.stdout:(101/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rp 907 kB/s | 95 kB 00:00 2024-06-13T06:38:24.022 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:24.095 INFO:teuthology.orchestra.run.smithi007.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 592 kB/s | 64 kB 00:00 2024-06-13T06:38:24.121 INFO:teuthology.orchestra.run.smithi007.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-06-13T06:38:24.146 INFO:teuthology.orchestra.run.smithi007.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-06-13T06:38:24.147 INFO:teuthology.orchestra.run.smithi007.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 5.1 MB/s | 309 kB 00:00 2024-06-13T06:38:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 809 kB/s | 19 kB 00:00 2024-06-13T06:38:24.309 INFO:teuthology.orchestra.run.smithi189.stdout:(39/160): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-06-13T06:38:24.311 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:24.347 INFO:teuthology.orchestra.run.smithi007.stdout:(107/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-06-13T06:38:24.364 INFO:teuthology.orchestra.run.smithi007.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-06-13T06:38:24.389 INFO:teuthology.orchestra.run.smithi007.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-06-13T06:38:24.423 INFO:teuthology.orchestra.run.smithi007.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-06-13T06:38:24.473 INFO:teuthology.orchestra.run.smithi007.stdout:(111/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-06-13T06:38:24.582 INFO:teuthology.orchestra.run.smithi007.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 964 kB/s | 151 kB 00:00 2024-06-13T06:38:24.605 INFO:teuthology.orchestra.run.smithi189.stdout:(40/160): fuse-2.9.9-15.el9.x86_64.rpm 93 kB/s | 80 kB 00:00 2024-06-13T06:38:24.657 INFO:teuthology.orchestra.run.smithi007.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 7.7 MB/s | 548 kB 00:00 2024-06-13T06:38:24.674 INFO:teuthology.orchestra.run.smithi007.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.7 MB/s | 43 kB 00:00 2024-06-13T06:38:24.691 INFO:teuthology.orchestra.run.smithi007.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 2.0 MB/s | 32 kB 00:00 2024-06-13T06:38:24.725 INFO:teuthology.orchestra.run.smithi007.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 462 kB/s | 15 kB 00:00 2024-06-13T06:38:24.767 INFO:teuthology.orchestra.run.smithi007.stdout:(117/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-06-13T06:38:24.801 INFO:teuthology.orchestra.run.smithi007.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-06-13T06:38:24.859 INFO:teuthology.orchestra.run.smithi007.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-06-13T06:38:24.893 INFO:teuthology.orchestra.run.smithi007.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.3 MB/s | 303 kB 00:00 2024-06-13T06:38:24.935 INFO:teuthology.orchestra.run.smithi007.stdout:(121/160): python3-google-auth-2.29.0-2.el9.noa 2.7 MB/s | 205 kB 00:00 2024-06-13T06:38:24.947 INFO:teuthology.orchestra.run.smithi189.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 678 kB/s | 803 kB 00:01 2024-06-13T06:38:24.977 INFO:teuthology.orchestra.run.smithi007.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 3.5 MB/s | 144 kB 00:00 2024-06-13T06:38:25.011 INFO:teuthology.orchestra.run.smithi007.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 327 kB/s | 11 kB 00:00 2024-06-13T06:38:25.036 INFO:teuthology.orchestra.run.smithi007.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 735 kB/s | 18 kB 00:00 2024-06-13T06:38:25.061 INFO:teuthology.orchestra.run.smithi007.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 962 kB/s | 23 kB 00:00 2024-06-13T06:38:25.087 INFO:teuthology.orchestra.run.smithi007.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 806 kB/s | 19 kB 00:00 2024-06-13T06:38:25.112 INFO:teuthology.orchestra.run.smithi007.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 815 kB/s | 20 kB 00:00 2024-06-13T06:38:25.130 INFO:teuthology.orchestra.run.smithi189.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 238 kB/s | 195 kB 00:00 2024-06-13T06:38:25.229 INFO:teuthology.orchestra.run.smithi007.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-06-13T06:38:25.254 INFO:teuthology.orchestra.run.smithi007.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.9 MB/s | 46 kB 00:00 2024-06-13T06:38:25.288 INFO:teuthology.orchestra.run.smithi007.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.4 MB/s | 79 kB 00:00 2024-06-13T06:38:25.322 INFO:teuthology.orchestra.run.smithi007.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-06-13T06:38:25.372 INFO:teuthology.orchestra.run.smithi007.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 5.4 MB/s | 272 kB 00:00 2024-06-13T06:38:25.397 INFO:teuthology.orchestra.run.smithi007.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 673 kB/s | 16 kB 00:00 2024-06-13T06:38:25.481 INFO:teuthology.orchestra.run.smithi189.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 289 kB/s | 253 kB 00:00 2024-06-13T06:38:25.481 INFO:teuthology.orchestra.run.smithi007.stdout:(134/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-06-13T06:38:25.515 INFO:teuthology.orchestra.run.smithi007.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 773 kB/s | 90 kB 00:00 2024-06-13T06:38:25.540 INFO:teuthology.orchestra.run.smithi007.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 489 kB/s | 28 kB 00:00 2024-06-13T06:38:25.565 INFO:teuthology.orchestra.run.smithi007.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 1.3 MB/s | 31 kB 00:00 2024-06-13T06:38:25.590 INFO:teuthology.orchestra.run.smithi007.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.4 MB/s | 59 kB 00:00 2024-06-13T06:38:25.616 INFO:teuthology.orchestra.run.smithi007.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.5 MB/s | 36 kB 00:00 2024-06-13T06:38:25.650 INFO:teuthology.orchestra.run.smithi007.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-06-13T06:38:25.666 INFO:teuthology.orchestra.run.smithi007.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-13T06:38:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.7 MB/s | 90 kB 00:00 2024-06-13T06:38:25.775 INFO:teuthology.orchestra.run.smithi007.stdout:(143/160): libarrow-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-06-13T06:38:25.792 INFO:teuthology.orchestra.run.smithi007.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 1.3 MB/s | 20 kB 00:00 2024-06-13T06:38:25.826 INFO:teuthology.orchestra.run.smithi007.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-06-13T06:38:25.868 INFO:teuthology.orchestra.run.smithi007.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-13T06:38:25.906 INFO:teuthology.orchestra.run.smithi189.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 137 kB/s | 106 kB 00:00 2024-06-13T06:38:25.910 INFO:teuthology.orchestra.run.smithi007.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-06-13T06:38:26.002 INFO:teuthology.orchestra.run.smithi007.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-06-13T06:38:26.031 INFO:teuthology.orchestra.run.smithi189.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-06-13T06:38:26.194 INFO:teuthology.orchestra.run.smithi007.stdout:(149/160): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-13T06:38:26.306 INFO:teuthology.orchestra.run.smithi189.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 164 kB/s | 135 kB 00:00 2024-06-13T06:38:26.335 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-13T06:38:26.511 INFO:teuthology.orchestra.run.smithi007.stdout:(150/160): NetworkManager-libnm-1.47.91-1.el9.x 5.7 MB/s | 1.8 MB 00:00 2024-06-13T06:38:26.598 INFO:teuthology.orchestra.run.smithi189.stdout:(47/160): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-06-13T06:38:26.601 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:26.603 INFO:teuthology.orchestra.run.smithi007.stdout:(151/160): librados2-19.0.0-2946.g3d89accf.el9. 4.6 MB/s | 3.4 MB 00:00 2024-06-13T06:38:26.628 INFO:teuthology.orchestra.run.smithi007.stdout:(152/160): NetworkManager-team-1.47.91-1.el9.x8 357 kB/s | 41 kB 00:00 2024-06-13T06:38:26.704 INFO:teuthology.orchestra.run.smithi007.stdout:(153/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-06-13T06:38:26.738 INFO:teuthology.orchestra.run.smithi007.stdout:(154/160): NetworkManager-tui-1.47.91-1.el9.x86 1.8 MB/s | 248 kB 00:00 2024-06-13T06:38:26.740 INFO:teuthology.orchestra.run.smithi189.stdout:(48/160): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-06-13T06:38:26.741 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:26.763 INFO:teuthology.orchestra.run.smithi007.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 275 kB/s | 37 kB 00:00 2024-06-13T06:38:26.813 INFO:teuthology.orchestra.run.smithi007.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-06-13T06:38:26.872 INFO:teuthology.orchestra.run.smithi007.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.4 MB/s | 260 kB 00:00 2024-06-13T06:38:26.915 INFO:teuthology.orchestra.run.smithi189.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-06-13T06:38:26.939 INFO:teuthology.orchestra.run.smithi007.stdout:(158/160): openssl-3.2.2-1.el9.x86_64.rpm 7.8 MB/s | 1.4 MB 00:00 2024-06-13T06:38:26.941 INFO:teuthology.orchestra.run.smithi189.stdout:(50/160): python3-urllib3-1.26.5-5.el9.noarch.r 236 kB/s | 215 kB 00:00 2024-06-13T06:38:26.958 INFO:teuthology.orchestra.run.smithi189.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-06-13T06:38:27.000 INFO:teuthology.orchestra.run.smithi189.stdout:(52/160): flexiblas-3.0.4-8.el9.x86_64.rpm 785 kB/s | 33 kB 00:00 2024-06-13T06:38:27.030 INFO:teuthology.orchestra.run.smithi007.stdout:(159/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-06-13T06:38:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:(160/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 5.5 MB/s | 1.1 MB 00:00 2024-06-13T06:38:27.096 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:38:27.096 INFO:teuthology.orchestra.run.smithi007.stdout:Total 14 MB/s | 226 MB 00:15 2024-06-13T06:38:27.150 INFO:teuthology.orchestra.run.smithi189.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-06-13T06:38:27.209 INFO:teuthology.orchestra.run.smithi189.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-06-13T06:38:27.267 INFO:teuthology.orchestra.run.smithi189.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-06-13T06:38:27.293 INFO:teuthology.orchestra.run.smithi189.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-13T06:38:27.334 INFO:teuthology.orchestra.run.smithi189.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-06-13T06:38:27.360 INFO:teuthology.orchestra.run.smithi189.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-06-13T06:38:27.402 INFO:teuthology.orchestra.run.smithi189.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-13T06:38:27.460 INFO:teuthology.orchestra.run.smithi189.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-06-13T06:38:27.486 INFO:teuthology.orchestra.run.smithi189.stdout:(61/160): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-06-13T06:38:27.519 INFO:teuthology.orchestra.run.smithi189.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-06-13T06:38:27.544 INFO:teuthology.orchestra.run.smithi189.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-06-13T06:38:27.578 INFO:teuthology.orchestra.run.smithi189.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-06-13T06:38:27.603 INFO:teuthology.orchestra.run.smithi189.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-06-13T06:38:27.870 INFO:teuthology.orchestra.run.smithi189.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-06-13T06:38:27.937 INFO:teuthology.orchestra.run.smithi189.stdout:(67/160): openssl-devel-3.2.2-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-06-13T06:38:28.021 INFO:teuthology.orchestra.run.smithi189.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-06-13T06:38:28.054 INFO:teuthology.orchestra.run.smithi189.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-06-13T06:38:28.080 INFO:teuthology.orchestra.run.smithi189.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-13T06:38:28.172 INFO:teuthology.orchestra.run.smithi189.stdout:(71/160): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-06-13T06:38:28.230 INFO:teuthology.orchestra.run.smithi189.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-06-13T06:38:28.398 INFO:teuthology.orchestra.run.smithi189.stdout:(73/160): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-06-13T06:38:28.442 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-06-13T06:38:28.511 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:38:28.558 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:38:28.611 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:38:28.645 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:38:28.671 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:38:28.672 INFO:teuthology.orchestra.run.smithi189.stdout:(74/160): bison-3.7.4-5.el9.x86_64.rpm 455 kB/s | 943 kB 00:02 2024-06-13T06:38:28.713 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:38:29.094 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:38:29.272 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:29.393 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:29.414 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:29.506 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:29.535 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:38:29.568 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:38:29.754 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:38:29.799 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:38:29.829 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:29.931 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:29.954 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:30.016 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:30.047 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:30.072 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:30.263 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:30.352 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:30.516 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-06-13T06:38:30.535 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:38:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:30.780 INFO:teuthology.orchestra.run.smithi189.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 116 kB/s | 244 kB 00:02 2024-06-13T06:38:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-06-13T06:38:30.826 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-06-13T06:38:30.862 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:30.939 INFO:teuthology.orchestra.run.smithi189.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-06-13T06:38:30.957 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:31.165 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:31.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:38:31.368 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:38:31.413 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:38:31.447 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:38:31.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:38:31.555 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:38:31.599 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:38:31.637 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:38:31.667 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:38:31.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:38:31.789 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:38:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:38:31.863 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:38:31.941 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:38:31.974 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:38:32.009 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:38:32.038 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:38:32.110 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:38:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:38:32.228 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:38:32.364 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:38:32.381 INFO:teuthology.orchestra.run.smithi189.stdout:(77/160): python3-colorama-0.4.6-3.el9.noarch.r 15 kB/s | 61 kB 00:03 2024-06-13T06:38:32.416 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:38:32.439 INFO:teuthology.orchestra.run.smithi189.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-06-13T06:38:32.573 INFO:teuthology.orchestra.run.smithi189.stdout:(79/160): python3-babel-2.9.1-2.el9.noarch.rpm 1.4 MB/s | 6.0 MB 00:04 2024-06-13T06:38:32.598 INFO:teuthology.orchestra.run.smithi189.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-13T06:38:32.624 INFO:teuthology.orchestra.run.smithi189.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-06-13T06:38:32.682 INFO:teuthology.orchestra.run.smithi189.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-06-13T06:38:32.749 INFO:teuthology.orchestra.run.smithi189.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-06-13T06:38:32.948 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:38:32.950 INFO:teuthology.orchestra.run.smithi189.stdout:(84/160): python3-jsonpatch-1.21-16.el9.noarch. 13 kB/s | 26 kB 00:02 2024-06-13T06:38:33.025 INFO:teuthology.orchestra.run.smithi189.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-06-13T06:38:33.037 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:38:33.074 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:38:33.118 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:38:33.134 INFO:teuthology.orchestra.run.smithi189.stdout:(86/160): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-06-13T06:38:33.175 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:38:33.192 INFO:teuthology.orchestra.run.smithi189.stdout:(87/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-06-13T06:38:33.213 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:38:33.251 INFO:teuthology.orchestra.run.smithi189.stdout:(88/160): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-06-13T06:38:33.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:38:33.301 INFO:teuthology.orchestra.run.smithi189.stdout:(89/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-13T06:38:33.343 INFO:teuthology.orchestra.run.smithi189.stdout:(90/160): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-06-13T06:38:33.419 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:38:33.483 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:38:33.539 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:38:33.557 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-06-13T06:38:33.558 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-06-13T06:38:33.612 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:38:33.642 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:38:33.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:38:33.954 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:38:33.987 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:38:34.031 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:38:34.073 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:38:34.143 INFO:teuthology.orchestra.run.smithi189.stdout:(91/160): python3-packaging-20.9-5.el9.noarch.r 55 kB/s | 77 kB 00:01 2024-06-13T06:38:34.193 INFO:teuthology.orchestra.run.smithi189.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-06-13T06:38:34.214 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:38:34.252 INFO:teuthology.orchestra.run.smithi189.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-06-13T06:38:34.257 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:38:34.299 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:38:34.336 INFO:teuthology.orchestra.run.smithi189.stdout:(94/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 5.1 MB 00:01 2024-06-13T06:38:34.378 INFO:teuthology.orchestra.run.smithi189.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-06-13T06:38:34.395 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:38:34.403 INFO:teuthology.orchestra.run.smithi189.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-06-13T06:38:34.431 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:38:34.436 INFO:teuthology.orchestra.run.smithi189.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-06-13T06:38:34.462 INFO:teuthology.orchestra.run.smithi189.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-06-13T06:38:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:38:34.496 INFO:teuthology.orchestra.run.smithi189.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-06-13T06:38:34.521 INFO:teuthology.orchestra.run.smithi189.stdout:(100/160): xfsprogs-devel-6.4.0-1.el9.x86_64.rp 1.6 MB/s | 95 kB 00:00 2024-06-13T06:38:34.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:38:34.547 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:38:34.554 INFO:teuthology.orchestra.run.smithi189.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-06-13T06:38:34.580 INFO:teuthology.orchestra.run.smithi189.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-06-13T06:38:34.623 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:38:34.640 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:34.646 INFO:teuthology.orchestra.run.smithi189.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-13T06:38:34.755 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:38:34.814 INFO:teuthology.orchestra.run.smithi189.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-06-13T06:38:34.872 INFO:teuthology.orchestra.run.smithi189.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-06-13T06:38:34.897 INFO:teuthology.orchestra.run.smithi189.stdout:(105/160): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-06-13T06:38:34.922 INFO:teuthology.orchestra.run.smithi189.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-06-13T06:38:34.948 INFO:teuthology.orchestra.run.smithi189.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-06-13T06:38:34.973 INFO:teuthology.orchestra.run.smithi189.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-13T06:38:34.998 INFO:teuthology.orchestra.run.smithi189.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-13T06:38:35.032 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:38:35.032 INFO:teuthology.orchestra.run.smithi189.stdout:(110/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-06-13T06:38:35.066 INFO:teuthology.orchestra.run.smithi189.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-06-13T06:38:35.067 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:38:35.100 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:38:35.125 INFO:teuthology.orchestra.run.smithi189.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-13T06:38:35.141 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:38:35.171 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:38:35.175 INFO:teuthology.orchestra.run.smithi189.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-06-13T06:38:35.200 INFO:teuthology.orchestra.run.smithi189.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-13T06:38:35.201 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:38:35.225 INFO:teuthology.orchestra.run.smithi189.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-13T06:38:35.227 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:38:35.250 INFO:teuthology.orchestra.run.smithi189.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 601 kB/s | 15 kB 00:00 2024-06-13T06:38:35.260 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:38:35.284 INFO:teuthology.orchestra.run.smithi189.stdout:(117/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-06-13T06:38:35.302 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:38:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-06-13T06:38:35.336 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:38:35.343 INFO:teuthology.orchestra.run.smithi189.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-13T06:38:35.369 INFO:teuthology.orchestra.run.smithi189.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-06-13T06:38:35.402 INFO:teuthology.orchestra.run.smithi189.stdout:(121/160): python3-google-auth-2.29.0-2.el9.noa 3.4 MB/s | 205 kB 00:00 2024-06-13T06:38:35.424 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:38:35.436 INFO:teuthology.orchestra.run.smithi189.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-06-13T06:38:35.475 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:38:35.494 INFO:teuthology.orchestra.run.smithi189.stdout:(123/160): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-06-13T06:38:35.511 INFO:teuthology.orchestra.run.smithi189.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-06-13T06:38:35.515 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:38:35.528 INFO:teuthology.orchestra.run.smithi189.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-06-13T06:38:35.545 INFO:teuthology.orchestra.run.smithi189.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 690 kB/s | 23 kB 00:00 2024-06-13T06:38:35.562 INFO:teuthology.orchestra.run.smithi189.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 583 kB/s | 19 kB 00:00 2024-06-13T06:38:35.562 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:38:35.579 INFO:teuthology.orchestra.run.smithi189.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 583 kB/s | 20 kB 00:00 2024-06-13T06:38:35.596 INFO:teuthology.orchestra.run.smithi189.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-06-13T06:38:35.630 INFO:teuthology.orchestra.run.smithi189.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-06-13T06:38:35.655 INFO:teuthology.orchestra.run.smithi189.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-13T06:38:35.662 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:38:35.697 INFO:teuthology.orchestra.run.smithi189.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-06-13T06:38:35.722 INFO:teuthology.orchestra.run.smithi189.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-06-13T06:38:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:38:35.749 INFO:teuthology.orchestra.run.smithi189.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-06-13T06:38:35.773 INFO:teuthology.orchestra.run.smithi189.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-06-13T06:38:35.815 INFO:teuthology.orchestra.run.smithi189.stdout:(136/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-06-13T06:38:35.840 INFO:teuthology.orchestra.run.smithi189.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-06-13T06:38:35.857 INFO:teuthology.orchestra.run.smithi189.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-06-13T06:38:35.874 INFO:teuthology.orchestra.run.smithi189.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-06-13T06:38:35.899 INFO:teuthology.orchestra.run.smithi189.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-06-13T06:38:35.916 INFO:teuthology.orchestra.run.smithi189.stdout:(141/160): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-06-13T06:38:35.933 INFO:teuthology.orchestra.run.smithi189.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 862 kB/s | 51 kB 00:00 2024-06-13T06:38:35.959 INFO:teuthology.orchestra.run.smithi189.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-06-13T06:38:35.984 INFO:teuthology.orchestra.run.smithi189.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-06-13T06:38:36.001 INFO:teuthology.orchestra.run.smithi189.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-06-13T06:38:36.026 INFO:teuthology.orchestra.run.smithi189.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-13T06:38:36.051 INFO:teuthology.orchestra.run.smithi189.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-06-13T06:38:36.127 INFO:teuthology.orchestra.run.smithi189.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-13T06:38:36.481 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:38:36.510 INFO:teuthology.orchestra.run.smithi189.stdout:(149/160): librados2-19.0.0-2946.g3d89accf.el9. 7.0 MB/s | 3.4 MB 00:00 2024-06-13T06:38:36.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:38:36.556 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:38:36.657 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:38:36.694 INFO:teuthology.orchestra.run.smithi189.stdout:(150/160): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-06-13T06:38:36.720 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:38:36.764 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:38:36.862 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:38:36.900 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:38:37.078 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:38:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:38:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:38:37.845 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:38:37.894 INFO:teuthology.orchestra.run.smithi189.stdout:(151/160): NetworkManager-team-1.47.91-1.el9.x8 34 kB/s | 41 kB 00:01 2024-06-13T06:38:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:38:38.203 INFO:teuthology.orchestra.run.smithi189.stdout:(152/160): NetworkManager-1.47.91-1.el9.x86_64. 1.1 MB/s | 2.3 MB 00:02 2024-06-13T06:38:38.578 INFO:teuthology.orchestra.run.smithi189.stdout:(153/160): NetworkManager-libnm-1.47.91-1.el9.x 905 kB/s | 1.8 MB 00:02 2024-06-13T06:38:39.138 INFO:teuthology.orchestra.run.smithi189.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 39 kB/s | 37 kB 00:00 2024-06-13T06:38:39.225 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:38:39.230 INFO:teuthology.orchestra.run.smithi189.stdout:(155/160): NetworkManager-tui-1.47.91-1.el9.x86 186 kB/s | 248 kB 00:01 2024-06-13T06:38:39.327 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:38:39.364 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:38:39.397 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:38:39.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:38:39.752 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:38:39.848 INFO:teuthology.orchestra.run.smithi189.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 163 kB/s | 207 kB 00:01 2024-06-13T06:38:39.859 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:38:39.890 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:38:39.926 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:38:40.400 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:38:40.430 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:38:40.501 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:38:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-06-13T06:38:40.531 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:38:40.557 INFO:teuthology.orchestra.run.smithi189.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 183 kB/s | 260 kB 00:01 2024-06-13T06:38:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:38:40.685 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:38:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:38:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:38:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:38:40.941 INFO:teuthology.orchestra.run.smithi189.stdout:(158/160): openssl-3.2.2-1.el9.x86_64.rpm 811 kB/s | 1.4 MB 00:01 2024-06-13T06:38:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:38:41.416 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:38:41.521 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:41.649 INFO:teuthology.orchestra.run.smithi189.stdout:(159/160): openssl-libs-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 2.4 MB 00:01 2024-06-13T06:38:41.702 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:41.738 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:41.877 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:41.913 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:38:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:38:42.141 INFO:teuthology.orchestra.run.smithi189.stdout:(160/160): xfsprogs-6.4.0-1.el9.x86_64.rpm 699 kB/s | 1.1 MB 00:01 2024-06-13T06:38:42.148 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:38:42.148 INFO:teuthology.orchestra.run.smithi189.stdout:Total 8.5 MB/s | 226 MB 00:26 2024-06-13T06:38:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:38:42.377 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:38:42.402 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:42.587 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:42.681 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:42.713 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:42.786 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:42.837 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:38:42.870 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:38:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:43.249 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:43.402 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-06-13T06:38:43.408 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:38:43.448 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:38:43.535 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:38:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:38:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:38:43.536 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:43.575 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:38:43.597 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-06-13T06:38:43.597 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-06-13T06:38:43.621 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:38:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:44.123 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:44.704 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:38:45.034 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:38:45.074 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:38:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:38:45.228 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:38:45.278 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:38:45.343 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:38:45.390 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:38:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:38:45.431 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-06-13T06:38:45.431 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-06-13T06:38:45.553 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:38:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:38:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:38:45.718 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:38:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:38:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:38:45.978 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:38:46.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:38:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:38:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:38:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:38:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:38:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:38:47.327 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:38:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:38:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:38:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:38:47.621 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:38:47.682 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:38:47.765 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:38:47.994 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:38:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:38:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:38:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:38:48.295 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:38:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:38:48.405 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:38:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:38:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:38:48.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:38:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:38:48.844 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:38:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:38:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:38:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:38:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:38:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:38:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:38:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:38:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:38:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:38:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:38:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:38:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:38:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:38:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:38:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:38:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:38:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:38:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:38:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:38:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:38:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:38:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:38:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:38:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:38:51.612 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:38:51.675 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:38:51.700 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:38:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-06-13T06:38:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.2.2-1.el9.x86_64 1/172 2024-06-13T06:38:51.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:38:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-06-13T06:38:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-06-13T06:38:51.892 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:38:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-06-13T06:38:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-06-13T06:38:51.978 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:38:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-06-13T06:38:52.094 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:38:52.143 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:38:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-06-13T06:38:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:38:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:38:52.473 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:38:52.593 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:38:52.615 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:52.706 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:38:52.736 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:38:52.769 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-06-13T06:38:52.967 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-06-13T06:38:53.018 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-06-13T06:38:53.046 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:38:53.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:53.234 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/172 2024-06-13T06:38:53.256 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:53.282 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-06-13T06:38:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:38:53.473 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-06-13T06:38:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:38:53.769 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsprogs-6.4.0-1.el9.x86_64 18/172 2024-06-13T06:38:53.931 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:53.991 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 19/172 2024-06-13T06:38:54.085 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:54.159 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:54.359 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 20/172 2024-06-13T06:38:54.465 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-06-13T06:38:54.569 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-06-13T06:38:54.606 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-06-13T06:38:54.641 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-06-13T06:38:54.706 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-06-13T06:38:54.740 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-06-13T06:38:54.786 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-06-13T06:38:54.818 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-06-13T06:38:54.846 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-06-13T06:38:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-06-13T06:38:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/172 2024-06-13T06:38:55.008 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-06-13T06:38:55.038 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:38:55.038 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-06-13T06:38:55.039 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-06-13T06:38:55.040 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:55.043 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-06-13T06:38:55.120 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-06-13T06:38:55.154 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 35/172 2024-06-13T06:38:55.197 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 36/172 2024-06-13T06:38:55.202 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:38:55.217 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:38:55.217 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:38:55.217 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:38:55.217 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:55.226 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 37/172 2024-06-13T06:38:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:38:55.298 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-06-13T06:38:55.341 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-06-13T06:38:55.371 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:38:55.408 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:38:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-06-13T06:38:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:38:55.552 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-06-13T06:38:55.611 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-06-13T06:38:55.832 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:38:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:38:55.999 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:38:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:38:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:38:56.149 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-06-13T06:38:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-06-13T06:38:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-06-13T06:38:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-06-13T06:38:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-06-13T06:38:56.384 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 48/172 2024-06-13T06:38:56.426 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-06-13T06:38:56.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-06-13T06:38:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:38:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.2.2-1.el9.x86_64 51/172 2024-06-13T06:38:56.680 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:38:56.705 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:38:56.706 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:38:56.706 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:38:56.706 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:38:56.706 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:38:56.706 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:38:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-06-13T06:38:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:38:56.807 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 53/172 2024-06-13T06:38:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:38:56.845 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 54/172 2024-06-13T06:38:56.882 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 55/172 2024-06-13T06:38:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/172 2024-06-13T06:38:56.931 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:38:56.954 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:56.965 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/172 2024-06-13T06:38:57.008 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 58/172 2024-06-13T06:38:57.051 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 59/172 2024-06-13T06:38:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 60/172 2024-06-13T06:38:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 61/172 2024-06-13T06:38:57.277 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 62/172 2024-06-13T06:38:57.373 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 63/172 2024-06-13T06:38:57.399 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:38:57.402 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 64/172 2024-06-13T06:38:57.421 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:38:57.421 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:38:57.422 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:38:57.422 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:38:57.422 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:38:57.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:38:57.442 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 65/172 2024-06-13T06:38:57.469 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 66/172 2024-06-13T06:38:57.492 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 67/172 2024-06-13T06:38:57.529 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:38:57.568 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 68/172 2024-06-13T06:38:57.586 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:38:57.641 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:38:57.696 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 69/172 2024-06-13T06:38:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 70/172 2024-06-13T06:38:57.769 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 71/172 2024-06-13T06:38:57.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/172 2024-06-13T06:38:57.836 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:38:57.899 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:38:57.909 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/172 2024-06-13T06:38:57.977 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:38:58.074 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:38:58.082 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 74/172 2024-06-13T06:38:58.215 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:38:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:38:58.288 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 75/172 2024-06-13T06:38:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:38:58.317 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:38:58.323 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 76/172 2024-06-13T06:38:58.356 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/172 2024-06-13T06:38:58.398 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/172 2024-06-13T06:38:58.432 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/172 2024-06-13T06:38:58.512 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/172 2024-06-13T06:38:58.554 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 81/172 2024-06-13T06:38:58.585 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 82/172 2024-06-13T06:38:58.632 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 83/172 2024-06-13T06:38:58.727 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:38:58.731 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 84/172 2024-06-13T06:38:58.757 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:38:58.778 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:38:58.788 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:38:58.859 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:38:58.888 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:38:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-06-13T06:38:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-06-13T06:38:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-06-13T06:38:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:38:59.698 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-06-13T06:38:59.748 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-06-13T06:38:59.794 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-06-13T06:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:38:59.882 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-06-13T06:38:59.920 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:38:59.929 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-06-13T06:38:59.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:00.101 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-06-13T06:39:00.135 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-06-13T06:39:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-06-13T06:39:00.511 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:39:00.555 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:39:00.873 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-06-13T06:39:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:39:00.885 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:39:00.886 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:39:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:00.920 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:00.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:39:00.949 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:01.002 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-06-13T06:39:01.052 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:01.211 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:39:01.242 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:01.266 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:01.266 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:01.266 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:39:01.267 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:01.267 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:01.267 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:01.318 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:01.330 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:01.426 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:39:01.474 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:39:02.251 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-06-13T06:39:02.331 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-06-13T06:39:02.362 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-06-13T06:39:02.370 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:39:02.392 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-06-13T06:39:02.403 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:02.427 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:02.427 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:02.427 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:39:02.427 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:02.428 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:02.428 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:02.471 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:02.493 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:02.493 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:02.493 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:39:02.494 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:02.538 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:02.561 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:02.705 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-06-13T06:39:02.747 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:39:02.838 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 104/172 2024-06-13T06:39:02.869 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 105/172 2024-06-13T06:39:02.872 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:02.895 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:02.905 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-06-13T06:39:03.387 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-06-13T06:39:03.409 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:39:03.479 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 108/172 2024-06-13T06:39:03.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 109/172 2024-06-13T06:39:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:39:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:39:06.385 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 110/172 2024-06-13T06:39:06.427 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:39:06.519 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 111/172 2024-06-13T06:39:06.519 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-13T06:39:06.519 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-13T06:39:06.519 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:06.562 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:39:07.383 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:39:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:39:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:39:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:39:07.524 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:39:07.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:07.762 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:39:07.794 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:39:07.827 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:39:07.856 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:07.911 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:39:08.485 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:08.525 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:08.545 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:09.392 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:39:09.487 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:39:09.528 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:39:09.610 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:39:09.652 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:39:09.674 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:39:09.699 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:09.726 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:39:09.726 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:09.831 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:09.860 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:39:09.860 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:09.944 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:09.969 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:39:09.974 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:09.974 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:10.154 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:10.154 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:10.239 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:10.263 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:10.289 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:39:10.352 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-06-13T06:39:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:39:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:39:14.349 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:14.350 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:39:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:15.915 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:39:15.916 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:39:15.917 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:39:15.918 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:39:15.930 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:39:15.931 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:39:15.932 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:39:15.933 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:39:15.934 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:39:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:39:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:39:15.937 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:39:15.938 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:39:15.939 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:15.940 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:39:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:39:16.106 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:39:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:39:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:39:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:39:16.107 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:16.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:39:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:39:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:39:17.047 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:17.048 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:39:17.048 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:39:17.048 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:39:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:17.183 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:39:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:39:17.330 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:39:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:39:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:39:17.750 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:39:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:39:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:39:18.101 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 112/172 2024-06-13T06:39:18.101 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-06-13T06:39:18.101 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-06-13T06:39:18.102 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:39:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:39:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:39:18.279 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:39:18.295 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 113/172 2024-06-13T06:39:18.296 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:18.296 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-13T06:39:18.296 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:18.748 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:39:18.777 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.778 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:39:18.779 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.780 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:39:18.781 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:39:18.782 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:39:18.783 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:39:18.784 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:39:18.785 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.786 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:39:18.787 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:39:18.788 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:39:18.789 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:39:18.790 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:39:18.791 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:39:18.792 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-06-13T06:39:18.793 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-06-13T06:39:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:39:18.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:39:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:39:19.241 DEBUG:teuthology.parallel:result is None 2024-06-13T06:39:19.774 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 114/172 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-13T06:39:19.801 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:20.024 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 115/172 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-13T06:39:20.047 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:39:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:20.492 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:39:20.515 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 116/172 2024-06-13T06:39:20.515 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:20.515 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-13T06:39:20.515 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:39:20.516 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-13T06:39:20.516 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:20.622 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-06-13T06:39:20.679 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-06-13T06:39:20.735 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-06-13T06:39:20.923 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-06-13T06:39:20.976 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-06-13T06:39:21.055 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-06-13T06:39:21.151 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-06-13T06:39:21.301 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-06-13T06:39:21.342 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-06-13T06:39:21.374 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-06-13T06:39:21.410 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-06-13T06:39:21.591 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:21.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:21.773 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:21.828 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:39:21.859 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 128/172 2024-06-13T06:39:21.889 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 129/172 2024-06-13T06:39:21.952 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 130/172 2024-06-13T06:39:21.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:39:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:39:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:22.104 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:22.104 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:39:22.104 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:22.104 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:22.163 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:22.203 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:22.318 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:39:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:39:23.033 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 131/172 2024-06-13T06:39:23.066 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:23.407 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:39:23.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:23.525 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:23.526 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:23.526 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:39:23.526 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:23.526 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:23.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:23.583 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:23.663 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:39:23.663 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:23.743 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:24.037 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 132/172 2024-06-13T06:39:24.075 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:24.179 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 133/172 2024-06-13T06:39:24.210 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 134/172 2024-06-13T06:39:24.396 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:24.419 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 135/172 2024-06-13T06:39:24.419 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:24.419 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-13T06:39:24.420 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:24.420 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-13T06:39:24.420 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:24.470 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:24.481 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 136/172 2024-06-13T06:39:24.587 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 137/172 2024-06-13T06:39:24.635 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 138/172 2024-06-13T06:39:25.532 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-45.el9.x86_64 139/172 2024-06-13T06:39:25.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:25.578 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 140/172 2024-06-13T06:39:25.578 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:25.578 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-13T06:39:25.578 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:25.579 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-13T06:39:25.579 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:25.622 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:25.645 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 141/172 2024-06-13T06:39:25.645 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:25.645 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-13T06:39:25.645 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:25.690 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:25.712 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/172 2024-06-13T06:39:25.712 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:25.712 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-13T06:39:25.712 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:25.712 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-13T06:39:25.713 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:26.015 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 143/172 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-13T06:39:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:29.507 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:39:29.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:39:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:39:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:39:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:39:29.997 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:30.053 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:39:30.087 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:39:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:39:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:39:30.569 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 144/172 2024-06-13T06:39:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 145/172 2024-06-13T06:39:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 146/172 2024-06-13T06:39:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.4.0-1.el9.x86_64 147/172 2024-06-13T06:39:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-06-13T06:39:30.899 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:30.956 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-06-13T06:39:30.988 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 151/172 2024-06-13T06:39:31.028 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-06-13T06:39:31.058 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:31.104 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 154/172 2024-06-13T06:39:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:31.721 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-06-13T06:39:32.632 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:39:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 156/172 2024-06-13T06:39:32.706 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:39:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:39:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.2-1.el9.x86_64 157/172 2024-06-13T06:39:32.831 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:39:32.861 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-06-13T06:39:32.876 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:39:32.901 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:39:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:39:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:33.006 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-06-13T06:39:33.043 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-06-13T06:39:33.044 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:33.076 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-06-13T06:39:33.077 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:39:33.077 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:33.111 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:33.153 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:33.162 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-06-13T06:39:33.162 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:33.178 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:39:33.185 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.185 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.303 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-06-13T06:39:33.349 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-06-13T06:39:33.349 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:33.363 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.363 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:33.460 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:33.485 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:33.494 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-06-13T06:39:33.511 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:39:33.533 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-06-13T06:39:33.575 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:33.583 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.583 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.920 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-06-13T06:39:33.921 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:34.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-06-13T06:39:34.110 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-06-13T06:39:34.292 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:34.561 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:34.562 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-13T06:39:34.562 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:38.991 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:38.991 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:39:38.992 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:39:38.993 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:39:38.994 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:39:38.994 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:39:39.008 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:39:39.009 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:39:39.010 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:39:39.011 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:39:39.011 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:39:39.011 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:39:39.011 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:39:39.011 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:39:39.012 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:39:39.013 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:39:39.014 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:39:39.015 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:39:39.016 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:39:39.017 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:39:39.018 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:39:39.019 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:39:39.020 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:39:39.021 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:39:39.021 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:39.021 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:39:40.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/172 2024-06-13T06:39:40.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/172 2024-06-13T06:39:40.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/172 2024-06-13T06:39:40.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/172 2024-06-13T06:39:40.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/172 2024-06-13T06:39:40.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-06-13T06:39:40.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-06-13T06:39:40.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.2-1.el9.x86_64 68/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-06-13T06:39:40.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-06-13T06:39:40.656 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.4.0-1.el9.x86_64 101/172 2024-06-13T06:39:40.657 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-06-13T06:39:40.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-06-13T06:39:40.659 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-06-13T06:39:40.660 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/172 2024-06-13T06:39:40.661 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-06-13T06:39:40.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-06-13T06:39:40.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-1.el9.x86_64 167/172 2024-06-13T06:39:40.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-06-13T06:39:40.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-1.el9.x86_64 169/172 2024-06-13T06:39:40.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-06-13T06:39:40.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-6.4.0-1.el9.x86_64 171/172 2024-06-13T06:39:41.855 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:39:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:39:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:39:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:39:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:39:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:39:41.864 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:39:41.865 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:39:41.866 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.867 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:39:41.868 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:39:41.869 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:39:41.869 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-06-13T06:39:42.319 DEBUG:teuthology.parallel:result is None 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-13T06:39:43.172 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-6.4.0-1.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.173 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.174 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-06-13T06:39:43.175 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-13T06:39:43.176 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-13T06:39:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.2-1.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-13T06:39:43.178 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-13T06:39:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-13T06:39:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-13T06:39:43.181 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.182 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-13T06:39:43.183 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.4.0-1.el9.x86_64 2024-06-13T06:39:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-13T06:39:43.185 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-13T06:39:43.185 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:39:43.185 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-06-13T06:39:43.777 DEBUG:teuthology.parallel:result is None 2024-06-13T06:39:43.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:43.934 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:39:43.997 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:43.998 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:43.998 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:39:44.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:44.153 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:39:44.179 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:44.179 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:44.180 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:39:44.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:44.324 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-13T06:39:44.350 INFO:teuthology.orchestra.run.smithi189.stdout:19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:44.350 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-13T06:39:44.350 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-13T06:39:44.352 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-13T06:39:44.352 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:39:44.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:39:44.427 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:39:44.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:39:44.458 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:39:44.458 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-13T06:39:44.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-13T06:39:44.489 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:39:44.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:39:44.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:39:44.663 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:39:44.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:39:44.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:39:44.818 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:39:44.818 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-06-13T06:39:44.850 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-13T06:39:44.919 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-13T06:39:44.919 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:39:44.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:39:44.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:39:45.090 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:39:45.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:39:45.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:39:45.187 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:39:45.187 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-13T06:39:45.215 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-13T06:39:45.286 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-13T06:39:45.286 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:39:45.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:39:45.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:39:45.451 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:39:45.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:39:45.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:39:45.547 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:39:45.547 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-06-13T06:39:45.576 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-13T06:39:45.643 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-13T06:39:45.741 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2}} 2024-06-13T06:39:45.741 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:45.742 INFO:tasks.cephadm:Cluster fsid is b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:39:45.742 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-13T06:39:45.742 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.175', 'mon.c': '172.21.15.189'} 2024-06-13T06:39:45.743 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-06-13T06:39:45.743 INFO:tasks.cephadm:First mgr is x 2024-06-13T06:39:45.743 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-13T06:39:45.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-06-13T06:39:45.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-06-13T06:39:45.835 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-06-13T06:39:45.864 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-13T06:39:45.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:46.176 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-13T06:39:46.327 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:39:46.328 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-13T06:39:46.328 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:47.334 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:39 /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:47.335 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:47.832 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:39 /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:47.833 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:48.822 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 13 06:39 /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:48.822 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:48.880 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:48.899 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-13T06:39:48.934 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-13T06:39:48.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:39:48.939 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:39:48.941 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-13T06:39:49.683 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:39:49.876 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:39:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:40:29.824 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T06:40:29.825 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:40:37.993 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-06-13T06:40:37.993 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:40:37.993 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:40:37.994 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-06-13T06:40:37.994 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:40:37.994 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-06-13T06:40:37.994 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-06-13T06:40:40.060 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-13T06:40:40.060 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:40:40.060 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-13T06:40:40.061 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-06-13T06:40:40.061 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-13T06:40:40.061 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-06-13T06:40:40.061 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-13T06:40:40.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-06-13T06:40:40.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-06-13T06:40:40.210 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-06-13T06:40:40.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-06-13T06:40:40.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-06-13T06:40:40.353 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-06-13T06:40:40.382 INFO:tasks.cephadm:Writing seed config... 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-13T06:40:40.383 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-13T06:40:40.384 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-13T06:40:40.385 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-13T06:40:40.386 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-13T06:40:40.386 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-13T06:40:40.386 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-13T06:40:40.386 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-13T06:40:40.386 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:40:40.387 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-13T06:40:40.444 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b9295806-294f-11ef-bc9c-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-13T06:40:40.444 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service 2024-06-13T06:40:40.492 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x.service 2024-06-13T06:40:40.538 INFO:tasks.cephadm:Bootstrapping... 2024-06-13T06:40:40.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid b9295806-294f-11ef-bc9c-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:40:40.815 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-06-13T06:40:40.815 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'b9295806-294f-11ef-bc9c-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-06-13T06:40:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-13T06:40:40.816 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-06-13T06:40:40.879 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.1.0 2024-06-13T06:40:40.879 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-06-13T06:40:40.879 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-06-13T06:40:40.894 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-13T06:40:40.894 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-13T06:40:40.911 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-13T06:40:40.911 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:40:40.935 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-06-13T06:40:40.956 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-06-13T06:40:40.956 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-06-13T06:40:40.956 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-06-13T06:40:41.018 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.1.0 2024-06-13T06:40:41.018 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 5.1.0 is present 2024-06-13T06:40:41.018 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-06-13T06:40:41.018 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-06-13T06:40:41.034 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-13T06:40:41.034 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-13T06:40:41.052 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-13T06:40:41.052 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:40:41.074 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-06-13T06:40:41.094 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-06-13T06:40:41.095 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-06-13T06:40:41.095 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-06-13T06:40:41.095 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:41.095 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 139857017013632 on /run/cephadm/b9295806-294f-11ef-bc9c-c7b262605968.lock 2024-06-13T06:40:41.096 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 139857017013632 acquired on /run/cephadm/b9295806-294f-11ef-bc9c-c7b262605968.lock 2024-06-13T06:40:41.097 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-06-13T06:40:41.098 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-06-13T06:40:41.098 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-06-13T06:40:41.111 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-06-13T06:40:41.111 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-06-13T06:40:41.122 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-13T06:40:41.122 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-13T06:40:41.122 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-13T06:40:41.132 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-13T06:40:41.132 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-13T06:40:41.132 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-13T06:40:41.132 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-13T06:40:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-13T06:40:41.133 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-13T06:40:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-06-13T06:40:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-06-13T06:40:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-13T06:40:41.137 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-13T06:40:41.139 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:40:41.793 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-13T06:40:41.793 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-13T06:40:41.793 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-06-13T06:40:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-13T06:40:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-13T06:40:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-13T06:40:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-13T06:40:43.348 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-13T06:40:43.348 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-13T06:40:43.348 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-06-13T06:40:44.570 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-06-13T06:40:44.570 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-06-13T06:40:45.790 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQBtlGpmgq3fAhAA8mozROOvcbOyr5vGw0gB8w== 2024-06-13T06:40:46.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQBulGpmOjmvDxAAv8WXky4wrv8zIOM0Z0YtEQ== 2024-06-13T06:40:47.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQBvlGpmdE7oFBAAwnzlIpzu3QnEK06nzxrjtw== 2024-06-13T06:40:47.951 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-06-13T06:40:49.136 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-13T06:40:49.137 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-13T06:40:49.137 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:49.137 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:40:49.138 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-06-13T06:40:50.277 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-06-13T06:40:50.677 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-13T06:40:50.925 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-13T06:40:51.231 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target → /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target. 2024-06-13T06:40:51.231 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target → /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target. 2024-06-13T06:40:51.643 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a 2024-06-13T06:40:51.643 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service: Unit ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service not loaded. 2024-06-13T06:40:51.950 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target.wants/ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968@.service. 2024-06-13T06:40:52.638 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 podman[31690]: 2024-06-13 06:40:52.383919234 +0000 UTC m=+0.136466911 container create ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-13T06:40:52.638 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 podman[31690]: 2024-06-13 06:40:52.637374566 +0000 UTC m=+0.389922267 container init ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-13T06:40:52.759 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:40:52.759 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-13T06:40:52.775 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:40:52.775 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:40:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-13T06:40:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-06-13T06:40:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-06-13T06:40:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-06-13T06:40:52.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 podman[31690]: 2024-06-13 06:40:52.651309947 +0000 UTC m=+0.403857648 container start ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-13T06:40:52.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 bash[31690]: ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 2024-06-13T06:40:52.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 ceph-mon[31709]: mkfs b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:52.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 systemd[1]: Started Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:40:52.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:52 smithi007 ceph-mon[31709]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: monmap epoch 1 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: last_changed 2024-06-13T06:40:48.478210+0000 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: min_mon_release 19 (squid) 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: election_strategy: 1 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:40:54.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: fsmap 2024-06-13T06:40:54.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: osdmap e1: 0 total, 0 up, 0 in 2024-06-13T06:40:54.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: mgrmap e1: no daemons active 2024-06-13T06:40:54.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:53 smithi007 ceph-mon[31709]: from='client.? 172.21.15.7:0/231046920' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-06-13T06:40:54.186 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.748677s) 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-13T06:40:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.382+0000 7f4bf2400700 1 Processor -- start 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.382+0000 7f4bf2400700 1 -- start start 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.383+0000 7f4bf2400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.383+0000 7f4bf2400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1087f0 con 0x7f4bec107e90 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.384+0000 7f4beb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.384+0000 7f4beb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46966/0 (socket says 172.21.15.7:46966) 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.384+0000 7f4beb400700 1 -- 172.21.15.7:0/3235427017 learned_addr learned my addr 172.21.15.7:0/3235427017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.385+0000 7f4beb400700 1 -- 172.21.15.7:0/3235427017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec108ff0 con 0x7f4bec107e90 2024-06-13T06:40:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.385+0000 7f4beb400700 1 --2- 172.21.15.7:0/3235427017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dc86c692b84e1f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.385+0000 7f4bea000700 1 -- 172.21.15.7:0/3235427017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc004160 con 0x7f4bec107e90 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.385+0000 7f4bea000700 1 -- 172.21.15.7:0/3235427017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc004300 con 0x7f4bec107e90 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.385+0000 7f4bea000700 1 -- 172.21.15.7:0/3235427017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc0036a0 con 0x7f4bec107e90 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 -- 172.21.15.7:0/3235427017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 msgr2=0x7f4bec1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 --2- 172.21.15.7:0/3235427017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).stop 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 -- 172.21.15.7:0/3235427017 shutdown_connections 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 --2- 172.21.15.7:0/3235427017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec107e90 0x7f4bec1082b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 -- 172.21.15.7:0/3235427017 >> 172.21.15.7:0/3235427017 conn(0x7f4bec103450 msgr2=0x7f4bec105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 -- 172.21.15.7:0/3235427017 shutdown_connections 2024-06-13T06:40:54.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.386+0000 7f4bf2400700 1 -- 172.21.15.7:0/3235427017 wait complete. 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4bf2400700 1 Processor -- start 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4bf2400700 1 -- start start 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4bf2400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4bf2400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec1094c0 con 0x7f4bec19d930 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4beb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4beb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46976/0 (socket says 172.21.15.7:46976) 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.388+0000 7f4beb400700 1 -- 172.21.15.7:0/231046920 learned_addr learned my addr 172.21.15.7:0/231046920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.389+0000 7f4beb400700 1 -- 172.21.15.7:0/231046920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec19e290 con 0x7f4bec19d930 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.389+0000 7f4beb400700 1 --2- 172.21.15.7:0/231046920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.389+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc003ca0 con 0x7f4bec19d930 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc003e40 con 0x7f4bec19d930 2024-06-13T06:40:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc011780 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec19e520 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1a1000 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4bdc011920 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.390+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4bdc025d30 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.391+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec063af0 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.394+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4bdc004260 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.475+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4bec1a17a0 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.476+0000 7f4be3e00700 1 -- 172.21.15.7:0/231046920 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4bdc011da0 con 0x7f4bec19d930 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 msgr2=0x7f4bec19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 --2- 172.21.15.7:0/231046920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004750 comp rx=0 tx=0).stop 2024-06-13T06:40:54.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 shutdown_connections 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 --2- 172.21.15.7:0/231046920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec19d930 0x7f4bec19dd50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 >> 172.21.15.7:0/231046920 conn(0x7f4bec103450 msgr2=0x7f4bec07d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 shutdown_connections 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:53.478+0000 7f4bf2400700 1 -- 172.21.15.7:0/231046920 wait complete. 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-06-13T06:40:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-06-13T06:40:55.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:54 smithi007 ceph-mon[31709]: from='client.? 172.21.15.7:0/1134353522' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-13T06:40:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:54 smithi007 ceph-mon[31709]: from='client.? 172.21.15.7:0/1134353522' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-13T06:40:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-06-13T06:40:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-06-13T06:40:55.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-13T06:40:55.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.822+0000 7fdc22c00700 1 Processor -- start 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.822+0000 7fdc22c00700 1 -- start start 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.823+0000 7fdc22c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.823+0000 7fdc22c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c07c660 con 0x7fdc1c1087f0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.823+0000 7fdc21800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.823+0000 7fdc21800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46986/0 (socket says 172.21.15.7:46986) 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.823+0000 7fdc21800700 1 -- 172.21.15.7:0/1643906589 learned_addr learned my addr 172.21.15.7:0/1643906589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.824+0000 7fdc21800700 1 -- 172.21.15.7:0/1643906589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c10b120 con 0x7fdc1c1087f0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.824+0000 7fdc21800700 1 --2- 172.21.15.7:0/1643906589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc100098a0 tx=0x7fdc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aff41b8ac6b6a4b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.824+0000 7fdc1be00700 1 -- 172.21.15.7:0/1643906589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc10004160 con 0x7fdc1c1087f0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.824+0000 7fdc1be00700 1 -- 172.21.15.7:0/1643906589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10004300 con 0x7fdc1c1087f0 2024-06-13T06:40:55.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.824+0000 7fdc1be00700 1 -- 172.21.15.7:0/1643906589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc100036a0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 -- 172.21.15.7:0/1643906589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 msgr2=0x7fdc1c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 --2- 172.21.15.7:0/1643906589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdc100098a0 tx=0x7fdc10009bb0 comp rx=0 tx=0).stop 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 -- 172.21.15.7:0/1643906589 shutdown_connections 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 --2- 172.21.15.7:0/1643906589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c10abe0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 -- 172.21.15.7:0/1643906589 >> 172.21.15.7:0/1643906589 conn(0x7fdc1c102220 msgr2=0x7fdc1c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 -- 172.21.15.7:0/1643906589 shutdown_connections 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.825+0000 7fdc22c00700 1 -- 172.21.15.7:0/1643906589 wait complete. 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.826+0000 7fdc22c00700 1 Processor -- start 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc22c00700 1 -- start start 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc22c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc22c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c10b530 con 0x7fdc1c1087f0 2024-06-13T06:40:55.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc21800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc21800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46988/0 (socket says 172.21.15.7:46988) 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.827+0000 7fdc21800700 1 -- 172.21.15.7:0/1134353522 learned_addr learned my addr 172.21.15.7:0/1134353522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc21800700 1 -- 172.21.15.7:0/1134353522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c19dad0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc21800700 1 --2- 172.21.15.7:0/1134353522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000c00 tx=0x7fdc10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc10003db0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc10011630 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc10011950 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.828+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c19dd60 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.829+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a0a50 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.829+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc1001a460 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.829+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc1001b780 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.830+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c063af0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.833+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdc1001b980 con 0x7fdc1c1087f0 2024-06-13T06:40:55.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.911+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdc1c02def0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.920+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fdc1001bb60 con 0x7fdc1c1087f0 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.921+0000 7fdc1a000700 1 -- 172.21.15.7:0/1134353522 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc1001ada0 con 0x7fdc1c1087f0 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 msgr2=0x7fdc1c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 --2- 172.21.15.7:0/1134353522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000c00 tx=0x7fdc10004750 comp rx=0 tx=0).stop 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 shutdown_connections 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 --2- 172.21.15.7:0/1134353522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc1c1087f0 0x7fdc1c19d590 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 >> 172.21.15.7:0/1134353522 conn(0x7fdc1c102220 msgr2=0x7fdc1c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 shutdown_connections 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:54.923+0000 7fdc22c00700 1 -- 172.21.15.7:0/1134353522 wait complete. 2024-06-13T06:40:55.501 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-06-13T06:40:56.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:56 smithi007 ceph-mon[31709]: from='client.? 172.21.15.7:0/2184046025' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:40:56.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.120+0000 7f5567e00700 1 Processor -- start 2024-06-13T06:40:56.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.120+0000 7f5567e00700 1 -- start start 2024-06-13T06:40:56.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.121+0000 7f5567e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:56.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.121+0000 7f5567e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f556810ab20 con 0x7f556810a130 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.121+0000 7f5566a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.121+0000 7f5566a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47004/0 (socket says 172.21.15.7:47004) 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.121+0000 7f5566a00700 1 -- 172.21.15.7:0/2542426838 learned_addr learned my addr 172.21.15.7:0/2542426838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.122+0000 7f5566a00700 1 -- 172.21.15.7:0/2542426838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556810aff0 con 0x7f556810a130 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.122+0000 7f5566a00700 1 --2- 172.21.15.7:0/2542426838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f55580098a0 tx=0x7f5558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4c47f676e6149 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.122+0000 7f5566000700 1 -- 172.21.15.7:0/2542426838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558004160 con 0x7f556810a130 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5566000700 1 -- 172.21.15.7:0/2542426838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5558004300 con 0x7f556810a130 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5566000700 1 -- 172.21.15.7:0/2542426838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558011b50 con 0x7f556810a130 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 -- 172.21.15.7:0/2542426838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 msgr2=0x7f556810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:56.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 --2- 172.21.15.7:0/2542426838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f55580098a0 tx=0x7f5558009bb0 comp rx=0 tx=0).stop 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 -- 172.21.15.7:0/2542426838 shutdown_connections 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 --2- 172.21.15.7:0/2542426838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556810a550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 -- 172.21.15.7:0/2542426838 >> 172.21.15.7:0/2542426838 conn(0x7f556807d2b0 msgr2=0x7f556807d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 -- 172.21.15.7:0/2542426838 shutdown_connections 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.123+0000 7f5567e00700 1 -- 172.21.15.7:0/2542426838 wait complete. 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.124+0000 7f5567e00700 1 Processor -- start 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5567e00700 1 -- start start 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5567e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5567e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f556810e2b0 con 0x7f556810a130 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5566a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5566a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47016/0 (socket says 172.21.15.7:47016) 2024-06-13T06:40:56.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.125+0000 7f5566a00700 1 -- 172.21.15.7:0/2184046025 learned_addr learned my addr 172.21.15.7:0/2184046025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f5566a00700 1 -- 172.21.15.7:0/2184046025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819e390 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f5566a00700 1 --2- 172.21.15.7:0/2184046025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5558011b60 tx=0x7f5558011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558003970 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5558003b10 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558024610 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.126+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819e620 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.127+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a1310 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.127+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5558024c50 con 0x7f556810a130 2024-06-13T06:40:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.127+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5558011e00 con 0x7f556810a130 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.128+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556810e430 con 0x7f556810a130 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.131+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f555801a9f0 con 0x7f556810a130 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.209+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5568063af0 con 0x7f556810a130 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.210+0000 7f555fe00700 1 -- 172.21.15.7:0/2184046025 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f555801a430 con 0x7f556810a130 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 msgr2=0x7f556819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 --2- 172.21.15.7:0/2184046025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5558011b60 tx=0x7f5558011c40 comp rx=0 tx=0).stop 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 shutdown_connections 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 --2- 172.21.15.7:0/2184046025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556810a130 0x7f556819de50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 >> 172.21.15.7:0/2184046025 conn(0x7f556807d2b0 msgr2=0x7f5568107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.211+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 shutdown_connections 2024-06-13T06:40:56.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:40:56.212+0000 7f5567e00700 1 -- 172.21.15.7:0/2184046025 wait complete. 2024-06-13T06:40:56.868 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-06-13T06:40:57.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:56 smithi007 systemd[1]: Stopping Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:40:57.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a[31705]: 2024-06-13T06:40:57.403+0000 7fa99b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T06:40:57.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a[31705]: 2024-06-13T06:40:57.403+0000 7fa99b200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-13T06:40:57.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:57 smithi007 podman[31952]: 2024-06-13 06:40:57.663094379 +0000 UTC m=+0.398568538 container died ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-13T06:40:58.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 podman[31952]: 2024-06-13 06:40:58.191204013 +0000 UTC m=+0.926678223 container cleanup ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-13T06:40:58.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 bash[31952]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a 2024-06-13T06:40:58.670 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 podman[31965]: 2024-06-13 06:40:58.448729439 +0000 UTC m=+0.765665495 container remove ed2514df2000924755b1936a4e9de6f44ec553ccaf8ce8cc95b148a0252e4b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:40:58.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 systemd[1]: ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-13T06:40:58.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 systemd[1]: Stopped Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:40:58.927 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 systemd[1]: ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service: Consumed 2.010s CPU time. 2024-06-13T06:40:59.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:58 smithi007 systemd[1]: Starting Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:40:59.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 podman[32059]: 2024-06-13 06:40:59.217904031 +0000 UTC m=+0.032498395 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:40:59.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 podman[32059]: 2024-06-13 06:40:59.315998719 +0000 UTC m=+0.130593145 container create 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-13T06:40:59.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 podman[32059]: 2024-06-13 06:40:59.552660816 +0000 UTC m=+0.367255262 container init 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-13T06:40:59.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 podman[32059]: 2024-06-13 06:40:59.564685123 +0000 UTC m=+0.379279530 container start 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:40:59.661 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-13T06:40:59.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: set uid:gid to 167:167 (ceph:ceph) 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: pidfile_write: ignore empty --pid-file 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: load: jerasure load: lrc 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: RocksDB version: 7.9.2 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Git sha 0 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: DB SUMMARY 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: DB Session ID: WHC70S4VSASNLU42IVMZ 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: CURRENT file: CURRENT 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: IDENTITY file: IDENTITY 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-13T06:40:59.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104244 ; 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.error_if_exists: 0 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.create_if_missing: 0 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.paranoid_checks: 1 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.env: 0x55f78f112c60 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.fs: PosixFileSystem 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.info_log: 0x55f7bae4fb60 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_file_opening_threads: 16 2024-06-13T06:40:59.821 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.statistics: (nil) 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.use_fsync: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_log_file_size: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.keep_log_file_num: 1000 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.recycle_log_file_num: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_fallocate: 1 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_mmap_reads: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_mmap_writes: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.use_direct_reads: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.create_missing_column_families: 0 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.db_log_dir: 2024-06-13T06:40:59.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.wal_dir: 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-13T06:40:59.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.advise_random_on_open: 1 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.db_write_buffer_size: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.write_buffer_manager: 0x55f7bae201e0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.rate_limiter: (nil) 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.wal_recovery_mode: 2 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enable_thread_tracking: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enable_pipelined_write: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.unordered_write: 0 2024-06-13T06:40:59.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.row_cache: None 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.wal_filter: None 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_ingest_behind: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.two_write_queues: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.manual_wal_flush: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.wal_compression: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.atomic_flush: 0 2024-06-13T06:40:59.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.log_readahead_size: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.best_efforts_recovery: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.allow_data_in_errors: 0 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.db_host_id: __hostname__ 2024-06-13T06:40:59.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-13T06:40:59.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_background_jobs: 2 2024-06-13T06:40:59.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_background_compactions: -1 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_subcompactions: 1 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_total_wal_size: 0 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_open_files: -1 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bytes_per_sync: 0 2024-06-13T06:40:59.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_readahead_size: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_background_flushes: -1 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Compression algorithms supported: 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kZSTD supported: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kXpressCompression supported: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kLZ4HCCompression supported: 1 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kZlibCompression supported: 1 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kSnappyCompression supported: 1 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kLZ4Compression supported: 1 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: kBZip2Compression supported: 0 2024-06-13T06:40:59.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.merge_operator: 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_filter: None 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_filter_factory: None 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.sst_partitioner_factory: None 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f7bae4fc80) 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-06-13T06:40:59.830 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x55f7bae51090 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-06-13T06:40:59.831 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-06-13T06:40:59.832 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.write_buffer_size: 33554432 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_write_buffer_number: 2 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression: NoCompression 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression: Disabled 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.prefix_extractor: nullptr 2024-06-13T06:40:59.833 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.num_levels: 7 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:40:59.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.level: 32767 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.strategy: 0 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.enabled: false 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.target_file_size_base: 67108864 2024-06-13T06:40:59.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.arena_block_size: 1048576 2024-06-13T06:40:59.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.disable_auto_compactions: 0 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-13T06:40:59.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.table_properties_collectors: 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.inplace_update_support: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.bloom_locality: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.max_successive_merges: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.paranoid_file_checks: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.force_consistency_checks: 1 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.report_bg_io_stats: 0 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.ttl: 2592000 2024-06-13T06:40:59.838 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enable_blob_files: false 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.min_blob_size: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_file_size: 268435456 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.blob_file_starting_level: 0 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-13T06:40:59.839 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: faf218fb-2193-4495-8ebe-7830aa295c28 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260859624151, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260859625962, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99403, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97311, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14054, "raw_average_key_size": 49, "raw_value_size": 89685, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718260859, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "faf218fb-2193-4495-8ebe-7830aa295c28", "db_session_id": "WHC70S4VSASNLU42IVMZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260859626216, "job": 1, "event": "recovery_finished"} 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f7baf40000 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: DB pointer 0x55f7baf2a000 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-13T06:40:59.840 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 98.91 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 98.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:40:59.841 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 14.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 14.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-13T06:40:59.842 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x55f7bae51090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???) e1 preinit fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).mds e1 new map 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).mds e1 print_map 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-06-13T06:40:59.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 bash[32059]: 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 systemd[1]: Started Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: monmap epoch 1 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: last_changed 2024-06-13T06:40:48.478210+0000 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: min_mon_release 19 (squid) 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: election_strategy: 1 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:40:59.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: fsmap 2024-06-13T06:40:59.845 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: osdmap e1: 0 total, 0 up, 0 in 2024-06-13T06:40:59.845 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:40:59 smithi007 ceph-mon[32077]: mgrmap e1: no daemons active 2024-06-13T06:41:01.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.346+0000 7f1152400700 1 Processor -- start 2024-06-13T06:41:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.346+0000 7f1152400700 1 -- start start 2024-06-13T06:41:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.347+0000 7f1152400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.347+0000 7f1152400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f114c07c800 con 0x7f114c07de10 2024-06-13T06:41:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.348+0000 7f1151000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.348+0000 7f1151000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47030/0 (socket says 172.21.15.7:47030) 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.348+0000 7f1151000700 1 -- 172.21.15.7:0/2047732464 learned_addr learned my addr 172.21.15.7:0/2047732464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.348+0000 7f1151000700 1 -- 172.21.15.7:0/2047732464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c07c980 con 0x7f114c07de10 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.349+0000 7f1151000700 1 --2- 172.21.15.7:0/2047732464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6fb7e1e36532be server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.349+0000 7f114b400700 1 -- 172.21.15.7:0/2047732464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1140004160 con 0x7f114c07de10 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.349+0000 7f114b400700 1 -- 172.21.15.7:0/2047732464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1140004300 con 0x7f114c07de10 2024-06-13T06:41:01.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.349+0000 7f114b400700 1 -- 172.21.15.7:0/2047732464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1140011b50 con 0x7f114c07de10 2024-06-13T06:41:01.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 -- 172.21.15.7:0/2047732464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 msgr2=0x7f114c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:01.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 --2- 172.21.15.7:0/2047732464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:01.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 -- 172.21.15.7:0/2047732464 shutdown_connections 2024-06-13T06:41:01.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 --2- 172.21.15.7:0/2047732464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c07c2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:01.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 -- 172.21.15.7:0/2047732464 >> 172.21.15.7:0/2047732464 conn(0x7f114c103260 msgr2=0x7f114c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:01.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 -- 172.21.15.7:0/2047732464 shutdown_connections 2024-06-13T06:41:01.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.350+0000 7f1152400700 1 -- 172.21.15.7:0/2047732464 wait complete. 2024-06-13T06:41:01.064 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.351+0000 7f1152400700 1 Processor -- start 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1152400700 1 -- start start 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1152400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1152400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f114c10e1d0 con 0x7f114c07de10 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1151000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1151000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47032/0 (socket says 172.21.15.7:47032) 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.352+0000 7f1151000700 1 -- 172.21.15.7:0/1495330128 learned_addr learned my addr 172.21.15.7:0/1495330128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.353+0000 7f1151000700 1 -- 172.21.15.7:0/1495330128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c19e390 con 0x7f114c07de10 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.353+0000 7f1151000700 1 --2- 172.21.15.7:0/1495330128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1140011b60 tx=0x7f1140011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.353+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1140003970 con 0x7f114c07de10 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.353+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1140003b10 con 0x7f114c07de10 2024-06-13T06:41:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.354+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c19e620 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.354+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1140024610 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.354+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a1310 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.354+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1140024ca0 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.354+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1140011e00 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.355+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c063af0 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.358+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f114001a770 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.436+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f114c1a1580 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.441+0000 7f1149600700 1 -- 172.21.15.7:0/1495330128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f114001a9f0 con 0x7f114c07de10 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.442+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 msgr2=0x7f114c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.442+0000 7f1152400700 1 --2- 172.21.15.7:0/1495330128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1140011b60 tx=0x7f1140011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:01.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.442+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 shutdown_connections 2024-06-13T06:41:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.442+0000 7f1152400700 1 --2- 172.21.15.7:0/1495330128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c07de10 0x7f114c19de50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.442+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 >> 172.21.15.7:0/1495330128 conn(0x7f114c103260 msgr2=0x7f114c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.443+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 shutdown_connections 2024-06-13T06:41:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:00.443+0000 7f1152400700 1 -- 172.21.15.7:0/1495330128 wait complete. 2024-06-13T06:41:01.068 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-13T06:41:01.097 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:41:01.097 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-06-13T06:41:01.100 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-13T06:41:01.101 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-13T06:41:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x 2024-06-13T06:41:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x.service: Unit ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x.service not loaded. 2024-06-13T06:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968.target.wants/ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b9295806-294f-11ef-bc9c-c7b262605968@.service. 2024-06-13T06:41:01.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:01 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1495330128' entity='client.admin' 2024-06-13T06:41:02.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:01 smithi007 systemd[1]: Starting Ceph mgr.x for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:41:02.569 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 podman[32288]: 2024-06-13 06:41:02.163784291 +0000 UTC m=+0.042293413 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-13T06:41:02.569 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 podman[32288]: 2024-06-13 06:41:02.259432407 +0000 UTC m=+0.137941526 container create 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:41:02.569 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 podman[32288]: 2024-06-13 06:41:02.488114582 +0000 UTC m=+0.366623736 container init 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-13T06:41:02.569 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 podman[32288]: 2024-06-13 06:41:02.498790433 +0000 UTC m=+0.377299553 container start 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528) 2024-06-13T06:41:02.569 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 bash[32288]: 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e 2024-06-13T06:41:02.600 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:41:02.600 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-13T06:41:02.617 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:41:02.617 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:41:02.618 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-13T06:41:02.618 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-06-13T06:41:02.648 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:41:02.648 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-06-13T06:41:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-06-13T06:41:02.826 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 systemd[1]: Started Ceph mgr.x for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:41:02.827 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:02 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:02.756+0000 7f924da3d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:41:03.106 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:03.026+0000 7f924da3d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:03.363 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:03.173+0000 7f924da3d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:41:03.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:03 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3132381981' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:41:03.663 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:03.611+0000 7f924da3d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:41:03.958 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:03.728+0000 7f924da3d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:41:03.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:41:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:41:03.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:41:03.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:40:52.732096+0000", 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.270+0000 7fa239400700 1 Processor -- start 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.271+0000 7fa239400700 1 -- start start 2024-06-13T06:41:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.271+0000 7fa239400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.272+0000 7fa239400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa23407c8f0 con 0x7fa23407dec0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.272+0000 7fa232800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.272+0000 7fa232800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52166/0 (socket says 172.21.15.7:52166) 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.272+0000 7fa232800700 1 -- 172.21.15.7:0/2432280704 learned_addr learned my addr 172.21.15.7:0/2432280704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.273+0000 7fa232800700 1 -- 172.21.15.7:0/2432280704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23407ca70 con 0x7fa23407dec0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.273+0000 7fa232800700 1 --2- 172.21.15.7:0/2432280704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2240098a0 tx=0x7fa224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0f89b685fa05ea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.273+0000 7fa231400700 1 -- 172.21.15.7:0/2432280704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa224004160 con 0x7fa23407dec0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.273+0000 7fa231400700 1 -- 172.21.15.7:0/2432280704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa224004300 con 0x7fa23407dec0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.273+0000 7fa231400700 1 -- 172.21.15.7:0/2432280704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa224011b50 con 0x7fa23407dec0 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 -- 172.21.15.7:0/2432280704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 msgr2=0x7fa23407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 --2- 172.21.15.7:0/2432280704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2240098a0 tx=0x7fa224009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 -- 172.21.15.7:0/2432280704 shutdown_connections 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 --2- 172.21.15.7:0/2432280704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa23407c320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:03.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 -- 172.21.15.7:0/2432280704 >> 172.21.15.7:0/2432280704 conn(0x7fa234103400 msgr2=0x7fa234105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.274+0000 7fa239400700 1 -- 172.21.15.7:0/2432280704 shutdown_connections 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.275+0000 7fa239400700 1 -- 172.21.15.7:0/2432280704 wait complete. 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa239400700 1 Processor -- start 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa239400700 1 -- start start 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa239400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa239400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa234109ff0 con 0x7fa23407dec0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa232800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa232800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52176/0 (socket says 172.21.15.7:52176) 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.276+0000 7fa232800700 1 -- 172.21.15.7:0/3132381981 learned_addr learned my addr 172.21.15.7:0/3132381981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.277+0000 7fa232800700 1 -- 172.21.15.7:0/3132381981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341a2900 con 0x7fa23407dec0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.277+0000 7fa232800700 1 --2- 172.21.15.7:0/3132381981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.277+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa224003b20 con 0x7fa23407dec0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.277+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa224003cc0 con 0x7fa23407dec0 2024-06-13T06:41:03.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.277+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341a2b90 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.278+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2240248a0 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.278+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341a5880 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.278+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa224022020 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.278+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa224011e00 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.279+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa234063af0 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.282+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa22402dcb0 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.363+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa23419b920 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.364+0000 7fa22b400700 1 -- 172.21.15.7:0/3132381981 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa23419b920 con 0x7fa23407dec0 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 msgr2=0x7fa2341a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 --2- 172.21.15.7:0/3132381981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224011c10 comp rx=0 tx=0).stop 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 shutdown_connections 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 --2- 172.21.15.7:0/3132381981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa23407dec0 0x7fa2341a23c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 >> 172.21.15.7:0/3132381981 conn(0x7fa234103400 msgr2=0x7fa2341057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:03.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 shutdown_connections 2024-06-13T06:41:03.989 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:03.365+0000 7fa239400700 1 -- 172.21.15.7:0/3132381981 wait complete. 2024-06-13T06:41:03.989 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-06-13T06:41:05.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:05.366+0000 7f924da3d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:41:05.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:05.598+0000 7f924da3d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:06.134 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:05.816+0000 7f924da3d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:41:06.387 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:06.214+0000 7f924da3d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:41:06.387 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:06.343+0000 7f924da3d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:41:06.640 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:06.606+0000 7f924da3d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:41:06.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:06 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2884003420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:41:06.913 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:06.873+0000 7f924da3d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:41:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:41:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:41:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:41:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:41:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:41:07.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:41:07.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:40:52.732096+0000", 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:41:07.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.548+0000 7f42e4c00700 1 Processor -- start 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.548+0000 7f42e4c00700 1 -- start start 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42e4c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42e4c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42e007be10 con 0x7f42e0108780 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52184/0 (socket says 172.21.15.7:52184) 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42df400700 1 -- 172.21.15.7:0/1318695715 learned_addr learned my addr 172.21.15.7:0/1318695715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.549+0000 7f42df400700 1 -- 172.21.15.7:0/1318695715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e010b0b0 con 0x7f42e0108780 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.550+0000 7f42df400700 1 --2- 172.21.15.7:0/1318695715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42d00098a0 tx=0x7f42d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49be9f265c8c06df server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.550+0000 7f42de000700 1 -- 172.21.15.7:0/1318695715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0004160 con 0x7f42e0108780 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.550+0000 7f42de000700 1 -- 172.21.15.7:0/1318695715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42d0004300 con 0x7f42e0108780 2024-06-13T06:41:07.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.550+0000 7f42de000700 1 -- 172.21.15.7:0/1318695715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0011b50 con 0x7f42e0108780 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 -- 172.21.15.7:0/1318695715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 msgr2=0x7f42e010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 --2- 172.21.15.7:0/1318695715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42d00098a0 tx=0x7f42d0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 -- 172.21.15.7:0/1318695715 shutdown_connections 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 --2- 172.21.15.7:0/1318695715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e010ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 -- 172.21.15.7:0/1318695715 >> 172.21.15.7:0/1318695715 conn(0x7f42e0102130 msgr2=0x7f42e0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 -- 172.21.15.7:0/1318695715 shutdown_connections 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.551+0000 7f42e4c00700 1 -- 172.21.15.7:0/1318695715 wait complete. 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.553+0000 7f42e4c00700 1 Processor -- start 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.553+0000 7f42e4c00700 1 -- start start 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.553+0000 7f42e4c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.553+0000 7f42e4c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010c000 con 0x7f42e0108780 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.554+0000 7f42df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.554+0000 7f42df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52194/0 (socket says 172.21.15.7:52194) 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.554+0000 7f42df400700 1 -- 172.21.15.7:0/2884003420 learned_addr learned my addr 172.21.15.7:0/2884003420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.554+0000 7f42df400700 1 -- 172.21.15.7:0/2884003420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e019e3a0 con 0x7f42e0108780 2024-06-13T06:41:07.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.554+0000 7f42df400700 1 --2- 172.21.15.7:0/2884003420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42d0011b60 tx=0x7f42d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0003b20 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42d0003cc0 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e019e630 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0024840 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e01a1320 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42d0022020 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.555+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42d0011e00 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.556+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e0063af0 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.559+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f42d0043dd0 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.639+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f42e01a14f0 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.640+0000 7f42d7e00700 1 -- 172.21.15.7:0/2884003420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f42d0032030 con 0x7f42e0108780 2024-06-13T06:41:07.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 msgr2=0x7f42e019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 --2- 172.21.15.7:0/2884003420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f42d0011b60 tx=0x7f42d0011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 shutdown_connections 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 --2- 172.21.15.7:0/2884003420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e0108780 0x7f42e019de60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 >> 172.21.15.7:0/2884003420 conn(0x7f42e0102130 msgr2=0x7f42e0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 shutdown_connections 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:06.642+0000 7f42e4c00700 1 -- 172.21.15.7:0/2884003420 wait complete. 2024-06-13T06:41:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-06-13T06:41:08.162 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:07.699+0000 7f924da3d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:41:08.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:08.269+0000 7f924da3d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:41:08.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:08.401+0000 7f924da3d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:41:09.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:08.674+0000 7f924da3d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:41:09.598 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:09.343+0000 7f924da3d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:41:09.598 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:09.473+0000 7f924da3d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:41:09.886 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:09.814+0000 7f924da3d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:41:10.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:10 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1327319542' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:41:10.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:10.004+0000 7f924da3d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:41:10.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:10.136+0000 7f924da3d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:41:10.561 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:10.297+0000 7f924da3d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:10.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:41:10.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:41:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:41:10.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:41:10.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:41:10.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:41:10.582 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:40:52.732096+0000", 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.887+0000 7f80d7a00700 1 Processor -- start 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.887+0000 7f80d7a00700 1 -- start start 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.887+0000 7f80d7a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.887+0000 7f80d7a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80d007c890 con 0x7f80d007de60 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.888+0000 7f80d5000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.888+0000 7f80d5000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52200/0 (socket says 172.21.15.7:52200) 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.888+0000 7f80d5000700 1 -- 172.21.15.7:0/281014422 learned_addr learned my addr 172.21.15.7:0/281014422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:10.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.888+0000 7f80d5000700 1 -- 172.21.15.7:0/281014422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d007ca10 con 0x7f80d007de60 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.889+0000 7f80d5000700 1 --2- 172.21.15.7:0/281014422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80c40098a0 tx=0x7f80c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7934e37e99b542a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.889+0000 7f80cf400700 1 -- 172.21.15.7:0/281014422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80c4004160 con 0x7f80d007de60 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.889+0000 7f80cf400700 1 -- 172.21.15.7:0/281014422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80c4004300 con 0x7f80d007de60 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.889+0000 7f80cf400700 1 -- 172.21.15.7:0/281014422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80c4011b50 con 0x7f80d007de60 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 -- 172.21.15.7:0/281014422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 msgr2=0x7f80d007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 --2- 172.21.15.7:0/281014422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80c40098a0 tx=0x7f80c4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 -- 172.21.15.7:0/281014422 shutdown_connections 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 --2- 172.21.15.7:0/281014422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d007c2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 -- 172.21.15.7:0/281014422 >> 172.21.15.7:0/281014422 conn(0x7f80d0103450 msgr2=0x7f80d0105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 -- 172.21.15.7:0/281014422 shutdown_connections 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.890+0000 7f80d7a00700 1 -- 172.21.15.7:0/281014422 wait complete. 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.891+0000 7f80d7a00700 1 Processor -- start 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.891+0000 7f80d7a00700 1 -- start start 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.891+0000 7f80d7a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.891+0000 7f80d7a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010e460 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.892+0000 7f80d5000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.892+0000 7f80d5000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52214/0 (socket says 172.21.15.7:52214) 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.892+0000 7f80d5000700 1 -- 172.21.15.7:0/1327319542 learned_addr learned my addr 172.21.15.7:0/1327319542 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.892+0000 7f80d5000700 1 -- 172.21.15.7:0/1327319542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d019e4e0 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.892+0000 7f80d5000700 1 --2- 172.21.15.7:0/1327319542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f80c4011b60 tx=0x7f80c4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80c4003b20 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80c4003cc0 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80c4024840 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d019e770 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d01a1460 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80c4022020 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.893+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80c4011e00 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.894+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d0063af0 con 0x7f80d007de60 2024-06-13T06:41:10.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.898+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f80c4043dd0 con 0x7f80d007de60 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.978+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f80d01a1b40 con 0x7f80d007de60 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.978+0000 7f80cd600700 1 -- 172.21.15.7:0/1327319542 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f80c4032020 con 0x7f80d007de60 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.979+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 msgr2=0x7f80d019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.979+0000 7f80d7a00700 1 --2- 172.21.15.7:0/1327319542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f80c4011b60 tx=0x7f80c4011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.979+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 shutdown_connections 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.979+0000 7f80d7a00700 1 --2- 172.21.15.7:0/1327319542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d007de60 0x7f80d019dfa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.979+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 >> 172.21.15.7:0/1327319542 conn(0x7f80d0103450 msgr2=0x7f80d0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.980+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 shutdown_connections 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:09.980+0000 7f80d7a00700 1 -- 172.21.15.7:0/1327319542 wait complete. 2024-06-13T06:41:10.587 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-06-13T06:41:10.913 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:10.676+0000 7f924da3d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:41:11.755 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:11.462+0000 7f924da3d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:41:12.162 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:11.756+0000 7f924da3d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: Activating manager daemon x 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: mgrmap e2: x(active, starting, since 0.0115879s) 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:41:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:41:12.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: Manager daemon x is now available 2024-06-13T06:41:12.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:41:12.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:11 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:41:13.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:12 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' 2024-06-13T06:41:13.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:12 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' 2024-06-13T06:41:13.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:12 smithi007 ceph-mon[32077]: from='mgr.14100 172.21.15.7:0/1600273197' entity='mgr.x' 2024-06-13T06:41:13.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:12 smithi007 ceph-mon[32077]: mgrmap e3: x(active, since 1.01885s) 2024-06-13T06:41:14.120 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:13 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4147384396' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-13T06:41:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-13T06:41:14.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-13T06:41:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-13T06:41:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-13T06:41:14.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-13T06:41:14.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-13T06:41:14.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-13T06:41:14.403 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:14.404 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.405 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-06-13T06:40:52.732096+0000", 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.214+0000 7fc46a000700 1 Processor -- start 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.215+0000 7fc46a000700 1 -- start start 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.215+0000 7fc46a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.215+0000 7fc46a000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc46407c800 con 0x7fc46407dd50 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.215+0000 7fc468c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.216+0000 7fc468c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49636/0 (socket says 172.21.15.7:49636) 2024-06-13T06:41:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.216+0000 7fc468c00700 1 -- 172.21.15.7:0/3066987431 learned_addr learned my addr 172.21.15.7:0/3066987431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.216+0000 7fc468c00700 1 -- 172.21.15.7:0/3066987431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46407c980 con 0x7fc46407dd50 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.216+0000 7fc468c00700 1 --2- 172.21.15.7:0/3066987431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e80 tx=0x7fc45800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d819d5dc2feb1565 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.217+0000 7fc463400700 1 -- 172.21.15.7:0/3066987431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458017070 con 0x7fc46407dd50 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.217+0000 7fc463400700 1 -- 172.21.15.7:0/3066987431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc45800b8c0 con 0x7fc46407dd50 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.217+0000 7fc463400700 1 -- 172.21.15.7:0/3066987431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4580045c0 con 0x7fc46407dd50 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 -- 172.21.15.7:0/3066987431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 msgr2=0x7fc46407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 --2- 172.21.15.7:0/3066987431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc458009e80 tx=0x7fc45800b330 comp rx=0 tx=0).stop 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 -- 172.21.15.7:0/3066987431 shutdown_connections 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 --2- 172.21.15.7:0/3066987431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46407c2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 -- 172.21.15.7:0/3066987431 >> 172.21.15.7:0/3066987431 conn(0x7fc4641032c0 msgr2=0x7fc464105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 -- 172.21.15.7:0/3066987431 shutdown_connections 2024-06-13T06:41:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.218+0000 7fc46a000700 1 -- 172.21.15.7:0/3066987431 wait complete. 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.219+0000 7fc46a000700 1 Processor -- start 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.219+0000 7fc46a000700 1 -- start start 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc46a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc46a000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc46410e120 con 0x7fc46407dd50 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc468c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc468c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49640/0 (socket says 172.21.15.7:49640) 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc468c00700 1 -- 172.21.15.7:0/4147384396 learned_addr learned my addr 172.21.15.7:0/4147384396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.220+0000 7fc468c00700 1 -- 172.21.15.7:0/4147384396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46419e320 con 0x7fc46407dd50 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc468c00700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc458009530 tx=0x7fc45800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458020020 con 0x7fc46407dd50 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc458003b50 con 0x7fc46407dd50 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc458017040 con 0x7fc46407dd50 2024-06-13T06:41:14.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46419e5b0 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.221+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a12a0 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.222+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fc458023020 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.222+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464063af0 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.223+0000 7fc461600700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 0x7fc44c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.223+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc45800f050 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.223+0000 7fc463e00700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 0x7fc44c03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.224+0000 7fc463e00700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 0x7fc44c03b3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc450006fd0 tx=0x7fc450006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.229+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc45802a020 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.547+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc4641a1470 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.548+0000 7fc461600700 1 -- 172.21.15.7:0/4147384396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc4641a1470 con 0x7fc46407dd50 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 msgr2=0x7fc44c03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 0x7fc44c03b3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc450006fd0 tx=0x7fc450006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 msgr2=0x7fc46419dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc458009530 tx=0x7fc45800bd50 comp rx=0 tx=0).stop 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 shutdown_connections 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fc44c038f30 0x7fc44c03b3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 --2- 172.21.15.7:0/4147384396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46407dd50 0x7fc46419dde0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.551+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 >> 172.21.15.7:0/4147384396 conn(0x7fc4641032c0 msgr2=0x7fc464103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.552+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 shutdown_connections 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:13.552+0000 7fc46a000700 1 -- 172.21.15.7:0/4147384396 wait complete. 2024-06-13T06:41:14.410 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-06-13T06:41:14.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:14 smithi007 ceph-mon[32077]: mgrmap e4: x(active, since 2s) 2024-06-13T06:41:15.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:15 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3365765813' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-13T06:41:15.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:15.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-06-13T06:41:15.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:15.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-13T06:41:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.990+0000 7f5705200700 1 Processor -- start 2024-06-13T06:41:15.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.990+0000 7f5705200700 1 -- start start 2024-06-13T06:41:15.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.991+0000 7f5705200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:15.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.991+0000 7f5705200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f570007be50 con 0x7f5700108950 2024-06-13T06:41:15.902 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.991+0000 7f56fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.991+0000 7f56fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49650/0 (socket says 172.21.15.7:49650) 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.991+0000 7f56fde00700 1 -- 172.21.15.7:0/3866884253 learned_addr learned my addr 172.21.15.7:0/3866884253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.992+0000 7f56fde00700 1 -- 172.21.15.7:0/3866884253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570010b280 con 0x7f5700108950 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.992+0000 7f56fde00700 1 --2- 172.21.15.7:0/3866884253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56f00098a0 tx=0x7f56f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623227ad8cceed9f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.992+0000 7f56fca00700 1 -- 172.21.15.7:0/3866884253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0004160 con 0x7f5700108950 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f56fca00700 1 -- 172.21.15.7:0/3866884253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56f0004300 con 0x7f5700108950 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f56fca00700 1 -- 172.21.15.7:0/3866884253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0011b50 con 0x7f5700108950 2024-06-13T06:41:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 -- 172.21.15.7:0/3866884253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 msgr2=0x7f570010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 --2- 172.21.15.7:0/3866884253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56f00098a0 tx=0x7f56f0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 -- 172.21.15.7:0/3866884253 shutdown_connections 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 --2- 172.21.15.7:0/3866884253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570010ad40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 -- 172.21.15.7:0/3866884253 >> 172.21.15.7:0/3866884253 conn(0x7f5700102360 msgr2=0x7f5700104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.993+0000 7f5705200700 1 -- 172.21.15.7:0/3866884253 shutdown_connections 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.994+0000 7f5705200700 1 -- 172.21.15.7:0/3866884253 wait complete. 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.994+0000 7f5705200700 1 Processor -- start 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f5705200700 1 -- start start 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f5705200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f5705200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f570010c230 con 0x7f5700108950 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f56fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f56fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49652/0 (socket says 172.21.15.7:49652) 2024-06-13T06:41:15.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.995+0000 7f56fde00700 1 -- 172.21.15.7:0/3365765813 learned_addr learned my addr 172.21.15.7:0/3365765813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f56fde00700 1 -- 172.21.15.7:0/3365765813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570019e4f0 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f56fde00700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56f0011b60 tx=0x7f56f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0003b20 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56f0003cc0 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56f0024840 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570019e780 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.996+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a1470 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.997+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f56f0022020 con 0x7f5700108950 2024-06-13T06:41:15.907 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.998+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700063af0 con 0x7f5700108950 2024-06-13T06:41:15.909 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.998+0000 7f56f6a00700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 0x7f56e403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:15.909 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.998+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56f0011e00 con 0x7f5700108950 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.998+0000 7f56fd400700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 0x7f56e403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:14.999+0000 7f56fd400700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 0x7f56e403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56ec006fd0 tx=0x7f56ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.004+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f56f000b3e0 con 0x7f5700108950 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.222+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f57001a1640 con 0x7f5700108950 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.224+0000 7f56f6a00700 1 -- 172.21.15.7:0/3365765813 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f57001a1640 con 0x7f5700108950 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 msgr2=0x7f56e403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 0x7f56e403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56ec006fd0 tx=0x7f56ec006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 msgr2=0x7f570019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56f0011b60 tx=0x7f56f0011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 shutdown_connections 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f56e40390a0 0x7f56e403b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:15.910 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 --2- 172.21.15.7:0/3365765813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5700108950 0x7f570019dfb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:15.911 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.226+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 >> 172.21.15.7:0/3365765813 conn(0x7f5700102360 msgr2=0x7f5700102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:15.911 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.227+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 shutdown_connections 2024-06-13T06:41:15.911 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:15.227+0000 7f5705200700 1 -- 172.21.15.7:0/3365765813 wait complete. 2024-06-13T06:41:15.911 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-06-13T06:41:17.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:16 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2871059577' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-13T06:41:17.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: ignoring --setuser ceph since I am not root 2024-06-13T06:41:17.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: ignoring --setgroup ceph since I am not root 2024-06-13T06:41:17.470 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:17.222+0000 7f7e15657200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:41:17.470 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:17.470+0000 7f7e15657200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:17.631 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.566+0000 7fcce3e00700 1 Processor -- start 2024-06-13T06:41:17.633 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.566+0000 7fcce3e00700 1 -- start start 2024-06-13T06:41:17.633 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.567+0000 7fcce3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:17.633 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.567+0000 7fcce3e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcce407f230 con 0x7fcce407e840 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.567+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.567+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49654/0 (socket says 172.21.15.7:49654) 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.567+0000 7fcce2a00700 1 -- 172.21.15.7:0/1287316258 learned_addr learned my addr 172.21.15.7:0/1287316258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.568+0000 7fcce2a00700 1 -- 172.21.15.7:0/1287316258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce407fa90 con 0x7fcce407e840 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.568+0000 7fcce2a00700 1 --2- 172.21.15.7:0/1287316258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a235409023731a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.568+0000 7fcce1600700 1 -- 172.21.15.7:0/1287316258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4004160 con 0x7fcce407e840 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce1600700 1 -- 172.21.15.7:0/1287316258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccd4004300 con 0x7fcce407e840 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce1600700 1 -- 172.21.15.7:0/1287316258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4011b50 con 0x7fcce407e840 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 -- 172.21.15.7:0/1287316258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 msgr2=0x7fcce407ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 --2- 172.21.15.7:0/1287316258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 -- 172.21.15.7:0/1287316258 shutdown_connections 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 --2- 172.21.15.7:0/1287316258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce407ec60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:17.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 -- 172.21.15.7:0/1287316258 >> 172.21.15.7:0/1287316258 conn(0x7fcce407d270 msgr2=0x7fcce407d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:17.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 -- 172.21.15.7:0/1287316258 shutdown_connections 2024-06-13T06:41:17.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.569+0000 7fcce3e00700 1 -- 172.21.15.7:0/1287316258 wait complete. 2024-06-13T06:41:17.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.570+0000 7fcce3e00700 1 Processor -- start 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce3e00700 1 -- start start 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce3e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4080e50 con 0x7fcce407e840 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49664/0 (socket says 172.21.15.7:49664) 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.571+0000 7fcce2a00700 1 -- 172.21.15.7:0/2871059577 learned_addr learned my addr 172.21.15.7:0/2871059577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fcce2a00700 1 -- 172.21.15.7:0/2871059577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce41a2780 con 0x7fcce407e840 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fcce2a00700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b60 tx=0x7fccd4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:17.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4003970 con 0x7fcce407e840 2024-06-13T06:41:17.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccd4003b10 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd4024610 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.572+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce41a2a10 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.573+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce41a5700 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.574+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce4063af0 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.574+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fccd401a9f0 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.575+0000 7fccdb400700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 0x7fccc803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.575+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fccd4011e00 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.575+0000 7fcce2000700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 0x7fccc803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.576+0000 7fcce2000700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 0x7fccc803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fccd0006fd0 tx=0x7fccd0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.580+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccd400b3e0 con 0x7fcce407e840 2024-06-13T06:41:17.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.871+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fcce402df60 con 0x7fcce407e840 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.925+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fccd404cbb0 con 0x7fcce407e840 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.928+0000 7fccdb400700 1 -- 172.21.15.7:0/2871059577 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fccd4024c40 con 0x7fcce407e840 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.929+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 msgr2=0x7fccc803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.929+0000 7fcce3e00700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 0x7fccc803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fccd0006fd0 tx=0x7fccd0006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.929+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 msgr2=0x7fcce41a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.929+0000 7fcce3e00700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b60 tx=0x7fccd4011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.930+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 shutdown_connections 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.930+0000 7fcce3e00700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7fccc8039050 0x7fccc803b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.930+0000 7fcce3e00700 1 --2- 172.21.15.7:0/2871059577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce407e840 0x7fcce41a2240 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.930+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 >> 172.21.15.7:0/2871059577 conn(0x7fcce407d270 msgr2=0x7fcce4198f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.931+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 shutdown_connections 2024-06-13T06:41:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:16.931+0000 7fcce3e00700 1 -- 172.21.15.7:0/2871059577 wait complete. 2024-06-13T06:41:17.804 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:17.607+0000 7f7e15657200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:41:18.074 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:18.043+0000 7f7e15657200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:41:18.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2871059577' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-13T06:41:18.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:17 smithi007 ceph-mon[32077]: mgrmap e5: x(active, since 5s) 2024-06-13T06:41:18.412 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:18.161+0000 7f7e15657200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:41:18.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:18 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2212839651' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-13T06:41:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.232+0000 7f507ee00700 1 Processor -- start 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.232+0000 7f507ee00700 1 -- start start 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.232+0000 7f507ee00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.232+0000 7f507ee00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f507807c820 con 0x7f507807dd70 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.233+0000 7f507da00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.233+0000 7f507da00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49688/0 (socket says 172.21.15.7:49688) 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.233+0000 7f507da00700 1 -- 172.21.15.7:0/2813329692 learned_addr learned my addr 172.21.15.7:0/2813329692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.233+0000 7f507da00700 1 -- 172.21.15.7:0/2813329692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507807c9a0 con 0x7f507807dd70 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.233+0000 7f507da00700 1 --2- 172.21.15.7:0/2813329692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f506c0098a0 tx=0x7f506c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e049c94d175a4f75 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.234+0000 7f5077e00700 1 -- 172.21.15.7:0/2813329692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f506c004160 con 0x7f507807dd70 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.234+0000 7f5077e00700 1 -- 172.21.15.7:0/2813329692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f506c004300 con 0x7f507807dd70 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.234+0000 7f5077e00700 1 -- 172.21.15.7:0/2813329692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f506c011b50 con 0x7f507807dd70 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.234+0000 7f507ee00700 1 -- 172.21.15.7:0/2813329692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 msgr2=0x7f507807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.234+0000 7f507ee00700 1 --2- 172.21.15.7:0/2813329692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f506c0098a0 tx=0x7f506c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.235+0000 7f507ee00700 1 -- 172.21.15.7:0/2813329692 shutdown_connections 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.235+0000 7f507ee00700 1 --2- 172.21.15.7:0/2813329692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507807c2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.235+0000 7f507ee00700 1 -- 172.21.15.7:0/2813329692 >> 172.21.15.7:0/2813329692 conn(0x7f50781032c0 msgr2=0x7f5078105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.235+0000 7f507ee00700 1 -- 172.21.15.7:0/2813329692 shutdown_connections 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.235+0000 7f507ee00700 1 -- 172.21.15.7:0/2813329692 wait complete. 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.236+0000 7f507ee00700 1 Processor -- start 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.236+0000 7f507ee00700 1 -- start start 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.236+0000 7f507ee00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.236+0000 7f507ee00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f507810e100 con 0x7f507807dd70 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.237+0000 7f507da00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.237+0000 7f507da00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49690/0 (socket says 172.21.15.7:49690) 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.237+0000 7f507da00700 1 -- 172.21.15.7:0/2212839651 learned_addr learned my addr 172.21.15.7:0/2212839651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.237+0000 7f507da00700 1 -- 172.21.15.7:0/2212839651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507819e380 con 0x7f507807dd70 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.237+0000 7f507da00700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f506c011b60 tx=0x7f506c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.238+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f506c003970 con 0x7f507807dd70 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.238+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f506c003b10 con 0x7f507807dd70 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.238+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f506c024610 con 0x7f507807dd70 2024-06-13T06:41:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.238+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507819e610 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.238+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a1300 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.239+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5078063af0 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.239+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f506c01a9f0 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.239+0000 7f5076000700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 0x7f506003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.240+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f506c011e00 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.240+0000 7f507d000700 1 -- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 msgr2=0x7f506003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.240+0000 7f507d000700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 0x7f506003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.245+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f506c016260 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.441+0000 7f507d000700 1 -- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 msgr2=0x7f506003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.441+0000 7f507d000700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 0x7f506003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.637+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f50781a14d0 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.638+0000 7f5076000700 1 -- 172.21.15.7:0/2212839651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f50781a14d0 con 0x7f507807dd70 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 msgr2=0x7f506003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:41:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 0x7f506003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 msgr2=0x7f507819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f506c011b60 tx=0x7f506c011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 shutdown_connections 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f50600390a0 0x7f506003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 --2- 172.21.15.7:0/2212839651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f507807dd70 0x7f507819de40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 >> 172.21.15.7:0/2212839651 conn(0x7f50781032c0 msgr2=0x7f5078103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 shutdown_connections 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:18.641+0000 7f507ee00700 1 -- 172.21.15.7:0/2212839651 wait complete. 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-06-13T06:41:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-06-13T06:41:20.145 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:19.911+0000 7f7e15657200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:41:20.145 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:20.145+0000 7f7e15657200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:20.412 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:20.365+0000 7f7e15657200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:41:21.143 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:20.758+0000 7f7e15657200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:41:21.143 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:20.885+0000 7f7e15657200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:41:21.404 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:21.143+0000 7f7e15657200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:41:21.404 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:21.405+0000 7f7e15657200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:22.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:22.220+0000 7f7e15657200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:41:23.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:22.788+0000 7f7e15657200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:41:23.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:22.919+0000 7f7e15657200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:41:23.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:23.190+0000 7f7e15657200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:41:24.162 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:23.862+0000 7f7e15657200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:41:24.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:23.988+0000 7f7e15657200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:41:24.633 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:24.320+0000 7f7e15657200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:41:24.634 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:24.504+0000 7f7e15657200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:41:24.912 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:24.634+0000 7f7e15657200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:41:24.913 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:24.792+0000 7f7e15657200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:25.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:25.166+0000 7f7e15657200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:41:26.243 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:25.949+0000 7f7e15657200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:26.243+0000 7f7e15657200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: Active manager daemon x restarted 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: Activating manager daemon x 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: osdmap e2: 0 total, 0 up, 0 in 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: mgrmap e6: x(active, starting, since 0.00880893s) 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:41:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:41:26.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:41:26.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: Manager daemon x is now available 2024-06-13T06:41:26.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:26 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: Found migration_current of "None". Setting to last migration. 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:27.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:27.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:27 smithi007 ceph-mon[32077]: mgrmap e7: x(active, since 1.01678s) 2024-06-13T06:41:27.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:27.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-13T06:41:27.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-13T06:41:27.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:27.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.883+0000 7f4c5ac00700 1 Processor -- start 2024-06-13T06:41:27.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.883+0000 7f4c5ac00700 1 -- start start 2024-06-13T06:41:27.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.883+0000 7f4c5ac00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:27.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.883+0000 7f4c5ac00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5407be90 con 0x7f4c541064c0 2024-06-13T06:41:27.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.884+0000 7f4c59800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:27.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.884+0000 7f4c59800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49696/0 (socket says 172.21.15.7:49696) 2024-06-13T06:41:27.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.884+0000 7f4c59800700 1 -- 172.21.15.7:0/2657839364 learned_addr learned my addr 172.21.15.7:0/2657839364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:27.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.885+0000 7f4c59800700 1 -- 172.21.15.7:0/2657839364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c54106e20 con 0x7f4c541064c0 2024-06-13T06:41:27.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.885+0000 7f4c59800700 1 --2- 172.21.15.7:0/2657839364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c480098a0 tx=0x7f4c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a7604c30ac7bc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:27.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.885+0000 7f4c53e00700 1 -- 172.21.15.7:0/2657839364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48004160 con 0x7f4c541064c0 2024-06-13T06:41:27.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.886+0000 7f4c53e00700 1 -- 172.21.15.7:0/2657839364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c48004300 con 0x7f4c541064c0 2024-06-13T06:41:27.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.886+0000 7f4c53e00700 1 -- 172.21.15.7:0/2657839364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48011b50 con 0x7f4c541064c0 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.886+0000 7f4c5ac00700 1 -- 172.21.15.7:0/2657839364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 msgr2=0x7f4c541068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.886+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/2657839364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c480098a0 tx=0x7f4c48009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.887+0000 7f4c5ac00700 1 -- 172.21.15.7:0/2657839364 shutdown_connections 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.887+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/2657839364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c541068e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.887+0000 7f4c5ac00700 1 -- 172.21.15.7:0/2657839364 >> 172.21.15.7:0/2657839364 conn(0x7f4c54102100 msgr2=0x7f4c54104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.887+0000 7f4c5ac00700 1 -- 172.21.15.7:0/2657839364 shutdown_connections 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.887+0000 7f4c5ac00700 1 -- 172.21.15.7:0/2657839364 wait complete. 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.888+0000 7f4c5ac00700 1 Processor -- start 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.888+0000 7f4c5ac00700 1 -- start start 2024-06-13T06:41:27.876 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.888+0000 7f4c5ac00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.888+0000 7f4c5ac00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54107e40 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.889+0000 7f4c59800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.889+0000 7f4c59800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49704/0 (socket says 172.21.15.7:49704) 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.889+0000 7f4c59800700 1 -- 172.21.15.7:0/3529995753 learned_addr learned my addr 172.21.15.7:0/3529995753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.889+0000 7f4c59800700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5419e360 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.889+0000 7f4c59800700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c48011b60 tx=0x7f4c48011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.890+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48003b20 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.890+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c48003cc0 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.890+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c48024840 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.890+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5419e5f0 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.890+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a12e0 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.891+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f4c48022020 con 0x7f4c541064c0 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.892+0000 7f4c52000700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:27.877 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.892+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c3c03bcd0 con 0x7f4c3c039100 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.892+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4c48011e00 con 0x7f4c541064c0 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.892+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:19.892+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:20.093+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:20.093+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:20.494+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:20.494+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:21.295+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:21.295+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:22.897+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:22.897+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-13T06:41:27.878 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:26.100+0000 7f4c58e00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4091601586 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:26.100+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:26.251+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f4c480249e0 con 0x7f4c541064c0 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:26.251+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 msgr2=0x7f4c3c03b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:26.251+0000 7f4c52000700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.257+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f4c4804e0b0 con 0x7f4c541064c0 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.258+0000 7f4c52000700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 0x7f4c3c03ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.258+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c48012ea0 con 0x7f4c3c03ca30 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.260+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 0x7f4c3c03ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.263+0000 7f4c58e00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 0x7f4c3c03ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4c40003a10 tx=0x7f4c400092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.265+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4c48012ea0 con 0x7f4c3c03ca30 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.274+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4c54063af0 con 0x7f4c3c03ca30 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.275+0000 7f4c52000700 1 -- 172.21.15.7:0/3529995753 <== mgr.14120 v2:172.21.15.7:6800/3124714037 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4c54063af0 con 0x7f4c3c03ca30 2024-06-13T06:41:27.879 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.275+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 msgr2=0x7f4c3c03ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.275+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 0x7f4c3c03ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4c40003a10 tx=0x7f4c400092b0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.275+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 msgr2=0x7f4c5419de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.275+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c48011b60 tx=0x7f4c48011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 shutdown_connections 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c3c03ca30 0x7f4c3c03ee20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c541064c0 0x7f4c5419de20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 --2- 172.21.15.7:0/3529995753 >> [v2:172.21.15.7:6800/4091601586,v1:172.21.15.7:6801/4091601586] conn(0x7f4c3c039100 0x7f4c3c03b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 >> 172.21.15.7:0/3529995753 conn(0x7f4c54102100 msgr2=0x7f4c54194ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 shutdown_connections 2024-06-13T06:41:27.880 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:27.276+0000 7f4c5ac00700 1 -- 172.21.15.7:0/3529995753 wait complete. 2024-06-13T06:41:27.881 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-06-13T06:41:27.881 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-06-13T06:41:28.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-13T06:41:28.560 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-13T06:41:28.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:27] ENGINE Bus STARTING 2024-06-13T06:41:28.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:27] ENGINE Serving on https://172.21.15.7:7150 2024-06-13T06:41:28.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:27] ENGINE Serving on http://172.21.15.7:8765 2024-06-13T06:41:28.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:27] ENGINE Bus STARTED 2024-06-13T06:41:28.561 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:28 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:41:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:29 smithi007 ceph-mon[32077]: mgrmap e8: x(active, since 2s) 2024-06-13T06:41:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:29 smithi007 ceph-mon[32077]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:29 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:29 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:41:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.560+0000 7fb8d4800700 1 Processor -- start 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.561+0000 7fb8d4800700 1 -- start start 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.561+0000 7fb8d4800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.561+0000 7fb8d4800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10aad0 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.562+0000 7fb8d1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.562+0000 7fb8d1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42258/0 (socket says 172.21.15.7:42258) 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.562+0000 7fb8d1e00700 1 -- 172.21.15.7:0/363784636 learned_addr learned my addr 172.21.15.7:0/363784636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.562+0000 7fb8d1e00700 1 -- 172.21.15.7:0/363784636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc10b2a0 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.562+0000 7fb8d1e00700 1 --2- 172.21.15.7:0/363784636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00098a0 tx=0x7fb8c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e9981eea61503b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d0a00700 1 -- 172.21.15.7:0/363784636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0004160 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d0a00700 1 -- 172.21.15.7:0/363784636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb8c0004300 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d0a00700 1 -- 172.21.15.7:0/363784636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0011b50 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d4800700 1 -- 172.21.15.7:0/363784636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 msgr2=0x7fb8cc10a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d4800700 1 --2- 172.21.15.7:0/363784636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00098a0 tx=0x7fb8c0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d4800700 1 -- 172.21.15.7:0/363784636 shutdown_connections 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.563+0000 7fb8d4800700 1 --2- 172.21.15.7:0/363784636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc10a500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.564+0000 7fb8d4800700 1 -- 172.21.15.7:0/363784636 >> 172.21.15.7:0/363784636 conn(0x7fb8cc07d450 msgr2=0x7fb8cc07d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.564+0000 7fb8d4800700 1 -- 172.21.15.7:0/363784636 shutdown_connections 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.564+0000 7fb8d4800700 1 -- 172.21.15.7:0/363784636 wait complete. 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d4800700 1 Processor -- start 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d4800700 1 -- start start 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d4800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d4800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10e650 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42270/0 (socket says 172.21.15.7:42270) 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.565+0000 7fb8d1e00700 1 -- 172.21.15.7:0/2826853583 learned_addr learned my addr 172.21.15.7:0/2826853583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8d1e00700 1 -- 172.21.15.7:0/2826853583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc19e510 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8d1e00700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0011b60 tx=0x7fb8c0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0003970 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb8c0003b10 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c0024610 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.566+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc19e7a0 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.567+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a1490 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.568+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb8c001a9f0 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.568+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc063af0 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.568+0000 7fb8c6a00700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 0x7fb8b403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.568+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb8c0011e00 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.569+0000 7fb8d1400700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 0x7fb8b403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.569+0000 7fb8d1400700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 0x7fb8b403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8006fd0 tx=0x7fb8b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.574+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8c0016260 con 0x7fb8cc10a0e0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.812+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb8cc1084c0 con 0x7fb8b40390a0 2024-06-13T06:41:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.823+0000 7fb8c6a00700 1 -- 172.21.15.7:0/2826853583 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb8cc1084c0 con 0x7fb8b40390a0 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.825+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 msgr2=0x7fb8b403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.825+0000 7fb8d4800700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 0x7fb8b403b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8006fd0 tx=0x7fb8b8006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.825+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 msgr2=0x7fb8cc19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.825+0000 7fb8d4800700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0011b60 tx=0x7fb8c0011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 shutdown_connections 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fb8b40390a0 0x7fb8b403b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 --2- 172.21.15.7:0/2826853583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc10a0e0 0x7fb8cc19dfd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 >> 172.21.15.7:0/2826853583 conn(0x7fb8cc07d450 msgr2=0x7fb8cc107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 shutdown_connections 2024-06-13T06:41:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:28.826+0000 7fb8d4800700 1 -- 172.21.15.7:0/2826853583 wait complete. 2024-06-13T06:41:30.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-13T06:41:30.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.105+0000 7fbcf9400700 1 Processor -- start 2024-06-13T06:41:30.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.105+0000 7fbcf9400700 1 -- start start 2024-06-13T06:41:30.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.106+0000 7fbcf9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:30.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.106+0000 7fbcf9400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4108640 con 0x7fbcf4107ce0 2024-06-13T06:41:30.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.106+0000 7fbcf3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:30.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.106+0000 7fbcf3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42274/0 (socket says 172.21.15.7:42274) 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.106+0000 7fbcf3e00700 1 -- 172.21.15.7:0/2190617695 learned_addr learned my addr 172.21.15.7:0/2190617695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.107+0000 7fbcf3e00700 1 -- 172.21.15.7:0/2190617695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf41087c0 con 0x7fbcf4107ce0 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.107+0000 7fbcf3e00700 1 --2- 172.21.15.7:0/2190617695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbce40098a0 tx=0x7fbce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f4a57eb76878e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.107+0000 7fbcf2a00700 1 -- 172.21.15.7:0/2190617695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4004160 con 0x7fbcf4107ce0 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.107+0000 7fbcf2a00700 1 -- 172.21.15.7:0/2190617695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbce4004300 con 0x7fbcf4107ce0 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.107+0000 7fbcf2a00700 1 -- 172.21.15.7:0/2190617695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4011b50 con 0x7fbcf4107ce0 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 -- 172.21.15.7:0/2190617695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 msgr2=0x7fbcf4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 --2- 172.21.15.7:0/2190617695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbce40098a0 tx=0x7fbce4009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:30.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 -- 172.21.15.7:0/2190617695 shutdown_connections 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 --2- 172.21.15.7:0/2190617695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf4108100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 -- 172.21.15.7:0/2190617695 >> 172.21.15.7:0/2190617695 conn(0x7fbcf4103340 msgr2=0x7fbcf4105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 -- 172.21.15.7:0/2190617695 shutdown_connections 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.108+0000 7fbcf9400700 1 -- 172.21.15.7:0/2190617695 wait complete. 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.109+0000 7fbcf9400700 1 Processor -- start 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.109+0000 7fbcf9400700 1 -- start start 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:30.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf9400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4109cd0 con 0x7fbcf4107ce0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42280/0 (socket says 172.21.15.7:42280) 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf3e00700 1 -- 172.21.15.7:0/1413348157 learned_addr learned my addr 172.21.15.7:0/1413348157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.110+0000 7fbcf3e00700 1 -- 172.21.15.7:0/1413348157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf407e090 con 0x7fbcf4107ce0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf3e00700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011b60 tx=0x7fbce4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4003970 con 0x7fbcf4107ce0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbce4003b10 con 0x7fbcf4107ce0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4024610 con 0x7fbcf4107ce0 2024-06-13T06:41:30.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf407c280 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.111+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf407c7c0 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.112+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbce401a430 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.112+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf4063af0 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.113+0000 7fbcf0c00700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 0x7fbcd803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.113+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbce4011e00 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.113+0000 7fbcf3400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 0x7fbcd803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.114+0000 7fbcf3400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 0x7fbcd803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006fd0 tx=0x7fbce0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.119+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbce4016260 con 0x7fbcf4107ce0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.351+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fbcf4104620 con 0x7fbcd80390a0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.352+0000 7fbcf0c00700 1 -- 172.21.15.7:0/1413348157 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fbcf4104620 con 0x7fbcd80390a0 2024-06-13T06:41:30.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 msgr2=0x7fbcd803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 0x7fbcd803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006fd0 tx=0x7fbce0006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 msgr2=0x7fbcf407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011b60 tx=0x7fbce4011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 shutdown_connections 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fbcd80390a0 0x7fbcd803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 --2- 172.21.15.7:0/1413348157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf4107ce0 0x7fbcf407db50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.355+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 >> 172.21.15.7:0/1413348157 conn(0x7fbcf4103340 msgr2=0x7fbcf4103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.356+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 shutdown_connections 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:30.356+0000 7fbcf9400700 1 -- 172.21.15.7:0/1413348157 wait complete. 2024-06-13T06:41:30.984 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-06-13T06:41:31.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:31 smithi007 ceph-mon[32077]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:32.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-mon[32077]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:32.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-mon[32077]: Generating ssh key... 2024-06-13T06:41:32.297 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: Generating public/private rsa key pair. 2024-06-13T06:41:32.297 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: Your identification has been saved in /tmp/tmpch0nn547/key. 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: Your public key has been saved in /tmp/tmpch0nn547/key.pub. 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: The key fingerprint is: 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: SHA256:F0587nd0WwtOltymbjCs7fUOHTZt6VqQzhWsNLCsveY ceph-b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: The key's randomart image is: 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: +---[RSA 3072]----+ 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | . | 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | .. o . | 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | +o.o o | 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | oo+o * +| 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | S.+..X O*| 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | . =B XoB| 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | oo+B.*.| 2024-06-13T06:41:32.298 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | .o.oo=. | 2024-06-13T06:41:32.299 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: | .E.o.o | 2024-06-13T06:41:32.299 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: +----[SHA256]-----+ 2024-06-13T06:41:32.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.571+0000 7f829dc00700 1 Processor -- start 2024-06-13T06:41:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.571+0000 7f829dc00700 1 -- start start 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.571+0000 7f829dc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.571+0000 7f829dc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f829807c8d0 con 0x7f829807dea0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.572+0000 7f8296a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.572+0000 7f8296a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37780/0 (socket says 172.21.15.7:37780) 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.572+0000 7f8296a00700 1 -- 172.21.15.7:0/617943075 learned_addr learned my addr 172.21.15.7:0/617943075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.573+0000 7f8296a00700 1 -- 172.21.15.7:0/617943075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829807ca50 con 0x7f829807dea0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.573+0000 7f8296a00700 1 --2- 172.21.15.7:0/617943075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82880098a0 tx=0x7f8288009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82391fbc5fc2b176 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.574+0000 7f8295600700 1 -- 172.21.15.7:0/617943075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288004160 con 0x7f829807dea0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.574+0000 7f8295600700 1 -- 172.21.15.7:0/617943075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8288004300 con 0x7f829807dea0 2024-06-13T06:41:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.574+0000 7f8295600700 1 -- 172.21.15.7:0/617943075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288011b50 con 0x7f829807dea0 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 -- 172.21.15.7:0/617943075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 msgr2=0x7f829807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 --2- 172.21.15.7:0/617943075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82880098a0 tx=0x7f8288009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 -- 172.21.15.7:0/617943075 shutdown_connections 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 --2- 172.21.15.7:0/617943075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829807c300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 -- 172.21.15.7:0/617943075 >> 172.21.15.7:0/617943075 conn(0x7f8298103450 msgr2=0x7f8298105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 -- 172.21.15.7:0/617943075 shutdown_connections 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.575+0000 7f829dc00700 1 -- 172.21.15.7:0/617943075 wait complete. 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.576+0000 7f829dc00700 1 Processor -- start 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.576+0000 7f829dc00700 1 -- start start 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f829dc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f829dc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f829810e420 con 0x7f829807dea0 2024-06-13T06:41:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f8296a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f8296a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37792/0 (socket says 172.21.15.7:37792) 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f8296a00700 1 -- 172.21.15.7:0/4186486635 learned_addr learned my addr 172.21.15.7:0/4186486635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.577+0000 7f8296a00700 1 -- 172.21.15.7:0/4186486635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829819e530 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f8296a00700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288003b20 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8288003cc0 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829819e7c0 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8288024910 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.578+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981a14b0 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.579+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8298063af0 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.580+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8288022020 con 0x7f829807dea0 2024-06-13T06:41:32.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.580+0000 7f828f400700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 0x7f827c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.580+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8288011e00 con 0x7f829807dea0 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.581+0000 7f8296000700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 0x7f827c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.581+0000 7f8296000700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 0x7f827c03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8284006fd0 tx=0x7f8284006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.586+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f828800f2e0 con 0x7f829807dea0 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:31.815+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f82981046f0 con 0x7f827c039050 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.224+0000 7f828f400700 1 -- 172.21.15.7:0/4186486635 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82981046f0 con 0x7f827c039050 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 msgr2=0x7f827c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 0x7f827c03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8284006fd0 tx=0x7f8284006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 msgr2=0x7f829819dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288011c10 comp rx=0 tx=0).stop 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 shutdown_connections 2024-06-13T06:41:32.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f827c039050 0x7f827c03b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 --2- 172.21.15.7:0/4186486635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829807dea0 0x7f829819dff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 >> 172.21.15.7:0/4186486635 conn(0x7f8298103450 msgr2=0x7f8298103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.227+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 shutdown_connections 2024-06-13T06:41:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:32.228+0000 7f829dc00700 1 -- 172.21.15.7:0/4186486635 wait complete. 2024-06-13T06:41:33.446 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:33 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:33.446 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:33 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:34.348 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:34 smithi007 ceph-mon[32077]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.446+0000 7f41e0400700 1 Processor -- start 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.447+0000 7f41e0400700 1 -- start start 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.447+0000 7f41e0400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.447+0000 7f41e0400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41d810aad0 con 0x7f41d810a0e0 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.448+0000 7f41dda00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.448+0000 7f41dda00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37806/0 (socket says 172.21.15.7:37806) 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.448+0000 7f41dda00700 1 -- 172.21.15.7:0/4032225697 learned_addr learned my addr 172.21.15.7:0/4032225697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.448+0000 7f41dda00700 1 -- 172.21.15.7:0/4032225697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d810b280 con 0x7f41d810a0e0 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.449+0000 7f41dda00700 1 --2- 172.21.15.7:0/4032225697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098a0 tx=0x7f41cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a469578b70a2ef server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.449+0000 7f41d7e00700 1 -- 172.21.15.7:0/4032225697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41cc004160 con 0x7f41d810a0e0 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.449+0000 7f41d7e00700 1 -- 172.21.15.7:0/4032225697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41cc004300 con 0x7f41d810a0e0 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.449+0000 7f41d7e00700 1 -- 172.21.15.7:0/4032225697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41cc011b50 con 0x7f41d810a0e0 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 -- 172.21.15.7:0/4032225697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 msgr2=0x7f41d810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 --2- 172.21.15.7:0/4032225697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098a0 tx=0x7f41cc009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 -- 172.21.15.7:0/4032225697 shutdown_connections 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 --2- 172.21.15.7:0/4032225697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d810a500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 -- 172.21.15.7:0/4032225697 >> 172.21.15.7:0/4032225697 conn(0x7f41d807d510 msgr2=0x7f41d807d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 -- 172.21.15.7:0/4032225697 shutdown_connections 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.450+0000 7f41e0400700 1 -- 172.21.15.7:0/4032225697 wait complete. 2024-06-13T06:41:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.451+0000 7f41e0400700 1 Processor -- start 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.451+0000 7f41e0400700 1 -- start start 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.452+0000 7f41e0400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.452+0000 7f41e0400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41d810e630 con 0x7f41d810a0e0 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.452+0000 7f41dda00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.452+0000 7f41dda00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37822/0 (socket says 172.21.15.7:37822) 2024-06-13T06:41:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.452+0000 7f41dda00700 1 -- 172.21.15.7:0/4189548645 learned_addr learned my addr 172.21.15.7:0/4189548645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41dda00700 1 -- 172.21.15.7:0/4189548645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d819e490 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41dda00700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41cc011670 tx=0x7f41cc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41cc0039c0 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41cc011e50 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41cc01a5c0 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d819e720 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.453+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81a1410 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.454+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d8063af0 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.455+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f41cc024430 con 0x7f41d810a0e0 2024-06-13T06:41:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.455+0000 7f41d6000700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 0x7f41c003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.455+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41cc04d540 con 0x7f41d810a0e0 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.456+0000 7f41dd000700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 0x7f41c003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.457+0000 7f41dd000700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 0x7f41c003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f41c4006fd0 tx=0x7f41c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.461+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41cc04ce90 con 0x7f41d810a0e0 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.693+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f41d81083f0 con 0x7f41c0039050 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.695+0000 7f41d6000700 1 -- 172.21.15.7:0/4189548645 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f41d81083f0 con 0x7f41c0039050 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.698+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 msgr2=0x7f41c003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.698+0000 7f41e0400700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 0x7f41c003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f41c4006fd0 tx=0x7f41c4006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.698+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 msgr2=0x7f41d819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.698+0000 7f41e0400700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41cc011670 tx=0x7f41cc011b60 comp rx=0 tx=0).stop 2024-06-13T06:41:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.698+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 shutdown_connections 2024-06-13T06:41:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.699+0000 7f41e0400700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f41c0039050 0x7f41c003b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.699+0000 7f41e0400700 1 --2- 172.21.15.7:0/4189548645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d810a0e0 0x7f41d819df50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.699+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 >> 172.21.15.7:0/4189548645 conn(0x7f41d807d510 msgr2=0x7f41d8107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.699+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 shutdown_connections 2024-06-13T06:41:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:33.699+0000 7f41e0400700 1 -- 172.21.15.7:0/4189548645 wait complete. 2024-06-13T06:41:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-13T06:41:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-06-13T06:41:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-06-13T06:41:36.274 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:36 smithi007 ceph-mon[32077]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:37.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:37 smithi007 ceph-mon[32077]: Deploying cephadm binary to smithi007 2024-06-13T06:41:42.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-06-13T06:41:42.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.983+0000 7f7548400700 1 Processor -- start 2024-06-13T06:41:42.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.983+0000 7f7548400700 1 -- start start 2024-06-13T06:41:42.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.984+0000 7f7548400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.984+0000 7f7548400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75401087d0 con 0x7f7540107e70 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.984+0000 7f7545a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.985+0000 7f7545a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37838/0 (socket says 172.21.15.7:37838) 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.985+0000 7f7545a00700 1 -- 172.21.15.7:0/4000218799 learned_addr learned my addr 172.21.15.7:0/4000218799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.985+0000 7f7545a00700 1 -- 172.21.15.7:0/4000218799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7540108fd0 con 0x7f7540107e70 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.985+0000 7f7545a00700 1 --2- 172.21.15.7:0/4000218799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75340098a0 tx=0x7f7534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca50222f7cb489f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f753fe00700 1 -- 172.21.15.7:0/4000218799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534004160 con 0x7f7540107e70 2024-06-13T06:41:42.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f753fe00700 1 -- 172.21.15.7:0/4000218799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7534004300 con 0x7f7540107e70 2024-06-13T06:41:42.361 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f753fe00700 1 -- 172.21.15.7:0/4000218799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534011b50 con 0x7f7540107e70 2024-06-13T06:41:42.361 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f7548400700 1 -- 172.21.15.7:0/4000218799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 msgr2=0x7f7540108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:42.361 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f7548400700 1 --2- 172.21.15.7:0/4000218799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75340098a0 tx=0x7f7534009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f7548400700 1 -- 172.21.15.7:0/4000218799 shutdown_connections 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f7548400700 1 --2- 172.21.15.7:0/4000218799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f7540108290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.986+0000 7f7548400700 1 -- 172.21.15.7:0/4000218799 >> 172.21.15.7:0/4000218799 conn(0x7f7540103450 msgr2=0x7f7540105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.987+0000 7f7548400700 1 -- 172.21.15.7:0/4000218799 shutdown_connections 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.987+0000 7f7548400700 1 -- 172.21.15.7:0/4000218799 wait complete. 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.988+0000 7f7548400700 1 Processor -- start 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.988+0000 7f7548400700 1 -- start start 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.988+0000 7f7548400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.988+0000 7f7548400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f754010c290 con 0x7f7540107e70 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.989+0000 7f7545a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.989+0000 7f7545a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37840/0 (socket says 172.21.15.7:37840) 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.989+0000 7f7545a00700 1 -- 172.21.15.7:0/862773021 learned_addr learned my addr 172.21.15.7:0/862773021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.989+0000 7f7545a00700 1 -- 172.21.15.7:0/862773021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754019e510 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.989+0000 7f7545a00700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7534011b60 tx=0x7f7534011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.990+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534003b20 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.990+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7534003cc0 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.990+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534024840 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.990+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754019e7a0 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.990+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a1490 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.991+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7534022020 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.991+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754010c410 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.992+0000 7f753e000700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 0x7f752803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.992+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7534011e00 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.992+0000 7f7545000700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 0x7f752803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.993+0000 7f7545000700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 0x7f752803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f752c006fd0 tx=0x7f752c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:34.998+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f753404cb30 con 0x7f7540107e70 2024-06-13T06:41:42.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:35.232+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7f7540104610 con 0x7f75280390a0 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.712+0000 7f753e000700 1 -- 172.21.15.7:0/862773021 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f7540104610 con 0x7f75280390a0 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 msgr2=0x7f752803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 0x7f752803b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f752c006fd0 tx=0x7f752c006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 msgr2=0x7f754019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7534011b60 tx=0x7f7534011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 shutdown_connections 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f75280390a0 0x7f752803b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 --2- 172.21.15.7:0/862773021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7540107e70 0x7f754019dfd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.717+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 >> 172.21.15.7:0/862773021 conn(0x7f7540103450 msgr2=0x7f7540103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.718+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 shutdown_connections 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:41.718+0000 7f7548400700 1 -- 172.21.15.7:0/862773021 wait complete. 2024-06-13T06:41:42.364 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-06-13T06:41:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:42 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:42 smithi007 ceph-mon[32077]: Added host smithi007 2024-06-13T06:41:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:42 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.967+0000 7f82f1e00700 1 Processor -- start 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.967+0000 7f82f1e00700 1 -- start start 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.968+0000 7f82f1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.968+0000 7f82f1e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec10ab20 con 0x7f82ec10a130 2024-06-13T06:41:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.968+0000 7f82f0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:43.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.968+0000 7f82f0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60894/0 (socket says 172.21.15.7:60894) 2024-06-13T06:41:43.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.968+0000 7f82f0a00700 1 -- 172.21.15.7:0/4023094274 learned_addr learned my addr 172.21.15.7:0/4023094274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:43.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.969+0000 7f82f0a00700 1 -- 172.21.15.7:0/4023094274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec10aff0 con 0x7f82ec10a130 2024-06-13T06:41:43.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.969+0000 7f82f0a00700 1 --2- 172.21.15.7:0/4023094274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098a0 tx=0x7f82e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9363bf28a5ddae4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:43.891 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.969+0000 7f82eb400700 1 -- 172.21.15.7:0/4023094274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e0004160 con 0x7f82ec10a130 2024-06-13T06:41:43.891 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82eb400700 1 -- 172.21.15.7:0/4023094274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82e0004300 con 0x7f82ec10a130 2024-06-13T06:41:43.891 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82eb400700 1 -- 172.21.15.7:0/4023094274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e0011b50 con 0x7f82ec10a130 2024-06-13T06:41:43.891 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82f1e00700 1 -- 172.21.15.7:0/4023094274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 msgr2=0x7f82ec10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82f1e00700 1 --2- 172.21.15.7:0/4023094274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82e00098a0 tx=0x7f82e0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82f1e00700 1 -- 172.21.15.7:0/4023094274 shutdown_connections 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82f1e00700 1 --2- 172.21.15.7:0/4023094274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec10a550 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.970+0000 7f82f1e00700 1 -- 172.21.15.7:0/4023094274 >> 172.21.15.7:0/4023094274 conn(0x7f82ec07d2b0 msgr2=0x7f82ec07d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.971+0000 7f82f1e00700 1 -- 172.21.15.7:0/4023094274 shutdown_connections 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.971+0000 7f82f1e00700 1 -- 172.21.15.7:0/4023094274 wait complete. 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f1e00700 1 Processor -- start 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f1e00700 1 -- start start 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f1e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec10e2b0 con 0x7f82ec10a130 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60904/0 (socket says 172.21.15.7:60904) 2024-06-13T06:41:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.972+0000 7f82f0a00700 1 -- 172.21.15.7:0/1702274766 learned_addr learned my addr 172.21.15.7:0/1702274766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82f0a00700 1 -- 172.21.15.7:0/1702274766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec19e390 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82f0a00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fb0 tx=0x7f82e0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e00038b0 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82e0011e50 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec19e620 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82e0019420 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.973+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a1310 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.975+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f82e001a430 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.975+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ec10e430 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.975+0000 7f82e9600700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 0x7f82d403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.975+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82e004d3a0 con 0x7f82ec10a130 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.975+0000 7f82ebe00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 0x7f82d403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.976+0000 7f82ebe00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 0x7f82d403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:42.981+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f82e00172b0 con 0x7f82ec10a130 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.225+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f82ec1084c0 con 0x7f82d4039050 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.236+0000 7f82e9600700 1 -- 172.21.15.7:0/1702274766 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f82ec1084c0 con 0x7f82d4039050 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 msgr2=0x7f82d403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 0x7f82d403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82dc006fd0 tx=0x7f82dc006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 msgr2=0x7f82ec19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009fb0 tx=0x7f82e0011bf0 comp rx=0 tx=0).stop 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 shutdown_connections 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f82d4039050 0x7f82d403b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 --2- 172.21.15.7:0/1702274766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82ec10a130 0x7f82ec19de50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 >> 172.21.15.7:0/1702274766 conn(0x7f82ec07d2b0 msgr2=0x7f82ec107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 shutdown_connections 2024-06-13T06:41:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:43.239+0000 7f82f1e00700 1 -- 172.21.15.7:0/1702274766 wait complete. 2024-06-13T06:41:43.895 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-06-13T06:41:44.461 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:44 smithi007 ceph-mon[32077]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:44.461 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:44 smithi007 ceph-mon[32077]: Saving service mon spec with placement count:5 2024-06-13T06:41:44.461 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:44 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:45.706 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-13T06:41:45.706 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.733+0000 7ff479400700 1 Processor -- start 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.733+0000 7ff479400700 1 -- start start 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.734+0000 7ff479400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.734+0000 7ff479400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff47407c8b0 con 0x7ff47407de80 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.735+0000 7ff472800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.735+0000 7ff472800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60906/0 (socket says 172.21.15.7:60906) 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.735+0000 7ff472800700 1 -- 172.21.15.7:0/3338228274 learned_addr learned my addr 172.21.15.7:0/3338228274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.735+0000 7ff472800700 1 -- 172.21.15.7:0/3338228274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47407ca30 con 0x7ff47407de80 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.736+0000 7ff472800700 1 --2- 172.21.15.7:0/3338228274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098a0 tx=0x7ff464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0d294eabf32761 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.736+0000 7ff471400700 1 -- 172.21.15.7:0/3338228274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464004160 con 0x7ff47407de80 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.736+0000 7ff471400700 1 -- 172.21.15.7:0/3338228274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff464004300 con 0x7ff47407de80 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.736+0000 7ff471400700 1 -- 172.21.15.7:0/3338228274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464011b50 con 0x7ff47407de80 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 -- 172.21.15.7:0/3338228274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 msgr2=0x7ff47407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 --2- 172.21.15.7:0/3338228274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098a0 tx=0x7ff464009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:45.707 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 -- 172.21.15.7:0/3338228274 shutdown_connections 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 --2- 172.21.15.7:0/3338228274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47407c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 -- 172.21.15.7:0/3338228274 >> 172.21.15.7:0/3338228274 conn(0x7ff474103400 msgr2=0x7ff474105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 -- 172.21.15.7:0/3338228274 shutdown_connections 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.737+0000 7ff479400700 1 -- 172.21.15.7:0/3338228274 wait complete. 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.738+0000 7ff479400700 1 Processor -- start 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.738+0000 7ff479400700 1 -- start start 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.738+0000 7ff479400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.738+0000 7ff479400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff47410e3f0 con 0x7ff47407de80 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.739+0000 7ff472800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.739+0000 7ff472800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60914/0 (socket says 172.21.15.7:60914) 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.739+0000 7ff472800700 1 -- 172.21.15.7:0/3154816258 learned_addr learned my addr 172.21.15.7:0/3154816258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:45.708 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.739+0000 7ff472800700 1 -- 172.21.15.7:0/3154816258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47419e4e0 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.739+0000 7ff472800700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.740+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464003b20 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.740+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff464003cc0 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.740+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47419e770 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.740+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464024910 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.740+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4741a1460 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.741+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff464022020 con 0x7ff47407de80 2024-06-13T06:41:45.709 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.741+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff474063af0 con 0x7ff47407de80 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.742+0000 7ff46b400700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 0x7ff45803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.742+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff464011e00 con 0x7ff47407de80 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.742+0000 7ff471e00700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 0x7ff45803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.743+0000 7ff471e00700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 0x7ff45803b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff460006fd0 tx=0x7ff460006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:44.748+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff464017ca0 con 0x7ff47407de80 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.004+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff4741046a0 con 0x7ff4580390a0 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.011+0000 7ff46b400700 1 -- 172.21.15.7:0/3154816258 <== mgr.14120 v2:172.21.15.7:6800/3124714037 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff4741046a0 con 0x7ff4580390a0 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.014+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 msgr2=0x7ff45803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.014+0000 7ff479400700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 0x7ff45803b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff460006fd0 tx=0x7ff460006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:45.710 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.014+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 msgr2=0x7ff47419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.014+0000 7ff479400700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464011c10 comp rx=0 tx=0).stop 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 shutdown_connections 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7ff4580390a0 0x7ff45803b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 --2- 172.21.15.7:0/3154816258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47407de80 0x7ff47419dfa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 >> 172.21.15.7:0/3154816258 conn(0x7ff474103400 msgr2=0x7ff474103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 shutdown_connections 2024-06-13T06:41:45.711 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:45.015+0000 7ff479400700 1 -- 172.21.15.7:0/3154816258 wait complete. 2024-06-13T06:41:46.374 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:46 smithi007 ceph-mon[32077]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:41:46.374 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:46 smithi007 ceph-mon[32077]: Saving service mgr spec with placement count:2 2024-06-13T06:41:46.374 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:46 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:47.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.373+0000 7f4c29000700 1 Processor -- start 2024-06-13T06:41:47.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.373+0000 7f4c29000700 1 -- start start 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.374+0000 7f4c29000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.374+0000 7f4c29000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24107620 con 0x7f4c24106c30 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.374+0000 7f4c23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.374+0000 7f4c23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60916/0 (socket says 172.21.15.7:60916) 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.374+0000 7f4c23400700 1 -- 172.21.15.7:0/585956465 learned_addr learned my addr 172.21.15.7:0/585956465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.375+0000 7f4c23400700 1 -- 172.21.15.7:0/585956465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24107de0 con 0x7f4c24106c30 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.375+0000 7f4c23400700 1 --2- 172.21.15.7:0/585956465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098a0 tx=0x7f4c14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53feb3e59de66cc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.375+0000 7f4c22000700 1 -- 172.21.15.7:0/585956465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c14004160 con 0x7f4c24106c30 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c22000700 1 -- 172.21.15.7:0/585956465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c14004300 con 0x7f4c24106c30 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c22000700 1 -- 172.21.15.7:0/585956465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c14011b50 con 0x7f4c24106c30 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 -- 172.21.15.7:0/585956465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 msgr2=0x7f4c24107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 --2- 172.21.15.7:0/585956465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098a0 tx=0x7f4c14009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:47.365 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 -- 172.21.15.7:0/585956465 shutdown_connections 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 --2- 172.21.15.7:0/585956465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24107050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 -- 172.21.15.7:0/585956465 >> 172.21.15.7:0/585956465 conn(0x7f4c241021f0 msgr2=0x7f4c24104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 -- 172.21.15.7:0/585956465 shutdown_connections 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.376+0000 7f4c29000700 1 -- 172.21.15.7:0/585956465 wait complete. 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c29000700 1 Processor -- start 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c29000700 1 -- start start 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c29000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c29000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2410b190 con 0x7f4c24106c30 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60918/0 (socket says 172.21.15.7:60918) 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.378+0000 7f4c23400700 1 -- 172.21.15.7:0/3807450688 learned_addr learned my addr 172.21.15.7:0/3807450688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.379+0000 7f4c23400700 1 -- 172.21.15.7:0/3807450688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24192240 con 0x7f4c24106c30 2024-06-13T06:41:47.366 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.379+0000 7f4c23400700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fb0 tx=0x7f4c14011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.379+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c140038b0 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.380+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c14011e50 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.380+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c24190430 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.380+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c14019550 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.380+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c24190970 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.381+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4c14003a50 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.381+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24063af0 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.381+0000 7f4c1be00700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 0x7f4c0803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.381+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c1404d290 con 0x7f4c24106c30 2024-06-13T06:41:47.367 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.382+0000 7f4c22a00700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 0x7f4c0803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.382+0000 7f4c22a00700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 0x7f4c0803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c10006fd0 tx=0x7f4c10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.387+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c1400fda0 con 0x7f4c24106c30 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.555+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c14023e00 con 0x7f4c24106c30 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.625+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4c24190bf0 con 0x7f4c24106c30 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.628+0000 7f4c1be00700 1 -- 172.21.15.7:0/3807450688 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4c14023390 con 0x7f4c24106c30 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 msgr2=0x7f4c0803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 0x7f4c0803b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c10006fd0 tx=0x7f4c10006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 msgr2=0x7f4c24191d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fb0 tx=0x7f4c14011bf0 comp rx=0 tx=0).stop 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 shutdown_connections 2024-06-13T06:41:47.368 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f4c08039050 0x7f4c0803b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:47.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 --2- 172.21.15.7:0/3807450688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c24106c30 0x7f4c24191d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:47.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 >> 172.21.15.7:0/3807450688 conn(0x7f4c241021f0 msgr2=0x7f4c24102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:47.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 shutdown_connections 2024-06-13T06:41:47.369 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:46.632+0000 7f4c29000700 1 -- 172.21.15.7:0/3807450688 wait complete. 2024-06-13T06:41:47.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:47 smithi007 ceph-mon[32077]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:41:47.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:47 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:47.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3807450688' entity='client.admin' 2024-06-13T06:41:47.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:47 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.949+0000 7f1e56400700 1 Processor -- start 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e56400700 1 -- start start 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e56400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e56400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5007c8b0 con 0x7f1e5007de80 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e4f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e4f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60920/0 (socket says 172.21.15.7:60920) 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.950+0000 7f1e4f400700 1 -- 172.21.15.7:0/3417019105 learned_addr learned my addr 172.21.15.7:0/3417019105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.951+0000 7f1e4f400700 1 -- 172.21.15.7:0/3417019105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5007ca30 con 0x7f1e5007de80 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.951+0000 7f1e4f400700 1 --2- 172.21.15.7:0/3417019105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e400098a0 tx=0x7f1e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f21139e2cd7521 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.952+0000 7f1e4e000700 1 -- 172.21.15.7:0/3417019105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40004160 con 0x7f1e5007de80 2024-06-13T06:41:48.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.952+0000 7f1e4e000700 1 -- 172.21.15.7:0/3417019105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e40004300 con 0x7f1e5007de80 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.952+0000 7f1e4e000700 1 -- 172.21.15.7:0/3417019105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40011b20 con 0x7f1e5007de80 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.952+0000 7f1e56400700 1 -- 172.21.15.7:0/3417019105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 msgr2=0x7f1e5007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.952+0000 7f1e56400700 1 --2- 172.21.15.7:0/3417019105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e400098a0 tx=0x7f1e40009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.953+0000 7f1e56400700 1 -- 172.21.15.7:0/3417019105 shutdown_connections 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.953+0000 7f1e56400700 1 --2- 172.21.15.7:0/3417019105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5007c2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.953+0000 7f1e56400700 1 -- 172.21.15.7:0/3417019105 >> 172.21.15.7:0/3417019105 conn(0x7f1e50103450 msgr2=0x7f1e50105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.953+0000 7f1e56400700 1 -- 172.21.15.7:0/3417019105 shutdown_connections 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.953+0000 7f1e56400700 1 -- 172.21.15.7:0/3417019105 wait complete. 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.954+0000 7f1e56400700 1 Processor -- start 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.954+0000 7f1e56400700 1 -- start start 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.954+0000 7f1e56400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.954+0000 7f1e56400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5010e4f0 con 0x7f1e5007de80 2024-06-13T06:41:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.955+0000 7f1e4f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.955+0000 7f1e4f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60922/0 (socket says 172.21.15.7:60922) 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.955+0000 7f1e4f400700 1 -- 172.21.15.7:0/3774605790 learned_addr learned my addr 172.21.15.7:0/3774605790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.955+0000 7f1e4f400700 1 -- 172.21.15.7:0/3774605790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5019e4c0 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e4f400700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e400117e0 tx=0x7f1e40011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40003bf0 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4001a910 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5019e750 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e40024970 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.956+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5019ebb0 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.957+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50063af0 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.958+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1e40022020 con 0x7f1e5007de80 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.959+0000 7f1e47e00700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 0x7f1e3403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.959+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1e4004d4a0 con 0x7f1e5007de80 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.959+0000 7f1e4ea00700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 0x7f1e3403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.960+0000 7f1e4ea00700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 0x7f1e3403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c006fd0 tx=0x7f1e3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:47.964+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1e400212c0 con 0x7f1e5007de80 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.196+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f1e501a1640 con 0x7f1e5007de80 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.199+0000 7f1e47e00700 1 -- 172.21.15.7:0/3774605790 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f1e4004cdf0 con 0x7f1e5007de80 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 msgr2=0x7f1e3403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 0x7f1e3403b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c006fd0 tx=0x7f1e3c006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 msgr2=0x7f1e5019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e400117e0 tx=0x7f1e40011ba0 comp rx=0 tx=0).stop 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 shutdown_connections 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f1e340390a0 0x7f1e3403b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 --2- 172.21.15.7:0/3774605790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5007de80 0x7f1e5019df80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:48.833 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 >> 172.21.15.7:0/3774605790 conn(0x7f1e50103450 msgr2=0x7f1e50103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:48.834 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 shutdown_connections 2024-06-13T06:41:48.834 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:48.202+0000 7f1e56400700 1 -- 172.21.15.7:0/3774605790 wait complete. 2024-06-13T06:41:48.834 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-06-13T06:41:49.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:49 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3774605790' entity='client.admin' 2024-06-13T06:41:49.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:49 smithi007 ceph-mon[32077]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:41:50.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3895262359' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-13T06:41:50.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-mon[32077]: from='mgr.14120 172.21.15.7:0/958023518' entity='mgr.x' 2024-06-13T06:41:50.524 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: ignoring --setuser ceph since I am not root 2024-06-13T06:41:50.524 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: ignoring --setgroup ceph since I am not root 2024-06-13T06:41:50.524 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:50.523+0000 7f7ff5dac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:41:50.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.399+0000 7fba2a600700 1 Processor -- start 2024-06-13T06:41:50.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.399+0000 7fba2a600700 1 -- start start 2024-06-13T06:41:50.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.400+0000 7fba2a600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.400+0000 7fba2a600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba2407c870 con 0x7fba2407de40 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.400+0000 7fba23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.401+0000 7fba23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60932/0 (socket says 172.21.15.7:60932) 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.401+0000 7fba23400700 1 -- 172.21.15.7:0/4060984416 learned_addr learned my addr 172.21.15.7:0/4060984416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.401+0000 7fba23400700 1 -- 172.21.15.7:0/4060984416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2407c9f0 con 0x7fba2407de40 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.401+0000 7fba23400700 1 --2- 172.21.15.7:0/4060984416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba140098a0 tx=0x7fba14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f52f049def9a51 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.402+0000 7fba22000700 1 -- 172.21.15.7:0/4060984416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba14004160 con 0x7fba2407de40 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.402+0000 7fba22000700 1 -- 172.21.15.7:0/4060984416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba14004300 con 0x7fba2407de40 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.402+0000 7fba22000700 1 -- 172.21.15.7:0/4060984416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba14011b20 con 0x7fba2407de40 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 -- 172.21.15.7:0/4060984416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 msgr2=0x7fba2407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 --2- 172.21.15.7:0/4060984416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba140098a0 tx=0x7fba14009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 -- 172.21.15.7:0/4060984416 shutdown_connections 2024-06-13T06:41:50.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 --2- 172.21.15.7:0/4060984416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2407c2a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:50.917 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 -- 172.21.15.7:0/4060984416 >> 172.21.15.7:0/4060984416 conn(0x7fba24103450 msgr2=0x7fba24105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:50.917 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 -- 172.21.15.7:0/4060984416 shutdown_connections 2024-06-13T06:41:50.917 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.403+0000 7fba2a600700 1 -- 172.21.15.7:0/4060984416 wait complete. 2024-06-13T06:41:50.917 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.404+0000 7fba2a600700 1 Processor -- start 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.404+0000 7fba2a600700 1 -- start start 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.404+0000 7fba2a600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.404+0000 7fba2a600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba2410e4d0 con 0x7fba2407de40 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.405+0000 7fba23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.405+0000 7fba23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60948/0 (socket says 172.21.15.7:60948) 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.405+0000 7fba23400700 1 -- 172.21.15.7:0/3895262359 learned_addr learned my addr 172.21.15.7:0/3895262359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.405+0000 7fba23400700 1 -- 172.21.15.7:0/3895262359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2419e490 con 0x7fba2407de40 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.405+0000 7fba23400700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba140117e0 tx=0x7fba14011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.406+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba14003ba0 con 0x7fba2407de40 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.406+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1401a910 con 0x7fba2407de40 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.406+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2419e720 con 0x7fba2407de40 2024-06-13T06:41:50.918 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.406+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba140248f0 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.406+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2419eb80 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.407+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fba14022020 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.408+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba24063af0 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.408+0000 7fba1be00700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 0x7fba0803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.408+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba1404d5b0 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.408+0000 7fba22a00700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 0x7fba0803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.409+0000 7fba22a00700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 0x7fba0803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba10006fd0 tx=0x7fba10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.414+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba1402cdf0 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:49.712+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fba241a1550 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.206+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fba1401ab30 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.209+0000 7fba1be00700 1 -- 172.21.15.7:0/3895262359 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fba1402dc90 con 0x7fba2407de40 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 msgr2=0x7fba0803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:50.919 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 0x7fba0803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba10006fd0 tx=0x7fba10006e40 comp rx=0 tx=0).stop 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 msgr2=0x7fba2419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba140117e0 tx=0x7fba14011ba0 comp rx=0 tx=0).stop 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 shutdown_connections 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fba080390a0 0x7fba0803b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 --2- 172.21.15.7:0/3895262359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba2407de40 0x7fba2419df50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 >> 172.21.15.7:0/3895262359 conn(0x7fba24103450 msgr2=0x7fba24103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 shutdown_connections 2024-06-13T06:41:50.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:50.211+0000 7fba2a600700 1 -- 172.21.15.7:0/3895262359 wait complete. 2024-06-13T06:41:51.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:50.773+0000 7f7ff5dac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:51.071 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:50.912+0000 7f7ff5dac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:41:51.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:51 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3895262359' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-13T06:41:51.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:51 smithi007 ceph-mon[32077]: mgrmap e9: x(active, since 23s) 2024-06-13T06:41:51.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:51.331+0000 7f7ff5dac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:41:51.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:51.451+0000 7f7ff5dac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:41:52.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:52 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3873358054' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.527+0000 7fc5a9400700 1 Processor -- start 2024-06-13T06:41:52.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a9400700 1 -- start start 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a9400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410aad0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46076/0 (socket says 172.21.15.7:46076) 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.528+0000 7fc5a2800700 1 -- 172.21.15.7:0/1663542584 learned_addr learned my addr 172.21.15.7:0/1663542584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.529+0000 7fc5a2800700 1 -- 172.21.15.7:0/1663542584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a410b2a0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.529+0000 7fc5a2800700 1 --2- 172.21.15.7:0/1663542584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5940098a0 tx=0x7fc594009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc544295ce7173da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.529+0000 7fc5a1400700 1 -- 172.21.15.7:0/1663542584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc594004160 con 0x7fc5a410a0e0 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.529+0000 7fc5a1400700 1 -- 172.21.15.7:0/1663542584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc594004300 con 0x7fc5a410a0e0 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.529+0000 7fc5a1400700 1 -- 172.21.15.7:0/1663542584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc594011b20 con 0x7fc5a410a0e0 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 -- 172.21.15.7:0/1663542584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 msgr2=0x7fc5a410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 --2- 172.21.15.7:0/1663542584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5940098a0 tx=0x7fc594009bb0 comp rx=0 tx=0).stop 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 -- 172.21.15.7:0/1663542584 shutdown_connections 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 --2- 172.21.15.7:0/1663542584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a410a500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 -- 172.21.15.7:0/1663542584 >> 172.21.15.7:0/1663542584 conn(0x7fc5a407d450 msgr2=0x7fc5a407d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.530+0000 7fc5a9400700 1 -- 172.21.15.7:0/1663542584 shutdown_connections 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.531+0000 7fc5a9400700 1 -- 172.21.15.7:0/1663542584 wait complete. 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a9400700 1 Processor -- start 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a9400700 1 -- start start 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a9400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410e650 con 0x7fc5a410a0e0 2024-06-13T06:41:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46088/0 (socket says 172.21.15.7:46088) 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.532+0000 7fc5a2800700 1 -- 172.21.15.7:0/3873358054 learned_addr learned my addr 172.21.15.7:0/3873358054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc5a2800700 1 -- 172.21.15.7:0/3873358054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a419e4b0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc5a2800700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc594011b60 tx=0x7fc594011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc594003b60 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59401a910 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc594024930 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.533+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a419e740 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.534+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a419eba0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.535+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a4063af0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.535+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc594022020 con 0x7fc5a410a0e0 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.535+0000 7fc59b400700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 0x7fc58803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:41:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.535+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc59404d310 con 0x7fc5a410a0e0 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.536+0000 7fc5a1e00700 1 -- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 msgr2=0x7fc58803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.536+0000 7fc5a1e00700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 0x7fc58803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.541+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc59400b3e0 con 0x7fc5a410a0e0 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.737+0000 7fc5a1e00700 1 -- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 msgr2=0x7fc58803b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.737+0000 7fc5a1e00700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 0x7fc58803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.829+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc5a41a1610 con 0x7fc5a410a0e0 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.829+0000 7fc59b400700 1 -- 172.21.15.7:0/3873358054 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc59404cc60 con 0x7fc5a410a0e0 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 msgr2=0x7fc58803b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 0x7fc58803b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 msgr2=0x7fc5a419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:41:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc594011b60 tx=0x7fc594011c40 comp rx=0 tx=0).stop 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 shutdown_connections 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7fc588039100 0x7fc58803b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 --2- 172.21.15.7:0/3873358054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a410a0e0 0x7fc5a419df70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 >> 172.21.15.7:0/3873358054 conn(0x7fc5a407d450 msgr2=0x7fc5a4107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 shutdown_connections 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:51.832+0000 7fc5a9400700 1 -- 172.21.15.7:0/3873358054 wait complete. 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-06-13T06:41:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-06-13T06:41:53.304 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:53.073+0000 7f7ff5dac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:41:53.304 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:53.303+0000 7f7ff5dac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:53.652 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:53.524+0000 7f7ff5dac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:41:54.300 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:53.915+0000 7f7ff5dac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:41:54.300 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:54.041+0000 7f7ff5dac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:41:54.561 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:54.299+0000 7f7ff5dac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:41:54.912 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:54.561+0000 7f7ff5dac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:41:55.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:55.381+0000 7f7ff5dac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:41:56.360 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:55.952+0000 7f7ff5dac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:41:56.360 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:56.086+0000 7f7ff5dac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:41:56.361 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:56.359+0000 7f7ff5dac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:41:57.412 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.035+0000 7f7ff5dac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:41:57.413 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.162+0000 7f7ff5dac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:41:57.813 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.496+0000 7f7ff5dac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:41:57.813 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.682+0000 7f7ff5dac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:41:57.813 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.812+0000 7f7ff5dac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:41:58.162 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:57.970+0000 7f7ff5dac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:41:58.662 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:58.347+0000 7f7ff5dac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:41:59.412 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:59.132+0000 7f7ff5dac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:41:59.912 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:41:59.452+0000 7f7ff5dac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: Active manager daemon x restarted 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: Activating manager daemon x 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: osdmap e3: 0 total, 0 up, 0 in 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: mgrmap e10: x(active, starting, since 0.00984045s) 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:41:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-13T06:41:59.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-13T06:41:59.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-13T06:41:59.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-13T06:41:59.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:41:59 smithi007 ceph-mon[32077]: Manager daemon x is now available 2024-06-13T06:42:00.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:00.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:59] ENGINE Bus STARTING 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: [13/Jun/2024:06:41:59] ENGINE Serving on http://172.21.15.7:8765 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: [13/Jun/2024:06:42:00] ENGINE Serving on https://172.21.15.7:7150 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: [13/Jun/2024:06:42:00] ENGINE Bus STARTED 2024-06-13T06:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:00 smithi007 ceph-mon[32077]: mgrmap e11: x(active, since 1.01376s) 2024-06-13T06:42:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-06-13T06:42:01.046 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-13T06:42:01.046 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-13T06:42:01.047 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-06-13T06:42:01.047 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.042+0000 7f48d2e00700 1 Processor -- start 2024-06-13T06:42:01.047 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.042+0000 7f48d2e00700 1 -- start start 2024-06-13T06:42:01.047 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.043+0000 7f48d2e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.043+0000 7f48d2e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc07c890 con 0x7f48cc07de60 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.043+0000 7f48cbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.043+0000 7f48cbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46104/0 (socket says 172.21.15.7:46104) 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.043+0000 7f48cbe00700 1 -- 172.21.15.7:0/605744402 learned_addr learned my addr 172.21.15.7:0/605744402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.044+0000 7f48cbe00700 1 -- 172.21.15.7:0/605744402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc07ca10 con 0x7f48cc07de60 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.044+0000 7f48cbe00700 1 --2- 172.21.15.7:0/605744402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098a0 tx=0x7f48bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf845358b32272c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.044+0000 7f48caa00700 1 -- 172.21.15.7:0/605744402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48bc004160 con 0x7f48cc07de60 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.044+0000 7f48caa00700 1 -- 172.21.15.7:0/605744402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc004300 con 0x7f48cc07de60 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.044+0000 7f48caa00700 1 -- 172.21.15.7:0/605744402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48bc011b20 con 0x7f48cc07de60 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 -- 172.21.15.7:0/605744402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 msgr2=0x7f48cc07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 --2- 172.21.15.7:0/605744402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0098a0 tx=0x7f48bc009bb0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 -- 172.21.15.7:0/605744402 shutdown_connections 2024-06-13T06:42:01.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 --2- 172.21.15.7:0/605744402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc07c2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 -- 172.21.15.7:0/605744402 >> 172.21.15.7:0/605744402 conn(0x7f48cc103450 msgr2=0x7f48cc105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 -- 172.21.15.7:0/605744402 shutdown_connections 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.045+0000 7f48d2e00700 1 -- 172.21.15.7:0/605744402 wait complete. 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.046+0000 7f48d2e00700 1 Processor -- start 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48d2e00700 1 -- start start 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48d2e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48d2e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e510 con 0x7f48cc07de60 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48cbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:01.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48cbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46106/0 (socket says 172.21.15.7:46106) 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48cbe00700 1 -- 172.21.15.7:0/3081259034 learned_addr learned my addr 172.21.15.7:0/3081259034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.047+0000 7f48cbe00700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc19e460 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48cbe00700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0117e0 tx=0x7f48bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48bc003ba0 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48d2e00700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc19e6f0 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc01a910 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48bc0248f0 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.048+0000 7f48d2e00700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc19eb50 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.049+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f48bc022020 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.050+0000 7f48c8c00700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.050+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48b003bc70 con 0x7f48b00390a0 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.050+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48bc04d9e0 con 0x7f48cc07de60 2024-06-13T06:42:01.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.050+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.050+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.251+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.251+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.652+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:53.652+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:54.453+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:54.453+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:56.055+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:56.055+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:59.258+0000 7f48cb400700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3124714037 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:59.258+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-06-13T06:42:01.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:59.460+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f48bc02d3f0 con 0x7f48cc07de60 2024-06-13T06:42:01.057 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:59.461+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 msgr2=0x7f48b003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:41:59.461+0000 7f48c8c00700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.463+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f48bc04fa30 con 0x7f48cc07de60 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.463+0000 7f48c8c00700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 0x7f48b003ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.463+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48bc023ec0 con 0x7f48b003cb60 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.464+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 0x7f48b003ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.466+0000 7f48cb400700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 0x7f48b003ef50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48b8003a10 tx=0x7f48b80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.467+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f48bc023ec0 con 0x7f48b003cb60 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.476+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f48cc063af0 con 0x7f48b003cb60 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.477+0000 7f48c8c00700 1 -- 172.21.15.7:0/3081259034 <== mgr.14152 v2:172.21.15.7:6800/2901233239 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f48cc063af0 con 0x7f48b003cb60 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 msgr2=0x7f48b003ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:01.058 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 0x7f48b003ef50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48b8003a10 tx=0x7f48b80092b0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 msgr2=0x7f48cc19df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0117e0 tx=0x7f48bc011ba0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 shutdown_connections 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f48b003cb60 0x7f48b003ef50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48cc07de60 0x7f48cc19df20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 --2- 172.21.15.7:0/3081259034 >> [v2:172.21.15.7:6800/3124714037,v1:172.21.15.7:6801/3124714037] conn(0x7f48b00390a0 0x7f48b003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 >> 172.21.15.7:0/3081259034 conn(0x7f48cc103450 msgr2=0x7f48cc103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.478+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 shutdown_connections 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:00.479+0000 7f48c2000700 1 -- 172.21.15.7:0/3081259034 wait complete. 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-06-13T06:42:01.059 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-06-13T06:42:01.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:01 smithi007 ceph-mon[32077]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-13T06:42:01.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:01 smithi007 ceph-mon[32077]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-13T06:42:02.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:02 smithi007 ceph-mon[32077]: mgrmap e12: x(active, since 2s) 2024-06-13T06:42:02.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:02 smithi007 ceph-mon[32077]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:02.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:02.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:02.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-13T06:42:02.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.657+0000 7f1a99400700 1 Processor -- start 2024-06-13T06:42:02.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.657+0000 7f1a99400700 1 -- start start 2024-06-13T06:42:02.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.658+0000 7f1a99400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:02.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.658+0000 7f1a99400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94108640 con 0x7f1a94107c50 2024-06-13T06:42:02.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.659+0000 7f1a93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:02.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.659+0000 7f1a93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42178/0 (socket says 172.21.15.7:42178) 2024-06-13T06:42:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.659+0000 7f1a93e00700 1 -- 172.21.15.7:0/2296063958 learned_addr learned my addr 172.21.15.7:0/2296063958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.659+0000 7f1a93e00700 1 -- 172.21.15.7:0/2296063958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a94108e50 con 0x7f1a94107c50 2024-06-13T06:42:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.659+0000 7f1a93e00700 1 --2- 172.21.15.7:0/2296063958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5061842522f7f8a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.660+0000 7f1a93400700 1 -- 172.21.15.7:0/2296063958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84017070 con 0x7f1a94107c50 2024-06-13T06:42:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.660+0000 7f1a93400700 1 -- 172.21.15.7:0/2296063958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8400b8c0 con 0x7f1a94107c50 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.660+0000 7f1a93400700 1 -- 172.21.15.7:0/2296063958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a840045c0 con 0x7f1a94107c50 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.660+0000 7f1a99400700 1 -- 172.21.15.7:0/2296063958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 msgr2=0x7f1a94108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.660+0000 7f1a99400700 1 --2- 172.21.15.7:0/2296063958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a8400b330 comp rx=0 tx=0).stop 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.661+0000 7f1a99400700 1 -- 172.21.15.7:0/2296063958 shutdown_connections 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.661+0000 7f1a99400700 1 --2- 172.21.15.7:0/2296063958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a94108070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.661+0000 7f1a99400700 1 -- 172.21.15.7:0/2296063958 >> 172.21.15.7:0/2296063958 conn(0x7f1a941031f0 msgr2=0x7f1a94105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.661+0000 7f1a99400700 1 -- 172.21.15.7:0/2296063958 shutdown_connections 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.661+0000 7f1a99400700 1 -- 172.21.15.7:0/2296063958 wait complete. 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.662+0000 7f1a99400700 1 Processor -- start 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.662+0000 7f1a99400700 1 -- start start 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.662+0000 7f1a99400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.662+0000 7f1a99400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9410c200 con 0x7f1a94107c50 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.663+0000 7f1a93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:02.845 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.663+0000 7f1a93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42182/0 (socket says 172.21.15.7:42182) 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.663+0000 7f1a93e00700 1 -- 172.21.15.7:0/2634982249 learned_addr learned my addr 172.21.15.7:0/2634982249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.663+0000 7f1a93e00700 1 -- 172.21.15.7:0/2634982249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9407e090 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.663+0000 7f1a93e00700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009530 tx=0x7f1a84003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.664+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84020020 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.664+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8400bba0 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.664+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9407c280 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.664+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84017040 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.664+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9407c7c0 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.665+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1a84023020 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.665+0000 7f1a91600700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 0x7f1a7403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.665+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a94063af0 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.665+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a8400f050 con 0x7f1a94107c50 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.666+0000 7f1a83e00700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 0x7f1a7403b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:02.846 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.666+0000 7f1a83e00700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 0x7f1a7403b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006fd0 tx=0x7f1a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.672+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a8402a020 con 0x7f1a94107c50 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:01.908+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1a9407cfd0 con 0x7f1a74039100 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.001+0000 7f1a91600700 1 -- 172.21.15.7:0/2634982249 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1a9407cfd0 con 0x7f1a74039100 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 msgr2=0x7f1a7403b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 0x7f1a7403b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006fd0 tx=0x7f1a7c006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 msgr2=0x7f1a9407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009530 tx=0x7f1a84003d00 comp rx=0 tx=0).stop 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 shutdown_connections 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1a74039100 0x7f1a7403b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 --2- 172.21.15.7:0/2634982249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a94107c50 0x7f1a9407db50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 >> 172.21.15.7:0/2634982249 conn(0x7f1a941031f0 msgr2=0x7f1a94103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.004+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 shutdown_connections 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:02.005+0000 7f1a99400700 1 -- 172.21.15.7:0/2634982249 wait complete. 2024-06-13T06:42:02.847 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-06-13T06:42:03.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$XvIxC43FdCuARD7zkNDhkuNbJrJRZIZzq0/OGLkIfjLuv9HgezcI6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718260923, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.406+0000 7fc88b200700 1 Processor -- start 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.406+0000 7fc88b200700 1 -- start start 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.407+0000 7fc88b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.407+0000 7fc88b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c07bdd0 con 0x7fc88c1087c0 2024-06-13T06:42:04.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.407+0000 7fc889e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.408+0000 7fc889e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42198/0 (socket says 172.21.15.7:42198) 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.408+0000 7fc889e00700 1 -- 172.21.15.7:0/3896028283 learned_addr learned my addr 172.21.15.7:0/3896028283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.408+0000 7fc889e00700 1 -- 172.21.15.7:0/3896028283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c10b0f0 con 0x7fc88c1087c0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.408+0000 7fc889e00700 1 --2- 172.21.15.7:0/3896028283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0098a0 tx=0x7fc87c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6bb2c99da17969 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.409+0000 7fc888a00700 1 -- 172.21.15.7:0/3896028283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc87c004160 con 0x7fc88c1087c0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.409+0000 7fc888a00700 1 -- 172.21.15.7:0/3896028283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc87c004300 con 0x7fc88c1087c0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.409+0000 7fc888a00700 1 -- 172.21.15.7:0/3896028283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc87c011b20 con 0x7fc88c1087c0 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 -- 172.21.15.7:0/3896028283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 msgr2=0x7fc88c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 --2- 172.21.15.7:0/3896028283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0098a0 tx=0x7fc87c009bb0 comp rx=0 tx=0).stop 2024-06-13T06:42:04.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 -- 172.21.15.7:0/3896028283 shutdown_connections 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 --2- 172.21.15.7:0/3896028283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c10abb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 -- 172.21.15.7:0/3896028283 >> 172.21.15.7:0/3896028283 conn(0x7fc88c102130 msgr2=0x7fc88c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 -- 172.21.15.7:0/3896028283 shutdown_connections 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.410+0000 7fc88b200700 1 -- 172.21.15.7:0/3896028283 wait complete. 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.412+0000 7fc88b200700 1 Processor -- start 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.412+0000 7fc88b200700 1 -- start start 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.412+0000 7fc88b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.412+0000 7fc88b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10c0a0 con 0x7fc88c1087c0 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.413+0000 7fc889e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.413+0000 7fc889e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.413+0000 7fc889e00700 1 -- 172.21.15.7:0/2324689838 learned_addr learned my addr 172.21.15.7:0/2324689838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.413+0000 7fc889e00700 1 -- 172.21.15.7:0/2324689838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c19e320 con 0x7fc88c1087c0 2024-06-13T06:42:04.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc889e00700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0117e0 tx=0x7fc87c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc87c003ba0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc87c01a910 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c19e5b0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc87c0248f0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.414+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c19ea10 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.415+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc87c022020 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.416+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c063af0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.416+0000 7fc882a00700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 0x7fc87003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.416+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc87c04d5d0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.418+0000 7fc889400700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 0x7fc87003b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.418+0000 7fc889400700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 0x7fc87003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc878006fd0 tx=0x7fc878006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.422+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc87c00b3e0 con 0x7fc88c1087c0 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.681+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc88c1a1440 con 0x7fc870039100 2024-06-13T06:42:04.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.949+0000 7fc882a00700 1 -- 172.21.15.7:0/2324689838 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc88c1a1440 con 0x7fc870039100 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 msgr2=0x7fc87003b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 0x7fc87003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc878006fd0 tx=0x7fc878006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 msgr2=0x7fc88c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0117e0 tx=0x7fc87c011ba0 comp rx=0 tx=0).stop 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 shutdown_connections 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc870039100 0x7fc87003b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 --2- 172.21.15.7:0/2324689838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc88c1087c0 0x7fc88c19dde0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 >> 172.21.15.7:0/2324689838 conn(0x7fc88c102130 msgr2=0x7fc88c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 shutdown_connections 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:03.952+0000 7fc88b200700 1 -- 172.21.15.7:0/2324689838 wait complete. 2024-06-13T06:42:04.594 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-06-13T06:42:05.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:04 smithi007 ceph-mon[32077]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:05.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:05.130 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:06.156 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:05 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4071337369' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-13T06:42:06.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.298+0000 7f55e3a00700 1 Processor -- start 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.299+0000 7f55e3a00700 1 -- start start 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.299+0000 7f55e3a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.299+0000 7f55e3a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc10ab20 con 0x7f55dc10a130 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.300+0000 7f55e1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.300+0000 7f55e1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42220/0 (socket says 172.21.15.7:42220) 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.300+0000 7f55e1000700 1 -- 172.21.15.7:0/2871198867 learned_addr learned my addr 172.21.15.7:0/2871198867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:06.173 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.301+0000 7f55e1000700 1 -- 172.21.15.7:0/2871198867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc10b2a0 con 0x7f55dc10a130 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.301+0000 7f55e1000700 1 --2- 172.21.15.7:0/2871198867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098a0 tx=0x7f55d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46beffceb01dec5b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.301+0000 7f55db400700 1 -- 172.21.15.7:0/2871198867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d0004160 con 0x7f55dc10a130 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.301+0000 7f55db400700 1 -- 172.21.15.7:0/2871198867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0004300 con 0x7f55dc10a130 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.301+0000 7f55db400700 1 -- 172.21.15.7:0/2871198867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d0011b20 con 0x7f55dc10a130 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.302+0000 7f55e3a00700 1 -- 172.21.15.7:0/2871198867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 msgr2=0x7f55dc10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.302+0000 7f55e3a00700 1 --2- 172.21.15.7:0/2871198867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098a0 tx=0x7f55d0009bb0 comp rx=0 tx=0).stop 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.302+0000 7f55e3a00700 1 -- 172.21.15.7:0/2871198867 shutdown_connections 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.302+0000 7f55e3a00700 1 --2- 172.21.15.7:0/2871198867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc10a130 0x7f55dc10a550 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.302+0000 7f55e3a00700 1 -- 172.21.15.7:0/2871198867 >> 172.21.15.7:0/2871198867 conn(0x7f55dc07d6a0 msgr2=0x7f55dc07dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.303+0000 7f55e3a00700 1 -- 172.21.15.7:0/2871198867 shutdown_connections 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.303+0000 7f55e3a00700 1 -- 172.21.15.7:0/2871198867 wait complete. 2024-06-13T06:42:06.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.304+0000 7f55e3a00700 1 Processor -- start 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.304+0000 7f55e3a00700 1 -- start start 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.304+0000 7f55e3a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.304+0000 7f55e3a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc10e650 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.305+0000 7f55e1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.305+0000 7f55e1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42222/0 (socket says 172.21.15.7:42222) 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.305+0000 7f55e1000700 1 -- 172.21.15.7:0/4071337369 learned_addr learned my addr 172.21.15.7:0/4071337369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.305+0000 7f55e1000700 1 -- 172.21.15.7:0/4071337369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc19eb70 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55e1000700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55dc10b100 tx=0x7f55d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d00036a0 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc19ee00 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d001a460 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55d001b440 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.306+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a18e0 con 0x7f55dc19e210 2024-06-13T06:42:06.175 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.307+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc063af0 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.307+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f55d0003840 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.308+0000 7f55d9600700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 0x7f55c403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.308+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f55d004d7e0 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.308+0000 7f55dbe00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 0x7f55c403b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.309+0000 7f55dbe00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 0x7f55c403b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.314+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55d000be50 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.548+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f55dc1a1c20 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.549+0000 7f55d9600700 1 -- 172.21.15.7:0/4071337369 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f55d0003b30 con 0x7f55dc19e210 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 msgr2=0x7f55c403b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 0x7f55c403b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 msgr2=0x7f55dc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f55dc10b100 tx=0x7f55d0011ba0 comp rx=0 tx=0).stop 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 shutdown_connections 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f55c4039100 0x7f55c403b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 --2- 172.21.15.7:0/4071337369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55dc19e210 0x7f55dc19e630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 >> 172.21.15.7:0/4071337369 conn(0x7f55dc07d6a0 msgr2=0x7f55dc107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 shutdown_connections 2024-06-13T06:42:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:05.552+0000 7f55e3a00700 1 -- 172.21.15.7:0/4071337369 wait complete. 2024-06-13T06:42:06.204 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-13T06:42:06.204 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-06-13T06:42:06.228 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-13T06:42:06.228 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-06-13T06:42:06.229 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-06-13T06:42:06.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout: Password: ji7pxa48wv 2024-06-13T06:42:06.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:06.240 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config directory 2024-06-13T06:42:06.242 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-06-13T06:42:07.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:06 smithi007 ceph-mon[32077]: mgrmap e13: x(active, since 6s) 2024-06-13T06:42:07.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.895+0000 7fa037400700 1 Processor -- start 2024-06-13T06:42:07.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.895+0000 7fa037400700 1 -- start start 2024-06-13T06:42:07.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa037400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:07.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa037400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0300828a0 con 0x7fa030081eb0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa034a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa034a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42228/0 (socket says 172.21.15.7:42228) 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa034a00700 1 -- 172.21.15.7:0/2460601986 learned_addr learned my addr 172.21.15.7:0/2460601986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.896+0000 7fa034a00700 1 -- 172.21.15.7:0/2460601986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03010f680 con 0x7fa030081eb0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.897+0000 7fa034a00700 1 --2- 172.21.15.7:0/2460601986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098a0 tx=0x7fa024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7afbdf5671f2fc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.897+0000 7fa02f400700 1 -- 172.21.15.7:0/2460601986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa024004160 con 0x7fa030081eb0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.897+0000 7fa02f400700 1 -- 172.21.15.7:0/2460601986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa024004300 con 0x7fa030081eb0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.897+0000 7fa02f400700 1 -- 172.21.15.7:0/2460601986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa024011b20 con 0x7fa030081eb0 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 -- 172.21.15.7:0/2460601986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 msgr2=0x7fa0300822d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 --2- 172.21.15.7:0/2460601986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0240098a0 tx=0x7fa024009bb0 comp rx=0 tx=0).stop 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 -- 172.21.15.7:0/2460601986 shutdown_connections 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 --2- 172.21.15.7:0/2460601986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0300822d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 -- 172.21.15.7:0/2460601986 >> 172.21.15.7:0/2460601986 conn(0x7fa03007d450 msgr2=0x7fa03007f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 -- 172.21.15.7:0/2460601986 shutdown_connections 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.898+0000 7fa037400700 1 -- 172.21.15.7:0/2460601986 wait complete. 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.899+0000 7fa037400700 1 Processor -- start 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa037400700 1 -- start start 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa037400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa037400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa030112a30 con 0x7fa030081eb0 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa034a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:07.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa034a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42234/0 (socket says 172.21.15.7:42234) 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.900+0000 7fa034a00700 1 -- 172.21.15.7:0/4062258901 learned_addr learned my addr 172.21.15.7:0/4062258901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.901+0000 7fa034a00700 1 -- 172.21.15.7:0/4062258901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0301a28e0 con 0x7fa030081eb0 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.901+0000 7fa034a00700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa024000c00 tx=0x7fa024011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.902+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa024003b60 con 0x7fa030081eb0 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.902+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa02401a910 con 0x7fa030081eb0 2024-06-13T06:42:07.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.902+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0301a2b70 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.902+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa024024930 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.902+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0301a2fd0 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.903+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa024022020 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.903+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa030063af0 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.904+0000 7fa02d600700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 0x7fa01803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.904+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa02404d560 con 0x7fa030081eb0 2024-06-13T06:42:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.904+0000 7fa02fe00700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 0x7fa01803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.905+0000 7fa02fe00700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 0x7fa01803b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:06.909+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa02402c350 con 0x7fa030081eb0 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.146+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa0301a59a0 con 0x7fa030081eb0 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.147+0000 7fa02d600700 1 -- 172.21.15.7:0/4062258901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa02401ab30 con 0x7fa030081eb0 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 msgr2=0x7fa01803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 0x7fa01803b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa01c006fd0 tx=0x7fa01c006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 msgr2=0x7fa0301a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa024000c00 tx=0x7fa024011c10 comp rx=0 tx=0).stop 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 shutdown_connections 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0180391a0 0x7fa01803b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 --2- 172.21.15.7:0/4062258901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa030081eb0 0x7fa0301a23a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 >> 172.21.15.7:0/4062258901 conn(0x7fa03007d450 msgr2=0x7fa03007df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 shutdown_connections 2024-06-13T06:42:07.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:07.150+0000 7fa037400700 1 -- 172.21.15.7:0/4062258901 wait complete. 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.953+0000 7f1ad1c00700 1 Processor -- start 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.953+0000 7f1ad1c00700 1 -- start start 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.954+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.954+0000 7f1ad1c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1067e0 con 0x7f1acc105df0 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.954+0000 7f1acaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.954+0000 7f1acaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42240/0 (socket says 172.21.15.7:42240) 2024-06-13T06:42:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.954+0000 7f1acaa00700 1 -- 172.21.15.7:0/4098734690 learned_addr learned my addr 172.21.15.7:0/4098734690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.955+0000 7f1acaa00700 1 -- 172.21.15.7:0/4098734690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc106e70 con 0x7f1acc105df0 2024-06-13T06:42:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.955+0000 7f1acaa00700 1 --2- 172.21.15.7:0/4098734690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0098a0 tx=0x7f1abc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189d7fd735f295b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.955+0000 7f1ac9600700 1 -- 172.21.15.7:0/4098734690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1abc004160 con 0x7f1acc105df0 2024-06-13T06:42:09.999 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.955+0000 7f1ac9600700 1 -- 172.21.15.7:0/4098734690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1abc004300 con 0x7f1acc105df0 2024-06-13T06:42:09.999 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.955+0000 7f1ac9600700 1 -- 172.21.15.7:0/4098734690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1abc011b20 con 0x7f1acc105df0 2024-06-13T06:42:09.999 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 -- 172.21.15.7:0/4098734690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 msgr2=0x7f1acc106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:09.999 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/4098734690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0098a0 tx=0x7f1abc009bb0 comp rx=0 tx=0).stop 2024-06-13T06:42:09.999 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 -- 172.21.15.7:0/4098734690 shutdown_connections 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/4098734690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc106210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 -- 172.21.15.7:0/4098734690 >> 172.21.15.7:0/4098734690 conn(0x7f1acc101620 msgr2=0x7f1acc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 -- 172.21.15.7:0/4098734690 shutdown_connections 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.956+0000 7f1ad1c00700 1 -- 172.21.15.7:0/4098734690 wait complete. 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.957+0000 7f1ad1c00700 1 Processor -- start 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1ad1c00700 1 -- start start 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1ad1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1ad1c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc108230 con 0x7f1acc105df0 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1acaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1acaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42244/0 (socket says 172.21.15.7:42244) 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.958+0000 7f1acaa00700 1 -- 172.21.15.7:0/2018042954 learned_addr learned my addr 172.21.15.7:0/2018042954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.959+0000 7f1acaa00700 1 -- 172.21.15.7:0/2018042954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19e4b0 con 0x7f1acc105df0 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.959+0000 7f1acaa00700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0117e0 tx=0x7f1abc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:10.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.959+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1abc003ba0 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.960+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1abc01a910 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.960+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1abc0248f0 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.960+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19e740 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.960+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc19eba0 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.961+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1abc022020 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.961+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc063af0 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.961+0000 7f1ac3400700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 0x7f1ab003b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.961+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1abc04d740 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.962+0000 7f1aca000700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 0x7f1ab003b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.962+0000 7f1aca000700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 0x7f1ab003b6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:08.967+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1abc01ab70 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.317+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1acc0630a0 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.322+0000 7f1ac3400700 1 -- 172.21.15.7:0/2018042954 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1abc010380 con 0x7f1acc105df0 2024-06-13T06:42:10.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 msgr2=0x7f1ab003b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 0x7f1ab003b6b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 msgr2=0x7f1acc19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0117e0 tx=0x7f1abc011ba0 comp rx=0 tx=0).stop 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 shutdown_connections 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1ab00391f0 0x7f1ab003b6b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 --2- 172.21.15.7:0/2018042954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc105df0 0x7f1acc19df70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 >> 172.21.15.7:0/2018042954 conn(0x7f1acc101620 msgr2=0x7f1acc194740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 shutdown_connections 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-06-13T06:42:09.326+0000 7f1ad1c00700 1 -- 172.21.15.7:0/2018042954 wait complete. 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:10.003 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-06-13T06:42:10.087 INFO:tasks.cephadm:Fetching config... 2024-06-13T06:42:10.088 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:42:10.088 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-13T06:42:10.137 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-13T06:42:10.137 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:42:10.137 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-13T06:42:10.219 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-13T06:42:10.219 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:42:10.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-13T06:42:10.346 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-13T06:42:10.346 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:42:10.346 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-13T06:42:10.399 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-13T06:42:10.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:42:10.514 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:10.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:10 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2018042954' entity='client.admin' 2024-06-13T06:42:10.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:42:10.586 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:10.597 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-13T06:42:10.637 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/jG4qOzxsNXR3mr2ODFts4uImQZSlDMdGbAPDzUSjb4NMgMrmWufOCWbyw/6rJeMo6sTROLGpeze9zroPMflWp+wzCPpx+4fHPO/LSdvJ7i5jTZXn5sV+rp/5Vp/Elw3He3VT5D+A0pEWNwu3GTOGe0VJMPJHsu+b2PT2CMhwoy8vsH5SPzW9eVL30GUMO7I9NQBYtJiTztjAtWQnMol0ugfW+j+WQ/NKBkY5d5apDijc68qNvbsXugo7C67cJ6W5GADzvn1ue5edg2I9vzQtkrQCA1zXydZD29yFvjNPOVxuUKvifC657nHzotrb81YiYsVPmYMnF0Omqyio0Eeddo6pmhNT5YlzK74AggyVWOjrEq5KivB+2nUxPiMv9TjiSE0NZwxLwQzY69UjtnSQo+2UYsHnwyAaTCPQJfeLP/pRv6YIXiohHgGBP/zesq4OkgFcM1jkQiUGhwj6l+RX4jnKXxdoC/TSVVI/4CDo7q32cfvOh1rszgu5BKp+ajU= ceph-b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:10.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-13T06:42:11.086 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 -- 172.21.15.7:0/1675121898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 msgr2=0x7ff1280748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 --2- 172.21.15.7:0/1675121898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1280748c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff118009910 tx=0x7ff118009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 -- 172.21.15.7:0/1675121898 shutdown_connections 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 --2- 172.21.15.7:0/1675121898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1280748c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 -- 172.21.15.7:0/1675121898 >> 172.21.15.7:0/1675121898 conn(0x7ff1280fd490 msgr2=0x7ff1280ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 -- 172.21.15.7:0/1675121898 shutdown_connections 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.766+0000 7ff12d800700 1 -- 172.21.15.7:0/1675121898 wait complete. 2024-06-13T06:42:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.768+0000 7ff12d800700 1 Processor -- start 2024-06-13T06:42:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.768+0000 7ff12d800700 1 -- start start 2024-06-13T06:42:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.768+0000 7ff12d800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.768+0000 7ff12d800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff12810a170 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.769+0000 7ff126a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.769+0000 7ff126a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56492/0 (socket says 172.21.15.7:56492) 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.769+0000 7ff126a00700 1 -- 172.21.15.7:0/3151106248 learned_addr learned my addr 172.21.15.7:0/3151106248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.769+0000 7ff126a00700 1 -- 172.21.15.7:0/3151106248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128199cf0 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.769+0000 7ff126a00700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff118004fa0 tx=0x7ff118005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.770+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff118027070 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.770+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff11800be40 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.770+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff128199f80 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.770+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff11800f460 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.770+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12819cc70 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.771+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff11802a070 con 0x7ff128076460 2024-06-13T06:42:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.771+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12810a8f0 con 0x7ff128076460 2024-06-13T06:42:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.772+0000 7ff11f400700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 0x7ff10c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.772+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff11801e030 con 0x7ff128076460 2024-06-13T06:42:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.772+0000 7ff126000700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 0x7ff10c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:13.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.773+0000 7ff126000700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 0x7ff10c03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff114006fd0 tx=0x7ff114006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:13.778+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff118013070 con 0x7ff128076460 2024-06-13T06:42:14.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.007+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff12802ddb0 con 0x7ff128076460 2024-06-13T06:42:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.011+0000 7ff11f400700 1 -- 172.21.15.7:0/3151106248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff11802bde0 con 0x7ff128076460 2024-06-13T06:42:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.014+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 msgr2=0x7ff10c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.014+0000 7ff12d800700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 0x7ff10c03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff114006fd0 tx=0x7ff114006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.014+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 msgr2=0x7ff1281997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.014+0000 7ff12d800700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff118004fa0 tx=0x7ff118005080 comp rx=0 tx=0).stop 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 shutdown_connections 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff10c039150 0x7ff10c03b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 --2- 172.21.15.7:0/3151106248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff128076460 0x7ff1281997b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 >> 172.21.15.7:0/3151106248 conn(0x7ff1280fd490 msgr2=0x7ff1280fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 shutdown_connections 2024-06-13T06:42:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:14.015+0000 7ff12d800700 1 -- 172.21.15.7:0/3151106248 wait complete. 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:14.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3151106248' entity='client.admin' 2024-06-13T06:42:14.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:14.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:14.710 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-13T06:42:14.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-13T06:42:15.162 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.018+0000 7feafa400700 1 -- 172.21.15.7:0/2818183318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 msgr2=0x7feaf4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.018+0000 7feafa400700 1 --2- 172.21.15.7:0/2818183318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf4108ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feae8009910 tx=0x7feae8009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.019+0000 7feafa400700 1 -- 172.21.15.7:0/2818183318 shutdown_connections 2024-06-13T06:42:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.019+0000 7feafa400700 1 --2- 172.21.15.7:0/2818183318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf4108ac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.019+0000 7feafa400700 1 -- 172.21.15.7:0/2818183318 >> 172.21.15.7:0/2818183318 conn(0x7feaf4077300 msgr2=0x7feaf4079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.019+0000 7feafa400700 1 -- 172.21.15.7:0/2818183318 shutdown_connections 2024-06-13T06:42:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.019+0000 7feafa400700 1 -- 172.21.15.7:0/2818183318 wait complete. 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.020+0000 7feafa400700 1 Processor -- start 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.020+0000 7feafa400700 1 -- start start 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.020+0000 7feafa400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.020+0000 7feafa400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feaf410e3c0 con 0x7feaf41086a0 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.021+0000 7feaf9000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.021+0000 7feaf9000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56504/0 (socket says 172.21.15.7:56504) 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.021+0000 7feaf9000700 1 -- 172.21.15.7:0/3201569853 learned_addr learned my addr 172.21.15.7:0/3201569853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:17.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.021+0000 7feaf9000700 1 -- 172.21.15.7:0/3201569853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf419df10 con 0x7feaf41086a0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.021+0000 7feaf9000700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feae8005850 tx=0x7feae8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.022+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae8026070 con 0x7feaf41086a0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.022+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae80044a0 con 0x7feaf41086a0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.022+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf419e1a0 con 0x7feaf41086a0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.022+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feae8003ab0 con 0x7feaf41086a0 2024-06-13T06:42:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.022+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf41a0e90 con 0x7feaf41086a0 2024-06-13T06:42:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.023+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7feae8029070 con 0x7feaf41086a0 2024-06-13T06:42:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.023+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf410eb40 con 0x7feaf41086a0 2024-06-13T06:42:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.024+0000 7feaf1600700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 0x7feadc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.024+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feae801e070 con 0x7feaf41086a0 2024-06-13T06:42:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.024+0000 7feaf3e00700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 0x7feadc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:17.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.025+0000 7feaf3e00700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 0x7feadc03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.029+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feae8013070 con 0x7feaf41086a0 2024-06-13T06:42:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.265+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7feaf402ddb0 con 0x7feadc039150 2024-06-13T06:42:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.271+0000 7feaf1600700 1 -- 172.21.15.7:0/3201569853 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feaf402ddb0 con 0x7feadc039150 2024-06-13T06:42:17.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 msgr2=0x7feadc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:17.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 0x7feadc03b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feae4006fd0 tx=0x7feae4006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:17.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 msgr2=0x7feaf419d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feae8005850 tx=0x7feae8005030 comp rx=0 tx=0).stop 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 shutdown_connections 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feadc039150 0x7feadc03b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 --2- 172.21.15.7:0/3201569853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf41086a0 0x7feaf419d9d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.274+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 >> 172.21.15.7:0/3201569853 conn(0x7feaf4077300 msgr2=0x7feaf4077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.275+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 shutdown_connections 2024-06-13T06:42:17.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:17.275+0000 7feafa400700 1 -- 172.21.15.7:0/3201569853 wait complete. 2024-06-13T06:42:17.997 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-06-13T06:42:17.998 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:42:17.998 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-06-13T06:42:18.016 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:42:18.016 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:18.072 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-06-13T06:42:18.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch host add smithi175 2024-06-13T06:42:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:18.631 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:18 smithi007 ceph-mon[32077]: Updating smithi007:/etc/ceph/ceph.conf 2024-06-13T06:42:19.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:19 smithi007 ceph-mon[32077]: Updating smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:20.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:20 smithi007 ceph-mon[32077]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:20.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:20 smithi007 ceph-mon[32077]: Updating smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:42:20.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:20.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:20.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.573+0000 7f37d1200700 1 -- 172.21.15.7:0/2740351059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 msgr2=0x7f37cc103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.573+0000 7f37d1200700 1 --2- 172.21.15.7:0/2740351059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc103480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009910 tx=0x7f37bc009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.574+0000 7f37d1200700 1 -- 172.21.15.7:0/2740351059 shutdown_connections 2024-06-13T06:42:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.574+0000 7f37d1200700 1 --2- 172.21.15.7:0/2740351059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc103480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.574+0000 7f37d1200700 1 -- 172.21.15.7:0/2740351059 >> 172.21.15.7:0/2740351059 conn(0x7f37cc0fc890 msgr2=0x7f37cc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.574+0000 7f37d1200700 1 -- 172.21.15.7:0/2740351059 shutdown_connections 2024-06-13T06:42:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.574+0000 7f37d1200700 1 -- 172.21.15.7:0/2740351059 wait complete. 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.575+0000 7f37d1200700 1 Processor -- start 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.575+0000 7f37d1200700 1 -- start start 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.575+0000 7f37d1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.575+0000 7f37d1200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc10c520 con 0x7f37cc1030a0 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.576+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.576+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56516/0 (socket says 172.21.15.7:56516) 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.576+0000 7f37c9e00700 1 -- 172.21.15.7:0/3831542523 learned_addr learned my addr 172.21.15.7:0/3831542523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.576+0000 7f37c9e00700 1 -- 172.21.15.7:0/3831542523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc102bf0 con 0x7f37cc1030a0 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.576+0000 7f37c9e00700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37bc006010 tx=0x7f37bc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.577+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37bc026070 con 0x7f37cc1030a0 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.577+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc00bc30 con 0x7f37cc1030a0 2024-06-13T06:42:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.577+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37bc0039a0 con 0x7f37cc1030a0 2024-06-13T06:42:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.577+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc101240 con 0x7f37cc1030a0 2024-06-13T06:42:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.577+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc101700 con 0x7f37cc1030a0 2024-06-13T06:42:20.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.578+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f37bc029070 con 0x7f37cc1030a0 2024-06-13T06:42:20.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.578+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc06a9c0 con 0x7f37cc1030a0 2024-06-13T06:42:20.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.578+0000 7f37c2a00700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 0x7f37b003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:20.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.579+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f37bc01e070 con 0x7f37cc1030a0 2024-06-13T06:42:20.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.579+0000 7f37c9400700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 0x7f37b003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.579+0000 7f37c9400700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 0x7f37b003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37b8006fd0 tx=0x7f37b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:20.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.584+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37bc013070 con 0x7f37cc1030a0 2024-06-13T06:42:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:20.820+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) -- 0x7f37cc1022a0 con 0x7f37b0039150 2024-06-13T06:42:21.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:21 smithi007 ceph-mon[32077]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:22.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:22 smithi007 ceph-mon[32077]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:23.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:23 smithi007 ceph-mon[32077]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:23.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:23 smithi007 ceph-mon[32077]: Deploying cephadm binary to smithi175 2024-06-13T06:42:25.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:25 smithi007 ceph-mon[32077]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.795+0000 7f37c2a00700 1 -- 172.21.15.7:0/3831542523 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f37cc1022a0 con 0x7f37b0039150 2024-06-13T06:42:25.795 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 msgr2=0x7f37b003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 0x7f37b003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37b8006fd0 tx=0x7f37b8006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 msgr2=0x7f37cc100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37bc006010 tx=0x7f37bc005030 comp rx=0 tx=0).stop 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 shutdown_connections 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f37b0039150 0x7f37b003b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 --2- 172.21.15.7:0/3831542523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37cc1030a0 0x7f37cc100d00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.799+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 >> 172.21.15.7:0/3831542523 conn(0x7f37cc0fc890 msgr2=0x7f37cc103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.800+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 shutdown_connections 2024-06-13T06:42:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:25.800+0000 7f37d1200700 1 -- 172.21.15.7:0/3831542523 wait complete. 2024-06-13T06:42:26.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch host ls --format=json 2024-06-13T06:42:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:27.059 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:26 smithi007 ceph-mon[32077]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:27.059 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:27.059 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:26 smithi007 ceph-mon[32077]: Added host smithi175 2024-06-13T06:42:27.059 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:27.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:28.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:28.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 -- 172.21.15.7:0/2419592296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 msgr2=0x7f29c410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:28.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 --2- 172.21.15.7:0/2419592296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c410e7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009910 tx=0x7f29b4009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 -- 172.21.15.7:0/2419592296 shutdown_connections 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 --2- 172.21.15.7:0/2419592296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c410e7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 -- 172.21.15.7:0/2419592296 >> 172.21.15.7:0/2419592296 conn(0x7f29c4077aa0 msgr2=0x7f29c4079ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 -- 172.21.15.7:0/2419592296 shutdown_connections 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.919+0000 7f29c9a00700 1 -- 172.21.15.7:0/2419592296 wait complete. 2024-06-13T06:42:28.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.921+0000 7f29c9a00700 1 Processor -- start 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.921+0000 7f29c9a00700 1 -- start start 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.921+0000 7f29c9a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.921+0000 7f29c9a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29c4112920 con 0x7f29c410e3d0 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.921+0000 7f29c3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.922+0000 7f29c3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58836/0 (socket says 172.21.15.7:58836) 2024-06-13T06:42:28.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.922+0000 7f29c3e00700 1 -- 172.21.15.7:0/1881228428 learned_addr learned my addr 172.21.15.7:0/1881228428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:28.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.922+0000 7f29c3e00700 1 -- 172.21.15.7:0/1881228428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c419e260 con 0x7f29c410e3d0 2024-06-13T06:42:28.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.922+0000 7f29c3e00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006010 tx=0x7f29b4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:28.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.922+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29b4026070 con 0x7f29c410e3d0 2024-06-13T06:42:28.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.923+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b400bbd0 con 0x7f29c410e3d0 2024-06-13T06:42:28.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.923+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c419e550 con 0x7f29c410e3d0 2024-06-13T06:42:28.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.923+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29b4003a30 con 0x7f29c410e3d0 2024-06-13T06:42:28.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.923+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c41a25f0 con 0x7f29c410e3d0 2024-06-13T06:42:28.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.924+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f29b4029070 con 0x7f29c410e3d0 2024-06-13T06:42:28.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.924+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c4109a10 con 0x7f29c410e3d0 2024-06-13T06:42:28.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.924+0000 7f29c0c00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 0x7f29a803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:28.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.924+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29b401e070 con 0x7f29c410e3d0 2024-06-13T06:42:28.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.925+0000 7f29c3400700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 0x7f29a803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:28.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.925+0000 7f29c3400700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 0x7f29a803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:28.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:28.930+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29b4013070 con 0x7f29c410e3d0 2024-06-13T06:42:29.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:28 smithi007 ceph-mon[32077]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:29.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:29.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.166+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29c419f080 con 0x7f29a8039150 2024-06-13T06:42:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.169+0000 7f29c0c00700 1 -- 172.21.15.7:0/1881228428 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f29c419f080 con 0x7f29a8039150 2024-06-13T06:42:29.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:29.168 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-06-13T06:42:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 msgr2=0x7f29a803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 0x7f29a803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 msgr2=0x7f29c419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006010 tx=0x7f29b4005050 comp rx=0 tx=0).stop 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 shutdown_connections 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f29a8039150 0x7f29a803b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 --2- 172.21.15.7:0/1881228428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c410e3d0 0x7f29c419dd20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.172+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 >> 172.21.15.7:0/1881228428 conn(0x7f29c4077aa0 msgr2=0x7f29c40792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.173+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 shutdown_connections 2024-06-13T06:42:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:29.173+0000 7f29c9a00700 1 -- 172.21.15.7:0/1881228428 wait complete. 2024-06-13T06:42:29.888 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-06-13T06:42:29.888 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:42:29.888 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-06-13T06:42:29.908 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:42:29.908 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:29.965 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-06-13T06:42:29.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch host add smithi189 2024-06-13T06:42:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:29 smithi007 ceph-mon[32077]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:42:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:31.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:30 smithi007 ceph-mon[32077]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 -- 172.21.15.7:0/1074865480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 msgr2=0x7fc6481024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1074865480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc6481024e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc638009910 tx=0x7fc638009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 -- 172.21.15.7:0/1074865480 shutdown_connections 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1074865480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc6481024e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 -- 172.21.15.7:0/1074865480 >> 172.21.15.7:0/1074865480 conn(0x7fc6480fd9d0 msgr2=0x7fc6480ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 -- 172.21.15.7:0/1074865480 shutdown_connections 2024-06-13T06:42:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.355+0000 7fc64ca00700 1 -- 172.21.15.7:0/1074865480 wait complete. 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.356+0000 7fc64ca00700 1 Processor -- start 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.356+0000 7fc64ca00700 1 -- start start 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.356+0000 7fc64ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.356+0000 7fc64ca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc64810c4d0 con 0x7fc648102100 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.357+0000 7fc645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.357+0000 7fc645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36878/0 (socket says 172.21.15.7:36878) 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.357+0000 7fc645e00700 1 -- 172.21.15.7:0/1189045453 learned_addr learned my addr 172.21.15.7:0/1189045453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.357+0000 7fc645e00700 1 -- 172.21.15.7:0/1189045453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64819fd60 con 0x7fc648102100 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc645e00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc638006010 tx=0x7fc638005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc638026070 con 0x7fc648102100 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc63800bbd0 con 0x7fc648102100 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc638003910 con 0x7fc648102100 2024-06-13T06:42:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6481a0050 con 0x7fc648102100 2024-06-13T06:42:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.358+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6481999e0 con 0x7fc648102100 2024-06-13T06:42:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.360+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6481035e0 con 0x7fc648102100 2024-06-13T06:42:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.360+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc638029070 con 0x7fc648102100 2024-06-13T06:42:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.360+0000 7fc63ea00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 0x7fc62c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.360+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc63801e070 con 0x7fc648102100 2024-06-13T06:42:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.361+0000 7fc645400700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 0x7fc62c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.362+0000 7fc645400700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 0x7fc62c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.367+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc638013070 con 0x7fc648102100 2024-06-13T06:42:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:32.605+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7fc64819af80 con 0x7fc62c039150 2024-06-13T06:42:32.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:32.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:32 smithi007 ceph-mon[32077]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:33 smithi007 ceph-mon[32077]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:33 smithi007 ceph-mon[32077]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:42:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:35.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:34 smithi007 ceph-mon[32077]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:35.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:34 smithi007 ceph-mon[32077]: Deploying cephadm binary to smithi189 2024-06-13T06:42:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:36 smithi007 ceph-mon[32077]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.884+0000 7fc63ea00700 1 -- 172.21.15.7:0/1189045453 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc64819af80 con 0x7fc62c039150 2024-06-13T06:42:37.884 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-06-13T06:42:37.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 msgr2=0x7fc62c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:37.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 0x7fc62c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:37.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 msgr2=0x7fc64819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:37.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc638006010 tx=0x7fc638005030 comp rx=0 tx=0).stop 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 shutdown_connections 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc62c039150 0x7fc62c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 --2- 172.21.15.7:0/1189045453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc648102100 0x7fc64819f820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 >> 172.21.15.7:0/1189045453 conn(0x7fc6480fd9d0 msgr2=0x7fc6480746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 shutdown_connections 2024-06-13T06:42:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:37.889+0000 7fc64ca00700 1 -- 172.21.15.7:0/1189045453 wait complete. 2024-06-13T06:42:38.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch host ls --format=json 2024-06-13T06:42:39.087 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:39.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:38 smithi007 ceph-mon[32077]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:39.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:38 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:39.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:38 smithi007 ceph-mon[32077]: Added host smithi189 2024-06-13T06:42:39.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:38 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:39.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:38 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:40.235 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 -- 172.21.15.7:0/1363319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 msgr2=0x7f8da0104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 --2- 172.21.15.7:0/1363319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da0104700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009910 tx=0x7f8d94009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 -- 172.21.15.7:0/1363319660 shutdown_connections 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 --2- 172.21.15.7:0/1363319660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da0104700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 -- 172.21.15.7:0/1363319660 >> 172.21.15.7:0/1363319660 conn(0x7f8da00ffbb0 msgr2=0x7f8da0101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 -- 172.21.15.7:0/1363319660 shutdown_connections 2024-06-13T06:42:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.090+0000 7f8da7600700 1 -- 172.21.15.7:0/1363319660 wait complete. 2024-06-13T06:42:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.091+0000 7f8da7600700 1 Processor -- start 2024-06-13T06:42:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da7600700 1 -- start start 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da7600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da7600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010e6f0 con 0x7f8da0104320 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da4c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da4c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48702/0 (socket says 172.21.15.7:48702) 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.092+0000 7f8da4c00700 1 -- 172.21.15.7:0/319277684 learned_addr learned my addr 172.21.15.7:0/319277684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8da4c00700 1 -- 172.21.15.7:0/319277684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da019fd60 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8da4c00700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d94006010 tx=0x7f8d94005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d94026070 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da01a0050 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d9400bbd0 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.093+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d94003910 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.094+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da01999e0 con 0x7f8da0104320 2024-06-13T06:42:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.095+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da0105800 con 0x7f8da0104320 2024-06-13T06:42:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.095+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f8d94029070 con 0x7f8da0104320 2024-06-13T06:42:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.095+0000 7f8d9d600700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 0x7f8d8803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.095+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8d9401e070 con 0x7f8da0104320 2024-06-13T06:42:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.095+0000 7f8d9fe00700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 0x7f8d8803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.096+0000 7f8d9fe00700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 0x7f8d8803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:41.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.101+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d94013070 con 0x7f8da0104320 2024-06-13T06:42:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.338+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8da019af80 con 0x7f8d88039150 2024-06-13T06:42:41.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:40 smithi007 ceph-mon[32077]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:41.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:41.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.341+0000 7f8d9d600700 1 -- 172.21.15.7:0/319277684 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8da019af80 con 0x7f8d88039150 2024-06-13T06:42:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:42:41.340 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 msgr2=0x7f8d8803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 0x7f8d8803b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c006fd0 tx=0x7f8d8c006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 msgr2=0x7f8da019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d94006010 tx=0x7f8d94005030 comp rx=0 tx=0).stop 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 shutdown_connections 2024-06-13T06:42:41.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8d88039150 0x7f8d8803b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:41.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 --2- 172.21.15.7:0/319277684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8da0104320 0x7f8da019f820 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:41.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 >> 172.21.15.7:0/319277684 conn(0x7f8da00ffbb0 msgr2=0x7f8da0105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:41.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 shutdown_connections 2024-06-13T06:42:41.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:41.345+0000 7f8da7600700 1 -- 172.21.15.7:0/319277684 wait complete. 2024-06-13T06:42:42.024 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:41 smithi007 ceph-mon[32077]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:42:42.089 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-13T06:42:42.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd crush tunables default 2024-06-13T06:42:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:42:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:42 smithi007 ceph-mon[32077]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:44.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 -- 172.21.15.7:0/2067213287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 msgr2=0x7f0ec0108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 --2- 172.21.15.7:0/2067213287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0108310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009910 tx=0x7f0eb4009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 -- 172.21.15.7:0/2067213287 shutdown_connections 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 --2- 172.21.15.7:0/2067213287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0108310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 -- 172.21.15.7:0/2067213287 >> 172.21.15.7:0/2067213287 conn(0x7f0ec00768f0 msgr2=0x7f0ec0076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 -- 172.21.15.7:0/2067213287 shutdown_connections 2024-06-13T06:42:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.483+0000 7f0ec8400700 1 -- 172.21.15.7:0/2067213287 wait complete. 2024-06-13T06:42:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.484+0000 7f0ec8400700 1 Processor -- start 2024-06-13T06:42:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.485+0000 7f0ec8400700 1 -- start start 2024-06-13T06:42:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.485+0000 7f0ec8400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.485+0000 7f0ec8400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010c4c0 con 0x7f0ec0107f30 2024-06-13T06:42:44.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.485+0000 7f0ec5a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:44.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.486+0000 7f0ec5a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48712/0 (socket says 172.21.15.7:48712) 2024-06-13T06:42:44.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.486+0000 7f0ec5a00700 1 -- 172.21.15.7:0/3130666416 learned_addr learned my addr 172.21.15.7:0/3130666416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.486+0000 7f0ec5a00700 1 -- 172.21.15.7:0/3130666416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec0102da0 con 0x7f0ec0107f30 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.486+0000 7f0ec5a00700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006010 tx=0x7f0eb4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.487+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4026070 con 0x7f0ec0107f30 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.487+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb400bc30 con 0x7f0ec0107f30 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.487+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb40039d0 con 0x7f0ec0107f30 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.487+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec0101450 con 0x7f0ec0107f30 2024-06-13T06:42:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.487+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec0101910 con 0x7f0ec0107f30 2024-06-13T06:42:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.488+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0eb4029070 con 0x7f0ec0107f30 2024-06-13T06:42:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.488+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec01035e0 con 0x7f0ec0107f30 2024-06-13T06:42:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.489+0000 7f0ebe000700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 0x7f0ea803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.489+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0eb401e070 con 0x7f0ec0107f30 2024-06-13T06:42:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.489+0000 7f0ec5000700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 0x7f0ea803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.490+0000 7f0ec5000700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 0x7f0ea803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006fd0 tx=0x7f0eac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:44.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.496+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0eb4013070 con 0x7f0ec0107f30 2024-06-13T06:42:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.735+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0ec01023e0 con 0x7f0ec0107f30 2024-06-13T06:42:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.964+0000 7f0ebe000700 1 -- 172.21.15.7:0/3130666416 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f0eb4030090 con 0x7f0ec0107f30 2024-06-13T06:42:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 msgr2=0x7f0ea803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 0x7f0ea803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006fd0 tx=0x7f0eac006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 msgr2=0x7f0ec0100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006010 tx=0x7f0eb4005030 comp rx=0 tx=0).stop 2024-06-13T06:42:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 shutdown_connections 2024-06-13T06:42:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea8039150 0x7f0ea803b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 --2- 172.21.15.7:0/3130666416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0107f30 0x7f0ec0100eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.970+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 >> 172.21.15.7:0/3130666416 conn(0x7f0ec00768f0 msgr2=0x7f0ec00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.971+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 shutdown_connections 2024-06-13T06:42:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:42:44.971+0000 7f0ec8400700 1 -- 172.21.15.7:0/3130666416 wait complete. 2024-06-13T06:42:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:44.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:44.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: Updating smithi189:/etc/ceph/ceph.conf 2024-06-13T06:42:44.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:44.995 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:44 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3130666416' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-13T06:42:45.640 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-06-13T06:42:45.640 INFO:tasks.cephadm:Adding mon.b on smithi175 2024-06-13T06:42:45.640 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-06-13T06:42:45.640 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c' 2024-06-13T06:42:45.869 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3130666416' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:46.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:47.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 -- 172.21.15.189:0/270991940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 msgr2=0x7fc62c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 --2- 172.21.15.189:0/270991940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c101420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009910 tx=0x7fc61c009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 -- 172.21.15.189:0/270991940 shutdown_connections 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 --2- 172.21.15.189:0/270991940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c101420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 -- 172.21.15.189:0/270991940 >> 172.21.15.189:0/270991940 conn(0x7fc62c0768f0 msgr2=0x7fc62c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 -- 172.21.15.189:0/270991940 shutdown_connections 2024-06-13T06:42:47.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 -- 172.21.15.189:0/270991940 wait complete. 2024-06-13T06:42:47.346 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:47 smithi007 ceph-mon[32077]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:47.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.345+0000 7fc631dc8700 1 Processor -- start 2024-06-13T06:42:47.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.346+0000 7fc631dc8700 1 -- start start 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.346+0000 7fc631dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.346+0000 7fc631dc8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10b3c0 con 0x7fc62c101040 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.346+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.347+0000 7fc62b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.189:51820/0 (socket says 172.21.15.189:51820) 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.347+0000 7fc62b7fe700 1 -- 172.21.15.189:0/720500550 learned_addr learned my addr 172.21.15.189:0/720500550 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:42:47.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.347+0000 7fc62b7fe700 1 -- 172.21.15.189:0/720500550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c19db90 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc62b7fe700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005850 tx=0x7fc61c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c026070 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c19de80 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c00bbd0 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61c003970 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.348+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1977d0 con 0x7fc62c101040 2024-06-13T06:42:47.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.349+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c1024d0 con 0x7fc62c101040 2024-06-13T06:42:47.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.350+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc61c029070 con 0x7fc62c101040 2024-06-13T06:42:47.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.350+0000 7fc6297fa700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 0x7fc61803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:47.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.350+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc61c01e070 con 0x7fc62c101040 2024-06-13T06:42:47.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.351+0000 7fc622ffd700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 0x7fc61803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:47.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.352+0000 7fc622ffd700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 0x7fc61803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc614006fd0 tx=0x7fc614006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:47.366 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.367+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc61c030020 con 0x7fc62c101040 2024-06-13T06:42:47.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.571+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7fc62c198dc0 con 0x7fc6180390f0 2024-06-13T06:42:47.585 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.585+0000 7fc6297fa700 1 -- 172.21.15.189:0/720500550 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc62c198dc0 con 0x7fc6180390f0 2024-06-13T06:42:47.585 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-06-13T06:42:47.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 msgr2=0x7fc61803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:47.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 0x7fc61803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc614006fd0 tx=0x7fc614006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:47.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 msgr2=0x7fc62c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:47.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005850 tx=0x7fc61c005030 comp rx=0 tx=0).stop 2024-06-13T06:42:47.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 shutdown_connections 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180390f0 0x7fc61803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 --2- 172.21.15.189:0/720500550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc62c101040 0x7fc62c19d650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 >> 172.21.15.189:0/720500550 conn(0x7fc62c0768f0 msgr2=0x7fc62c0698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 shutdown_connections 2024-06-13T06:42:47.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:47.588+0000 7fc631dc8700 1 -- 172.21.15.189:0/720500550 wait complete. 2024-06-13T06:42:48.174 DEBUG:teuthology.orchestra.run.smithi175:mon.b> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.b.service 2024-06-13T06:42:48.177 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.c.service 2024-06-13T06:42:48.179 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:42:48.179 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:42:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c;count:3 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:48 smithi007 ceph-mon[32077]: Deploying daemon mon.c on smithi189 2024-06-13T06:42:50.196 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:42:50.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:50 smithi007 ceph-mon[32077]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:51.571 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:51 smithi189 systemd[1]: Starting Ceph mon.c for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:42:52.061 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- 172.21.15.189:0/4194844661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 msgr2=0x7efd1c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:52.061 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 --2- 172.21.15.189:0/4194844661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c1014c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd10009910 tx=0x7efd10009c20 comp rx=0 tx=0).stop 2024-06-13T06:42:52.061 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- 172.21.15.189:0/4194844661 shutdown_connections 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 --2- 172.21.15.189:0/4194844661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c1014c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- 172.21.15.189:0/4194844661 >> 172.21.15.189:0/4194844661 conn(0x7efd1c0fc970 msgr2=0x7efd1c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- 172.21.15.189:0/4194844661 shutdown_connections 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- 172.21.15.189:0/4194844661 wait complete. 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 Processor -- start 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.061+0000 7efd225b1700 1 -- start start 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.062+0000 7efd225b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:52.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.062+0000 7efd225b1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10b4b0 con 0x7efd1c1010e0 2024-06-13T06:42:52.063 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.062+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:52.063 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.062+0000 7efd1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.189:51836/0 (socket says 172.21.15.189:51836) 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.062+0000 7efd1bfff700 1 -- 172.21.15.189:0/2837601983 learned_addr learned my addr 172.21.15.189:0/2837601983 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.063+0000 7efd1bfff700 1 -- 172.21.15.189:0/2837601983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c104a70 con 0x7efd1c1010e0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.063+0000 7efd1bfff700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd10006010 tx=0x7efd10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.064+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd10026070 con 0x7efd1c1010e0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.064+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c103120 con 0x7efd1c1010e0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.064+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1035e0 con 0x7efd1c1010e0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.064+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1000bbd0 con 0x7efd1c1010e0 2024-06-13T06:42:52.064 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.064+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd10030060 con 0x7efd1c1010e0 2024-06-13T06:42:52.065 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.065+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08005330 con 0x7efd1c1010e0 2024-06-13T06:42:52.065 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.065+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7efd10029070 con 0x7efd1c1010e0 2024-06-13T06:42:52.065 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.066+0000 7efd197fa700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 0x7efd0403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:52.065 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.066+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efd1001e070 con 0x7efd1c1010e0 2024-06-13T06:42:52.065 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.066+0000 7efd1b7fe700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 0x7efd0403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:52.066 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.067+0000 7efd1b7fe700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 0x7efd0403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:52.080 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.081+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd10013070 con 0x7efd1c1010e0 2024-06-13T06:42:52.369 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.369+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efd080058f0 con 0x7efd1c1010e0 2024-06-13T06:42:52.370 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.371+0000 7efd197fa700 1 -- 172.21.15.189:0/2837601983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7efd1001f310 con 0x7efd1c1010e0 2024-06-13T06:42:52.370 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:42:52.370 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:40:48.478210Z","created":"2024-06-13T06:40:48.478210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-13T06:42:52.372 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 msgr2=0x7efd0403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:52.372 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 0x7efd0403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c006e40 comp rx=0 tx=0).stop 2024-06-13T06:42:52.372 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 msgr2=0x7efd1c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd10006010 tx=0x7efd10005030 comp rx=0 tx=0).stop 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 shutdown_connections 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efd040390f0 0x7efd0403b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 --2- 172.21.15.189:0/2837601983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd1c1010e0 0x7efd1c102b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 >> 172.21.15.189:0/2837601983 conn(0x7efd1c0fc970 msgr2=0x7efd1c069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 shutdown_connections 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:52.373+0000 7efd225b1700 1 -- 172.21.15.189:0/2837601983 wait complete. 2024-06-13T06:42:52.373 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 podman[36566]: 2024-06-13 06:42:52.084809177 +0000 UTC m=+0.197775735 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 podman[36566]: 2024-06-13 06:42:52.192686089 +0000 UTC m=+0.305652636 container create 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 podman[36566]: 2024-06-13 06:42:52.394960996 +0000 UTC m=+0.507927556 container init 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 podman[36566]: 2024-06-13 06:42:52.397991378 +0000 UTC m=+0.510957936 container start 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: set uid:gid to 167:167 (ceph:ceph) 2024-06-13T06:42:52.438 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pidfile_write: ignore empty --pid-file 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: load: jerasure load: lrc 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: RocksDB version: 7.9.2 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Git sha 0 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: DB SUMMARY 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: DB Session ID: BUACVWQHJW7K2UWVOE9D 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: CURRENT file: CURRENT 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: IDENTITY file: IDENTITY 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-13T06:42:52.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.error_if_exists: 0 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.create_if_missing: 0 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.paranoid_checks: 1 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-13T06:42:52.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.env: 0x56251b283c60 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.fs: PosixFileSystem 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.info_log: 0x56251bb03b20 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_file_opening_threads: 16 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.statistics: (nil) 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.use_fsync: 0 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_log_file_size: 0 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.keep_log_file_num: 1000 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.recycle_log_file_num: 0 2024-06-13T06:42:52.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_fallocate: 1 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_mmap_reads: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_mmap_writes: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.use_direct_reads: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.create_missing_column_families: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.db_log_dir: 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.wal_dir: 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-13T06:42:52.442 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.advise_random_on_open: 1 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.db_write_buffer_size: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.write_buffer_manager: 0x56251bad41e0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.rate_limiter: (nil) 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.wal_recovery_mode: 2 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enable_thread_tracking: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enable_pipelined_write: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.unordered_write: 0 2024-06-13T06:42:52.443 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.row_cache: None 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.wal_filter: None 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_ingest_behind: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.two_write_queues: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.manual_wal_flush: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.wal_compression: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.atomic_flush: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-13T06:42:52.444 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.log_readahead_size: 0 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.best_efforts_recovery: 0 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.allow_data_in_errors: 0 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.db_host_id: __hostname__ 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_background_jobs: 2 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_background_compactions: -1 2024-06-13T06:42:52.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_subcompactions: 1 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_total_wal_size: 0 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-13T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_open_files: -1 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bytes_per_sync: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_readahead_size: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_background_flushes: -1 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Compression algorithms supported: 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kZSTD supported: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kXpressCompression supported: 0 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kLZ4HCCompression supported: 1 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kZlibCompression supported: 1 2024-06-13T06:42:52.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kSnappyCompression supported: 1 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kLZ4Compression supported: 1 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: kBZip2Compression supported: 0 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.merge_operator: 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_filter: None 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_filter_factory: None 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.sst_partitioner_factory: None 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-13T06:42:52.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56251bb03c40) 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_type: 0 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_index_type: 0 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_shortening: 1 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: checksum: 4 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: no_block_cache: 0 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache: 0x56251bb05090 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_options: 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: capacity : 536870912 2024-06-13T06:42:52.449 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_shard_bits : 4 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: strict_capacity_limit : 0 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_compressed: (nil) 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: persistent_cache: (nil) 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size: 4096 2024-06-13T06:42:52.450 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size_deviation: 10 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_restart_interval: 16 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_block_restart_interval: 1 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: metadata_block_size: 4096 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: partition_filters: 0 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: use_delta_encoding: 1 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: filter_policy: bloomfilter 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: whole_key_filtering: 1 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: verify_compression: 0 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-06-13T06:42:52.451 INFO:journalctl@ceph.mon.c.smithi189.stdout: format_version: 5 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_index_compression: 1 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_align: 0 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: max_auto_readahead_size: 262144 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: prepopulate_block_cache: 0 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: initial_auto_readahead_size: 8192 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.write_buffer_size: 33554432 2024-06-13T06:42:52.452 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_write_buffer_number: 2 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression: NoCompression 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression: Disabled 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.prefix_extractor: nullptr 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.num_levels: 7 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-13T06:42:52.453 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.level: 32767 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.strategy: 0 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.enabled: false 2024-06-13T06:42:52.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.target_file_size_base: 67108864 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-13T06:42:52.455 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.arena_block_size: 1048576 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.disable_auto_compactions: 0 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-13T06:42:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.table_properties_collectors: 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.inplace_update_support: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.bloom_locality: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.max_successive_merges: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-13T06:42:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.paranoid_file_checks: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.force_consistency_checks: 1 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.report_bg_io_stats: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.ttl: 2592000 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enable_blob_files: false 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.min_blob_size: 0 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_file_size: 268435456 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-13T06:42:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.blob_file_starting_level: 0 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c0e534d9-ac37-43b1-8c17-151dd09a6e93 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260972439040, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260972439576, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718260972, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c0e534d9-ac37-43b1-8c17-151dd09a6e93", "db_session_id": "BUACVWQHJW7K2UWVOE9D", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718260972439682, "job": 1, "event": "recovery_finished"} 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-13T06:42:52.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56251bbf4000 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: DB pointer 0x56251bbde000 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** DB Stats ** 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-13T06:42:52.460 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.461 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache BinnedLRUCache@0x56251bb05090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-13T06:42:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-13T06:42:52.463 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-06-13T06:42:52.463 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: starting mon.c rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:52.463 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(???) e0 preinit fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).mds e1 new map 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-06-13T06:42:52.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Deploying cephadm binary to smithi175 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Added host smithi175 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi175:/etc/ceph/ceph.conf 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Deploying cephadm binary to smithi189 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Added host smithi189 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:42:52.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi189:/etc/ceph/ceph.conf 2024-06-13T06:42:52.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3130666416' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.client.admin.keyring 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3130666416' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c;count:3 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: Deploying daemon mon.c on smithi189 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:52.828 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 ceph-mon[36616]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-13T06:42:52.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 bash[36566]: 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 2024-06-13T06:42:52.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:52 smithi189 systemd[1]: Started Ceph mon.c for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:42:54.087 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:42:54.087 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:42:54.331 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:42:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: mon.a calling monitor election 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: mon.c calling monitor election 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: monmap epoch 2 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: last_changed 2024-06-13T06:42:52.474177+0000 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: min_mon_release 19 (squid) 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: election_strategy: 1 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: fsmap 2024-06-13T06:42:57.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:42:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: mgrmap e13: x(active, since 58s) 2024-06-13T06:42:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T06:42:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:57.839 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: mon.a calling monitor election 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: mon.c calling monitor election 2024-06-13T06:42:57.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: monmap epoch 2 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: last_changed 2024-06-13T06:42:52.474177+0000 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: min_mon_release 19 (squid) 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: election_strategy: 1 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: fsmap 2024-06-13T06:42:57.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:42:57.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: mgrmap e13: x(active, since 58s) 2024-06-13T06:42:57.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T06:42:57.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:57.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 -- 172.21.15.189:0/3316169172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc00035c0 msgr2=0x7f0bc0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/3316169172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc00035c0 0x7f0bc0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4100db0 tx=0x7f0bc400feb0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 -- 172.21.15.189:0/3316169172 shutdown_connections 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/3316169172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc00035c0 0x7f0bc0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 -- 172.21.15.189:0/3316169172 >> 172.21.15.189:0/3316169172 conn(0x7f0bd40fc970 msgr2=0x7f0bd40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 -- 172.21.15.189:0/3316169172 shutdown_connections 2024-06-13T06:42:58.659 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.659+0000 7f0bd8b11700 1 -- 172.21.15.189:0/3316169172 wait complete. 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.660+0000 7f0bd8b11700 1 Processor -- start 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd8b11700 1 -- start start 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd8b11700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd8b11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 0x7f0bd4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd8b11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4108f10 con 0x7f0bd419dcd0 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd8b11700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4108d90 con 0x7f0bc00035c0 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54998/0 (socket says 172.21.15.189:54998) 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 -- 172.21.15.189:0/276305182 learned_addr learned my addr 172.21.15.189:0/276305182 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:42:58.661 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 -- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 msgr2=0x7f0bd419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-13T06:42:58.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 -- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 msgr2=0x7f0bd419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-13T06:42:58.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-13T06:42:58.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bd259c700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:42:58.662 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.661+0000 7f0bc9d9b700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 0x7f0bd4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.662+0000 7f0bc9d9b700 1 -- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 msgr2=0x7f0bd419d790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.662+0000 7f0bc9d9b700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.662+0000 7f0bc9d9b700 1 -- 172.21.15.189:0/276305182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419e310 con 0x7f0bd419dcd0 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bc9d9b700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 0x7f0bd4197950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b4d0 tx=0x7f0bbc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0bbc011630 con 0x7f0bd419dcd0 2024-06-13T06:42:58.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd4197e90 con 0x7f0bd419dcd0 2024-06-13T06:42:58.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd4198420 con 0x7f0bd419dcd0 2024-06-13T06:42:58.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc011cb0 con 0x7f0bd419dcd0 2024-06-13T06:42:58.664 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.663+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0bbc01ae10 con 0x7f0bd419dcd0 2024-06-13T06:42:58.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.664+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0002160 con 0x7f0bd419dcd0 2024-06-13T06:42:58.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.665+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0bbc018020 con 0x7f0bd419dcd0 2024-06-13T06:42:58.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.665+0000 7f0bcbfff700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 0x7f0bb403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:42:58.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.665+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0bbc04bd10 con 0x7f0bd419dcd0 2024-06-13T06:42:58.665 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.665+0000 7f0bd259c700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 0x7f0bb403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:42:58.666 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.666+0000 7f0bd259c700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 0x7f0bb403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400fa60 tx=0x7f0bc4005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:42:58.678 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.679+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0bbc01f020 con 0x7f0bd419dcd0 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: Deploying daemon mon.b on smithi175 2024-06-13T06:42:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:58.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:42:58.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:42:58.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:42:58.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:42:58.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: Deploying daemon mon.b on smithi175 2024-06-13T06:42:58.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:42:58.954 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.955+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0bc0002c50 con 0x7f0bd419dcd0 2024-06-13T06:42:58.955 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.956+0000 7f0bcbfff700 1 -- 172.21.15.189:0/276305182 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f0bbc017760 con 0x7f0bd419dcd0 2024-06-13T06:42:58.956 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:42:58.956 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:42:52.474177Z","created":"2024-06-13T06:40:48.478210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 msgr2=0x7f0bb403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 0x7f0bb403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400fa60 tx=0x7f0bc4005c20 comp rx=0 tx=0).stop 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 msgr2=0x7f0bd4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 0x7f0bd4197950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b4d0 tx=0x7f0bbc00b890 comp rx=0 tx=0).stop 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 shutdown_connections 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bb4038e90 0x7f0bb403b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bc00035c0 0x7f0bd419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 --2- 172.21.15.189:0/276305182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd419dcd0 0x7f0bd4197950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 >> 172.21.15.189:0/276305182 conn(0x7f0bd40fc970 msgr2=0x7f0bd40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:42:58.958 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 shutdown_connections 2024-06-13T06:42:58.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:42:58.958+0000 7f0bd8b11700 1 -- 172.21.15.189:0/276305182 wait complete. 2024-06-13T06:42:58.959 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-06-13T06:42:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:42:59 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/276305182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:42:59.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:42:59 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/276305182' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-13T06:43:00.513 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-13T06:43:00.513 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph mon dump -f json 2024-06-13T06:43:00.756 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:43:00.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:00 smithi189 ceph-mon[36616]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:00.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:00 smithi007 ceph-mon[32077]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:01.667 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:01 smithi175 ceph-mon[36886]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-13T06:43:01.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:01 smithi175 bash[36868]: c3844d35f6f48a36b3dae4fc65b61fc758b53a08f541b2dd5f3aa0d22c9fdbaf 2024-06-13T06:43:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:01 smithi175 systemd[1]: Started Ceph mon.b for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:43:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: mon.c calling monitor election 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: mon.a calling monitor election 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: monmap epoch 3 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: min_mon_release 19 (squid) 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: election_strategy: 1 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: fsmap 2024-06-13T06:43:07.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: mgrmap e13: x(active, since 67s) 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: mon.c calling monitor election 2024-06-13T06:43:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:43:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: mon.a calling monitor election 2024-06-13T06:43:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: monmap epoch 3 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: min_mon_release 19 (squid) 2024-06-13T06:43:07.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: election_strategy: 1 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: fsmap 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: mgrmap e13: x(active, since 67s) 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:07.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:07 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:07 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:08.157 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 -- 172.21.15.189:0/3521759203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 msgr2=0x7fa6b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:08.157 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/3521759203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6b0005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4101ee0 tx=0x7fa6c000ff50 comp rx=0 tx=0).stop 2024-06-13T06:43:08.157 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 -- 172.21.15.189:0/3521759203 shutdown_connections 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/3521759203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6b0005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 -- 172.21.15.189:0/3521759203 >> 172.21.15.189:0/3521759203 conn(0x7fa6c40fdaa0 msgr2=0x7fa6c40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 -- 172.21.15.189:0/3521759203 shutdown_connections 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.154+0000 7fa6cbd24700 1 -- 172.21.15.189:0/3521759203 wait complete. 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 Processor -- start 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 -- start start 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6c4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4109fd0 con 0x7fa6b00035c0 2024-06-13T06:43:08.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4109e50 con 0x7fa6c41a3e30 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.155+0000 7fa6cbd24700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c410a150 con 0x7fa6c4074cd0 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6ca2c1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:39622/0 (socket says 172.21.15.189:39622) 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 -- 172.21.15.189:0/20019845 learned_addr learned my addr 172.21.15.189:0/20019845 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6ca2c1700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 msgr2=0x7fa6c41a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6ca2c1700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 msgr2=0x7fa6c41a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6ca2c1700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6ca2c1700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c9ac0700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6c4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 msgr2=0x7fa6c41a6220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:43:08.159 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 msgr2=0x7fa6c4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6c4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 -- 172.21.15.189:0/20019845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c41a6950 con 0x7fa6c4074cd0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6c92bf700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b780 tx=0x7fa6b400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004230 con 0x7fa6c4074cd0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b40043d0 con 0x7fa6c4074cd0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4011c60 con 0x7fa6c4074cd0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c41a6c40 con 0x7fa6c4074cd0 2024-06-13T06:43:08.160 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.156+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c41a7130 con 0x7fa6c4074cd0 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.157+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa6b401a400 con 0x7fa6c4074cd0 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.157+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c4063af0 con 0x7fa6c4074cd0 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.157+0000 7fa6baffd700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 0x7fa6a803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.157+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6b404c020 con 0x7fa6c4074cd0 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.158+0000 7fa6c9ac0700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 0x7fa6a803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:08.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.159+0000 7fa6c9ac0700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 0x7fa6a803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005d10 tx=0x7fa6c0005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:08.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.163+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa6b4003d60 con 0x7fa6c4074cd0 2024-06-13T06:43:08.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:08.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:08.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.450+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa6c402df00 con 0x7fa6c4074cd0 2024-06-13T06:43:08.451 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.452+0000 7fa6baffd700 1 -- 172.21.15.189:0/20019845 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fa6b4017d50 con 0x7fa6c4074cd0 2024-06-13T06:43:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-06-13T06:43:08.452 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","modified":"2024-06-13T06:43:01.627380Z","created":"2024-06-13T06:40:48.478210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-13T06:43:08.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 msgr2=0x7fa6a803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:08.454 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 0x7fa6a803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005d10 tx=0x7fa6c0005bc0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 msgr2=0x7fa6c4075150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:08.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b780 tx=0x7fa6b400bb40 comp rx=0 tx=0).stop 2024-06-13T06:43:08.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 shutdown_connections 2024-06-13T06:43:08.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6a8039580 0x7fa6a803ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6c41a3e30 0x7fa6c41a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b00035c0 0x7fa6c4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 --2- 172.21.15.189:0/20019845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6c4074cd0 0x7fa6c4075150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 >> 172.21.15.189:0/20019845 conn(0x7fa6c40fdaa0 msgr2=0x7fa6c40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 shutdown_connections 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:08.455+0000 7fa6cbd24700 1 -- 172.21.15.189:0/20019845 wait complete. 2024-06-13T06:43:08.456 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-06-13T06:43:09.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:09.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: mon.c calling monitor election 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: mon.a calling monitor election 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: monmap epoch 3 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: min_mon_release 19 (squid) 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: election_strategy: 1 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: fsmap 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: mgrmap e13: x(active, since 67s) 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: overall HEALTH_OK 2024-06-13T06:43:09.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:09.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:09.162 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-13T06:43:09.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph config generate-minimal-conf 2024-06-13T06:43:09.581 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:43:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mon.b calling monitor election 2024-06-13T06:43:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mon.b calling monitor election 2024-06-13T06:43:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mon.c calling monitor election 2024-06-13T06:43:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mon.a calling monitor election 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: monmap epoch 3 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: min_mon_release 19 (squid) 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: election_strategy: 1 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: fsmap 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: mgrmap e13: x(active, since 69s) 2024-06-13T06:43:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:09 smithi007 ceph-mon[32077]: Updating smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mon.b calling monitor election 2024-06-13T06:43:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mon.b calling monitor election 2024-06-13T06:43:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mon.c calling monitor election 2024-06-13T06:43:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mon.a calling monitor election 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: monmap epoch 3 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: min_mon_release 19 (squid) 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: election_strategy: 1 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: fsmap 2024-06-13T06:43:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: mgrmap e13: x(active, since 69s) 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.838 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:09 smithi189 ceph-mon[36616]: Updating smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mon.b calling monitor election 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mon.b calling monitor election 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mon.c calling monitor election 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mon.a calling monitor election 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: monmap epoch 3 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: last_changed 2024-06-13T06:43:01.627380+0000 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: created 2024-06-13T06:40:48.478210+0000 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: min_mon_release 19 (squid) 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: election_strategy: 1 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: 2: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: fsmap 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: osdmap e4: 0 total, 0 up, 0 in 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: mgrmap e13: x(active, since 69s) 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: overall HEALTH_OK 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: Updating smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: Updating smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.869 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.869 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:09.869 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:09 smithi175 ceph-mon[36886]: Updating smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/config/ceph.conf 2024-06-13T06:43:10.792 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:43:10.628+0000 7f7fb2200700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-06-13T06:43:10.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:10 smithi007 ceph-mon[32077]: Reconfiguring daemon mon.a on smithi007 2024-06-13T06:43:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:10.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:10 smithi189 ceph-mon[36616]: Reconfiguring daemon mon.a on smithi007 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: Reconfiguring mon.a (unknown last config time)... 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:10.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:10 smithi175 ceph-mon[36886]: Reconfiguring daemon mon.a on smithi007 2024-06-13T06:43:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.039+0000 7fbecbc00700 1 -- 172.21.15.7:0/3678034662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 msgr2=0x7fbec41024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.039+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3678034662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec41024e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009910 tx=0x7fbeac009c20 comp rx=0 tx=0).stop 2024-06-13T06:43:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.040+0000 7fbecbc00700 1 -- 172.21.15.7:0/3678034662 shutdown_connections 2024-06-13T06:43:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.040+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3678034662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec41024e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.040+0000 7fbecbc00700 1 -- 172.21.15.7:0/3678034662 >> 172.21.15.7:0/3678034662 conn(0x7fbec40fd9d0 msgr2=0x7fbec40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.040+0000 7fbecbc00700 1 -- 172.21.15.7:0/3678034662 shutdown_connections 2024-06-13T06:43:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.040+0000 7fbecbc00700 1 -- 172.21.15.7:0/3678034662 wait complete. 2024-06-13T06:43:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.041+0000 7fbecbc00700 1 Processor -- start 2024-06-13T06:43:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.041+0000 7fbecbc00700 1 -- start start 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec419de10 0x7fbec4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 0x7fbec4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410c490 con 0x7fbec4102100 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410c310 con 0x7fbec419de10 2024-06-13T06:43:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbecbc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410c610 con 0x7fbec4197fd0 2024-06-13T06:43:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbec9200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbec9200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59366/0 (socket says 172.21.15.7:59366) 2024-06-13T06:43:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbec9200700 1 -- 172.21.15.7:0/3201748450 learned_addr learned my addr 172.21.15.7:0/3201748450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:43:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbec9c00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 0x7fbec4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.042+0000 7fbebbe00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec419de10 0x7fbec4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 -- 172.21.15.7:0/3201748450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 msgr2=0x7fbec4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 0x7fbec4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 -- 172.21.15.7:0/3201748450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec419de10 msgr2=0x7fbec4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec419de10 0x7fbec4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 -- 172.21.15.7:0/3201748450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec41a47a0 con 0x7fbec4102100 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9c00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 0x7fbec4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec9200700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00bdc0 tx=0x7fbeac00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.043+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac026070 con 0x7fbec4102100 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.044+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac005da0 con 0x7fbec4102100 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.044+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0038e0 con 0x7fbec4102100 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.044+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec41a49d0 con 0x7fbec4102100 2024-06-13T06:43:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.044+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec41a4ec0 con 0x7fbec4102100 2024-06-13T06:43:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.045+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec4063af0 con 0x7fbec4102100 2024-06-13T06:43:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.045+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fbeac029070 con 0x7fbec4102100 2024-06-13T06:43:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.046+0000 7fbec2000700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 0x7fbea403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.046+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbeac04cdd0 con 0x7fbec4102100 2024-06-13T06:43:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.046+0000 7fbebbe00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 0x7fbea403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.047+0000 7fbebbe00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 0x7fbea403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009ad0 tx=0x7fbeb4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.052+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbeac013070 con 0x7fbec4102100 2024-06-13T06:43:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.283+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbec41a5150 con 0x7fbec4102100 2024-06-13T06:43:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.284+0000 7fbec2000700 1 -- 172.21.15.7:0/3201748450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fbeac030020 con 0x7fbec4102100 2024-06-13T06:43:12.284 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:12.284 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-06-13T06:43:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:43:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-06-13T06:43:12.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 msgr2=0x7fbea403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 0x7fbea403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009ad0 tx=0x7fbeb4009210 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 msgr2=0x7fbec419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00bdc0 tx=0x7fbeac00bdf0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 shutdown_connections 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbec4197fd0 0x7fbec4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbec4102100 0x7fbec419d8d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbea4039580 0x7fbea403ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 --2- 172.21.15.7:0/3201748450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbec419de10 0x7fbec4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.287+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 >> 172.21.15.7:0/3201748450 conn(0x7fbec40fd9d0 msgr2=0x7fbec4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.288+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 shutdown_connections 2024-06-13T06:43:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:12.288+0000 7fbecbc00700 1 -- 172.21.15.7:0/3201748450 wait complete. 2024-06-13T06:43:12.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:12 smithi189 ceph-mon[36616]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:12.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:12 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3201748450' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:12.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:12.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:12 smithi175 ceph-mon[36886]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:12 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3201748450' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:12.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:12 smithi007 ceph-mon[32077]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:12 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3201748450' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:13.076 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-13T06:43:13.076 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:43:13.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:43:13.154 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:43:13.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:43:13.273 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:43:13.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:43:13.299 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:43:13.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:43:13.363 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:43:13.363 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-06-13T06:43:13.390 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:43:13.391 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-13T06:43:13.455 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-06-13T06:43:13.455 INFO:tasks.cephadm:Adding mgr.y on smithi175 2024-06-13T06:43:13.455 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-06-13T06:43:13.455 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch apply mgr '3;smithi007=x;smithi175=y;smithi189=z' 2024-06-13T06:43:13.724 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:43:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:13 smithi189 ceph-mon[36616]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:43:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:13 smithi189 ceph-mon[36616]: Reconfiguring daemon mon.b on smithi175 2024-06-13T06:43:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:13 smithi175 ceph-mon[36886]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:43:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:13 smithi175 ceph-mon[36886]: Reconfiguring daemon mon.b on smithi175 2024-06-13T06:43:13.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:13 smithi007 ceph-mon[32077]: Reconfiguring mon.b (monmap changed)... 2024-06-13T06:43:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:13 smithi007 ceph-mon[32077]: Reconfiguring daemon mon.b on smithi175 2024-06-13T06:43:15.271 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 -- 172.21.15.189:0/4071196110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec100f30 msgr2=0x7f80ec101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:15.271 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 --2- 172.21.15.189:0/4071196110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec100f30 0x7f80ec101350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80e400b0a0 tx=0x7f80e400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.271 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 -- 172.21.15.189:0/4071196110 shutdown_connections 2024-06-13T06:43:15.271 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 --2- 172.21.15.189:0/4071196110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec100f30 0x7f80ec101350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.271 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 -- 172.21.15.189:0/4071196110 >> 172.21.15.189:0/4071196110 conn(0x7f80ec0fc4d0 msgr2=0x7f80ec0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 -- 172.21.15.189:0/4071196110 shutdown_connections 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.270+0000 7f80f3045700 1 -- 172.21.15.189:0/4071196110 wait complete. 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 Processor -- start 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 -- start start 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec199fe0 0x7f80ec19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec19a460 0x7f80ec1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec103dc0 con 0x7f80ec19a460 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec103c40 con 0x7f80ec199fe0 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f3045700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec103f40 con 0x7f80ec100f30 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:57628/0 (socket says 172.21.15.189:57628) 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 -- 172.21.15.189:0/175312198 learned_addr learned my addr 172.21.15.189:0/175312198 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:43:15.272 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80ebfff700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec199fe0 0x7f80ec19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 -- 172.21.15.189:0/175312198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec199fe0 msgr2=0x7f80ec19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec199fe0 0x7f80ec19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 -- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec19a460 msgr2=0x7f80ec1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec19a460 0x7f80ec1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 -- 172.21.15.189:0/175312198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec1a1eb0 con 0x7f80ec100f30 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.271+0000 7f80f0de1700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80e4007c10 tx=0x7f80e4009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.272+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4022070 con 0x7f80ec100f30 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.272+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e40057a0 con 0x7f80ec100f30 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.272+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec1a2140 con 0x7f80ec100f30 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.272+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4017070 con 0x7f80ec100f30 2024-06-13T06:43:15.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.272+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec1a2750 con 0x7f80ec100f30 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.273+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f80e401b070 con 0x7f80ec100f30 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.273+0000 7f80e9ffb700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 0x7f80d803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.273+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f80e400f020 con 0x7f80ec100f30 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.273+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec063af0 con 0x7f80ec100f30 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.273+0000 7f80ebfff700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 0x7f80d803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:15.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.274+0000 7f80ebfff700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 0x7f80d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009850 tx=0x7f80dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:15.277 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.278+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80e400e1d0 con 0x7f80ec100f30 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:15 smithi189 ceph-mon[36616]: Reconfiguring daemon mon.c on smithi189 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:15 smithi175 ceph-mon[36886]: Reconfiguring daemon mon.c on smithi189 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: Reconfiguring mon.c (monmap changed)... 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:15 smithi007 ceph-mon[32077]: Reconfiguring daemon mon.c on smithi189 2024-06-13T06:43:15.490 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.491+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi175=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f80ec106af0 con 0x7f80d8039580 2024-06-13T06:43:15.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.504+0000 7f80e9ffb700 1 -- 172.21.15.189:0/175312198 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f80ec106af0 con 0x7f80d8039580 2024-06-13T06:43:15.504 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 msgr2=0x7f80d803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 0x7f80d803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009850 tx=0x7f80dc008040 comp rx=0 tx=0).stop 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 msgr2=0x7f80ec199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80e4007c10 tx=0x7f80e4009540 comp rx=0 tx=0).stop 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 shutdown_connections 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80ec19a460 0x7f80ec1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80ec100f30 0x7f80ec199aa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d8039580 0x7f80d803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.506 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 --2- 172.21.15.189:0/175312198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80ec199fe0 0x7f80ec19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:15.507 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.506+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 >> 172.21.15.189:0/175312198 conn(0x7f80ec0fc4d0 msgr2=0x7f80ec104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:15.507 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.507+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 shutdown_connections 2024-06-13T06:43:15.507 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:43:15.507+0000 7f80f3045700 1 -- 172.21.15.189:0/175312198 wait complete. 2024-06-13T06:43:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:16 smithi189 ceph-mon[36616]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:16 smithi189 ceph-mon[36616]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi175=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:16 smithi189 ceph-mon[36616]: Saving service mgr spec with placement smithi007=x;smithi175=y;smithi189=z;count:3 2024-06-13T06:43:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:16 smithi175 ceph-mon[36886]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:16 smithi175 ceph-mon[36886]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi175=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:16 smithi175 ceph-mon[36886]: Saving service mgr spec with placement smithi007=x;smithi175=y;smithi189=z;count:3 2024-06-13T06:43:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:16.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:16 smithi007 ceph-mon[32077]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:16.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:16 smithi007 ceph-mon[32077]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi175=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:16.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:16 smithi007 ceph-mon[32077]: Saving service mgr spec with placement smithi007=x;smithi175=y;smithi189=z;count:3 2024-06-13T06:43:16.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:16.958 DEBUG:teuthology.orchestra.run.smithi175:mgr.y> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.y.service 2024-06-13T06:43:16.960 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.z.service 2024-06-13T06:43:16.963 INFO:tasks.cephadm:Deploying OSDs... 2024-06-13T06:43:16.963 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:43:16.963 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:43:17.025 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:43:17.025 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-13 06:42:11.800585695 +0000 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-13 06:36:47.576315807 +0000 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-13 06:36:47.576315807 +0000 2024-06-13T06:43:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-13 06:36:47.576315807 +0000 2024-06-13T06:43:17.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:43:17.159 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-13T06:43:17.159 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-13T06:43:17.159 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000239491 s, 2.1 MB/s 2024-06-13T06:43:17.162 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:43:17.221 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:17.274 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-13 06:42:11.924583735 +0000 2024-06-13T06:43:17.275 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-13 06:36:48.151306729 +0000 2024-06-13T06:43:17.275 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-13 06:36:48.151306729 +0000 2024-06-13T06:43:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-13 06:36:48.151306729 +0000 2024-06-13T06:43:17.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:43:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-13T06:43:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-13T06:43:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000292129 s, 1.8 MB/s 2024-06-13T06:43:17.364 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:43:17.427 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-06-13T06:43:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-13 06:42:12.022582186 +0000 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-13 06:36:48.758297146 +0000 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-13 06:36:48.758297146 +0000 2024-06-13T06:43:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-13 06:36:48.758297146 +0000 2024-06-13T06:43:17.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:43:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-13T06:43:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-13T06:43:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000267322 s, 1.9 MB/s 2024-06-13T06:43:17.606 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:43:17.666 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-06-13 06:42:12.111580779 +0000 2024-06-13T06:43:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-06-13 06:36:49.380287327 +0000 2024-06-13T06:43:17.714 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-06-13 06:36:49.380287327 +0000 2024-06-13T06:43:17.714 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-06-13 06:36:49.380287327 +0000 2024-06-13T06:43:17.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:43:17.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:17 smithi007 ceph-mon[32077]: Deploying daemon mgr.z on smithi189 2024-06-13T06:43:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-06-13T06:43:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-06-13T06:43:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00025247 s, 2.0 MB/s 2024-06-13T06:43:17.799 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:17.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:17 smithi189 ceph-mon[36616]: Deploying daemon mgr.z on smithi189 2024-06-13T06:43:17.860 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-06-13T06:43:17.861 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:17 smithi175 ceph-mon[36886]: Deploying daemon mgr.z on smithi189 2024-06-13T06:43:17.879 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:43:17.879 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-06-13T06:43:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:43:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-13 06:43:01.075636369 +0000 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-13 06:35:15.548989298 +0000 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-13 06:35:15.548989298 +0000 2024-06-13T06:43:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-13 06:35:15.548989298 +0000 2024-06-13T06:43:17.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:43:18.003 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-13T06:43:18.003 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-13T06:43:18.003 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.00020055 s, 2.6 MB/s 2024-06-13T06:43:18.004 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:43:18.062 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-13 06:43:01.075636369 +0000 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-13 06:35:15.870984367 +0000 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-13 06:35:15.870984367 +0000 2024-06-13T06:43:18.119 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-13 06:35:15.870984367 +0000 2024-06-13T06:43:18.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:43:18.182 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-13T06:43:18.182 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-13T06:43:18.183 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.00020521 s, 2.5 MB/s 2024-06-13T06:43:18.184 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:43:18.240 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-13 06:43:01.076636354 +0000 2024-06-13T06:43:18.295 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-13 06:35:16.194979404 +0000 2024-06-13T06:43:18.296 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-13 06:35:16.194979404 +0000 2024-06-13T06:43:18.296 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-13 06:35:16.194979404 +0000 2024-06-13T06:43:18.296 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:43:18.358 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-13T06:43:18.358 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-13T06:43:18.358 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000215446 s, 2.4 MB/s 2024-06-13T06:43:18.360 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:43:18.415 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-06-13T06:43:18.470 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-06-13 06:43:01.076636354 +0000 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-06-13 06:35:16.518974442 +0000 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-06-13 06:35:16.518974442 +0000 2024-06-13T06:43:18.471 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-06-13 06:35:16.518974442 +0000 2024-06-13T06:43:18.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:43:18.533 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-06-13T06:43:18.533 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-06-13T06:43:18.533 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000183802 s, 2.8 MB/s 2024-06-13T06:43:18.534 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:43:18.590 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-06-13T06:43:18.590 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-06-13T06:43:18.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-13T06:43:18.606 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-06-13T06:43:18.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:18 smithi175 ceph-mon[36886]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:18.661 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-06-13 06:43:18.540385814 +0000 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-06-13 06:35:32.999560357 +0000 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-06-13 06:35:32.999560357 +0000 2024-06-13T06:43:18.662 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-06-13 06:35:32.999560357 +0000 2024-06-13T06:43:18.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-13T06:43:18.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:18 smithi189 ceph-mon[36616]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:18.726 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:18 smithi189 systemd[1]: Starting Ceph mgr.z for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:43:18.729 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-06-13T06:43:18.729 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-06-13T06:43:18.729 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000248187 s, 2.1 MB/s 2024-06-13T06:43:18.730 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-13T06:43:18.786 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-06-13 06:43:18.541385795 +0000 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-06-13 06:35:33.328553935 +0000 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-06-13 06:35:33.328553935 +0000 2024-06-13T06:43:18.841 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-06-13 06:35:33.328553935 +0000 2024-06-13T06:43:18.842 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-13T06:43:18.904 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-06-13T06:43:18.904 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-06-13T06:43:18.904 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000319385 s, 1.6 MB/s 2024-06-13T06:43:18.905 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-13T06:43:18.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:18 smithi007 ceph-mon[32077]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:18.960 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-06-13 06:43:18.541385795 +0000 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-06-13 06:35:33.670547259 +0000 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-06-13 06:35:33.670547259 +0000 2024-06-13T06:43:19.016 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-06-13 06:35:33.670547259 +0000 2024-06-13T06:43:19.016 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-13T06:43:19.076 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:18 smithi189 podman[38250]: 2024-06-13 06:43:18.729962943 +0000 UTC m=+0.018270904 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:43:19.076 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:18 smithi189 podman[38250]: 2024-06-13 06:43:18.86468022 +0000 UTC m=+0.152988180 container create 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-13T06:43:19.079 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-06-13T06:43:19.079 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-06-13T06:43:19.079 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000200822 s, 2.5 MB/s 2024-06-13T06:43:19.081 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-13T06:43:19.152 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-06-13 06:43:18.541385795 +0000 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-06-13 06:35:34.012540584 +0000 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-06-13 06:35:34.012540584 +0000 2024-06-13T06:43:19.207 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-06-13 06:35:34.012540584 +0000 2024-06-13T06:43:19.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-13T06:43:19.270 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-06-13T06:43:19.270 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-06-13T06:43:19.270 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000199667 s, 2.6 MB/s 2024-06-13T06:43:19.271 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-13T06:43:19.327 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-06-13T06:43:19.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:43:19.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 podman[38250]: 2024-06-13 06:43:19.18404348 +0000 UTC m=+0.472351440 container init 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-13T06:43:19.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 podman[38250]: 2024-06-13 06:43:19.186722657 +0000 UTC m=+0.475030617 container start 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-13T06:43:19.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 ceph-mgr[38412]: -- 172.21.15.189:0/2314549312 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55791eb2c5a0 con 0x55791eb00800 2024-06-13T06:43:19.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 bash[38250]: 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 2024-06-13T06:43:19.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 systemd[1]: Started Ceph mgr.z for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:43:19.634 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:19.350+0000 7f481affe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:43:19.635 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:19.535+0000 7f481affe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:43:19.635 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:19.635+0000 7f481affe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:43:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:43:20.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:19.940+0000 7f481affe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:43:20.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:20.024+0000 7f481affe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:20.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:20.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:20.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-13T06:43:20.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:20.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:21.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-mon[36616]: Deploying daemon mgr.y on smithi175 2024-06-13T06:43:21.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-mon[36616]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:21.545 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:21.209+0000 7f481affe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:43:21.545 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:21.375+0000 7f481affe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:43:21.546 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:21.545+0000 7f481affe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:43:21.594 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:21 smithi007 ceph-mon[32077]: Deploying daemon mgr.y on smithi175 2024-06-13T06:43:21.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:21 smithi007 ceph-mon[32077]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:21 smithi175 ceph-mon[36886]: Deploying daemon mgr.y on smithi175 2024-06-13T06:43:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:21 smithi175 ceph-mon[36886]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:21.617 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 systemd[1]: Starting Ceph mgr.y for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 podman[38252]: 2024-06-13 06:43:21.552798555 +0000 UTC m=+0.017036361 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 podman[38252]: 2024-06-13 06:43:21.667732008 +0000 UTC m=+0.131969800 container create fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 podman[38252]: 2024-06-13 06:43:21.85285112 +0000 UTC m=+0.317088913 container init fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 podman[38252]: 2024-06-13 06:43:21.855958507 +0000 UTC m=+0.320196301 container start fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 ceph-mgr[38271]: -- 172.21.15.175:0/3008923369 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56205bcf25a0 con 0x56205bcc7000 2024-06-13T06:43:21.927 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 bash[38252]: fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 2024-06-13T06:43:22.086 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:21.835+0000 7f481affe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:43:22.086 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:21.928+0000 7f481affe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:43:22.194 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:21 smithi175 systemd[1]: Started Ceph mgr.y for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:43:22.194 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:22.008+0000 7f523f3bb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-13T06:43:22.194 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:22.194+0000 7f523f3bb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:43:22.416 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:43:22.416 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:43:22.416 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-13T06:43:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-13T06:43:22.417 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.056377 s, 186 MB/s 2024-06-13T06:43:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-13T06:43:22.586 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:22.133+0000 7f481affe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:43:22.586 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:22.327+0000 7f481affe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:43:22.617 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:22.299+0000 7f523f3bb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-13T06:43:23.002 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:22.624+0000 7f523f3bb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-13T06:43:23.002 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:22.712+0000 7f523f3bb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-13T06:43:23.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.293 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:23.336 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:22.972+0000 7f481affe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:43:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:43:23.716 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:23.408+0000 7f481affe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:43:23.716 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:23.508+0000 7f481affe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:43:23.716 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:23.717+0000 7f481affe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:43:24.331 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:23.994+0000 7f523f3bb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-13T06:43:24.331 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:24.161+0000 7f523f3bb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-13T06:43:24.331 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:24.331+0000 7f523f3bb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-13T06:43:24.564 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.238+0000 7f481affe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:43:24.565 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.327+0000 7f481affe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:43:24.764 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:24.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:24.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: Reconfiguring daemon mgr.x on smithi007 2024-06-13T06:43:24.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:24 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.836 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.565+0000 7f481affe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:43:24.922 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.697+0000 7f481affe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:43:24.922 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.789+0000 7f481affe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:43:24.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:24.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: Reconfiguring daemon mgr.x on smithi007 2024-06-13T06:43:24.923 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:24.616+0000 7f523f3bb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:24.709+0000 7f523f3bb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: Reconfiguring mgr.x (unknown last config time)... 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: Reconfiguring daemon mgr.x on smithi007 2024-06-13T06:43:24.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:25.200 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:24.903+0000 7f481affe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:43:25.200 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:25.200+0000 7f481affe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:43:25.367 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:24.914+0000 7f523f3bb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-13T06:43:25.367 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:25.102+0000 7f523f3bb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-13T06:43:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.934+0000 7f3016200700 1 -- 172.21.15.7:0/674856007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101081e0 msgr2=0x7f301010a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.934+0000 7f3016200700 1 --2- 172.21.15.7:0/674856007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101081e0 0x7f301010a650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f300800b0a0 tx=0x7f300800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 -- 172.21.15.7:0/674856007 shutdown_connections 2024-06-13T06:43:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 --2- 172.21.15.7:0/674856007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101081e0 0x7f301010a650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 --2- 172.21.15.7:0/674856007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f3010105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 --2- 172.21.15.7:0/674856007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101058b0 0x7f3010107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 -- 172.21.15.7:0/674856007 >> 172.21.15.7:0/674856007 conn(0x7f30100fc990 msgr2=0x7f30100fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 -- 172.21.15.7:0/674856007 shutdown_connections 2024-06-13T06:43:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.935+0000 7f3016200700 1 -- 172.21.15.7:0/674856007 wait complete. 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.936+0000 7f3016200700 1 Processor -- start 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.936+0000 7f3016200700 1 -- start start 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101081e0 0x7f30101a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f301010e150 con 0x7f30101081e0 2024-06-13T06:43:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f301010dfd0 con 0x7f30101058b0 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f3016200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f301010e2d0 con 0x7f3010102f80 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101081e0 0x7f30101a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40084/0 (socket says 172.21.15.7:40084) 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55236/0 (socket says 172.21.15.7:55236) 2024-06-13T06:43:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 -- 172.21.15.7:0/2251497829 learned_addr learned my addr 172.21.15.7:0/2251497829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 -- 172.21.15.7:0/2251497829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 msgr2=0x7f301019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 -- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101081e0 msgr2=0x7f30101a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101081e0 0x7f30101a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.937+0000 7f300f400700 1 -- 172.21.15.7:0/2251497829 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30101a25d0 con 0x7f3010102f80 2024-06-13T06:43:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f300ea00700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f300f400700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f300000eb60 tx=0x7f300000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300000cbd0 con 0x7f3010102f80 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000004500 con 0x7f3010102f80 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010430 con 0x7f3010102f80 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30101a28c0 con 0x7f3010102f80 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.938+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30101a2e50 con 0x7f3010102f80 2024-06-13T06:43:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.939+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3000004020 con 0x7f3010102f80 2024-06-13T06:43:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.939+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3010069180 con 0x7f3010102f80 2024-06-13T06:43:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.940+0000 7f3007e00700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 0x7f2ff403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.940+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f300004c5e0 con 0x7f3010102f80 2024-06-13T06:43:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.940+0000 7f300ea00700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 0x7f2ff403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.941+0000 7f300ea00700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 0x7f2ff403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc011490 tx=0x7f2ffc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:25.945+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f300001e2e0 con 0x7f3010102f80 2024-06-13T06:43:26.086 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:25.791+0000 7f481affe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:43:26.086 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z[38387]: 2024-06-13T06:43:26.005+0000 7f481affe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:43:26.114 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:25.700+0000 7f523f3bb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-13T06:43:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:26.190+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f30100628d0 con 0x7f2ff4039330 2024-06-13T06:43:26.367 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:26.114+0000 7f523f3bb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-13T06:43:26.367 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:26.210+0000 7f523f3bb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-13T06:43:26.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:26.672+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f3000003680 con 0x7f3010102f80 2024-06-13T06:43:26.867 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:26.408+0000 7f523f3bb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-13T06:43:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:43:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: Standby manager daemon z started 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:26.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: Standby manager daemon z started 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: Standby manager daemon z started 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.189:0/451347853' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:27.266 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:26.935+0000 7f523f3bb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-13T06:43:27.266 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.026+0000 7f523f3bb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-13T06:43:27.606 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.266+0000 7f523f3bb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-13T06:43:27.606 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.399+0000 7f523f3bb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-13T06:43:27.606 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.493+0000 7f523f3bb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-13T06:43:27.606 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.606+0000 7f523f3bb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-13T06:43:27.862 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:27 smithi007 ceph-mon[32077]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-13T06:43:27.862 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:43:27.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-mon[36886]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-13T06:43:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:43:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:27 smithi189 ceph-mon[36616]: mgrmap e14: x(active, since 87s), standbys: z 2024-06-13T06:43:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:27 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-13T06:43:28.367 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:27.911+0000 7f523f3bb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-13T06:43:28.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:28 smithi007 ceph-mon[32077]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:28.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:28 smithi175 ceph-mon[36886]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:28.867 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:28.505+0000 7f523f3bb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-13T06:43:28.867 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 06:43:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y[38267]: 2024-06-13T06:43:28.719+0000 7f523f3bb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-13T06:43:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:28 smithi189 ceph-mon[36616]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:29.698+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mon.1 v2:172.21.15.189:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f3000021cd0 con 0x7f3010102f80 2024-06-13T06:43:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:29 smithi175 ceph-mon[36886]: Standby manager daemon y started 2024-06-13T06:43:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:29 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:43:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:29 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:29 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:43:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:29 smithi175 ceph-mon[36886]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:29 smithi189 ceph-mon[36616]: Standby manager daemon y started 2024-06-13T06:43:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:29 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:43:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:29 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:29 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:43:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:29 smithi189 ceph-mon[36616]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:29 smithi007 ceph-mon[32077]: Standby manager daemon y started 2024-06-13T06:43:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:29 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-13T06:43:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:29 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-13T06:43:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:29 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-13T06:43:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:29 smithi007 ceph-mon[32077]: from='mgr.? 172.21.15.175:0/3588420626' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-13T06:43:30.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:30.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-13T06:43:30.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:43:30.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]: dispatch 2024-06-13T06:43:30.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]': finished 2024-06-13T06:43:30.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:43:30.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-13T06:43:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:43:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]: dispatch 2024-06-13T06:43:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]': finished 2024-06-13T06:43:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:43:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]: dispatch 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2724101387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e144dfd9-116a-4342-8fe5-e40f94ed6071"}]': finished 2024-06-13T06:43:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: osdmap e5: 1 total, 0 up, 1 in 2024-06-13T06:43:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:32.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:31 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1757883401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:43:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:31 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1757883401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:43:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:31 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1757883401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:43:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:32 smithi189 ceph-mon[36616]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:32 smithi175 ceph-mon[36886]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:33.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:32 smithi007 ceph-mon[32077]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:34.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:34 smithi007 ceph-mon[32077]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:34 smithi189 ceph-mon[36616]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:34 smithi175 ceph-mon[36886]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:37.020 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:36 smithi007 ceph-mon[32077]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:36 smithi189 ceph-mon[36616]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:36 smithi175 ceph-mon[36886]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:38.810 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:38 smithi007 ceph-mon[32077]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:38 smithi189 ceph-mon[36616]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:38 smithi175 ceph-mon[36886]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:40.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:43:40.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:40.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:39 smithi007 ceph-mon[32077]: Deploying daemon osd.0 on smithi007 2024-06-13T06:43:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:43:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:39 smithi189 ceph-mon[36616]: Deploying daemon osd.0 on smithi007 2024-06-13T06:43:40.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-13T06:43:40.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:40.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:39 smithi175 ceph-mon[36886]: Deploying daemon osd.0 on smithi007 2024-06-13T06:43:40.974 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:40 smithi007 ceph-mon[32077]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:40.989 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:40 smithi189 ceph-mon[36616]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:40 smithi175 ceph-mon[36886]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:42 smithi189 ceph-mon[36616]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:42 smithi175 ceph-mon[36886]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:43.124 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:42 smithi007 ceph-mon[32077]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:44 smithi189 ceph-mon[36616]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:44 smithi175 ceph-mon[36886]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:45.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:44 smithi007 ceph-mon[32077]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:46.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:46 smithi007 ceph-mon[32077]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:46 smithi189 ceph-mon[36616]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:46 smithi175 ceph-mon[36886]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:48.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:48.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:48.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:48 smithi007 ceph-mon[32077]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:48 smithi189 ceph-mon[36616]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:48 smithi175 ceph-mon[36886]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:51.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:51.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:51.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.585+0000 7f3007e00700 1 -- 172.21.15.7:0/2251497829 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30100628d0 con 0x7f2ff4039330 2024-06-13T06:43:51.585 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-06-13T06:43:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 msgr2=0x7f2ff403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 0x7f2ff403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc011490 tx=0x7f2ffc009260 comp rx=0 tx=0).stop 2024-06-13T06:43:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 msgr2=0x7f301019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f300000eb60 tx=0x7f300000ee70 comp rx=0 tx=0).stop 2024-06-13T06:43:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 shutdown_connections 2024-06-13T06:43:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30101081e0 0x7f30101a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010102f80 0x7f301019a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ff4039330 0x7f2ff403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 --2- 172.21.15.7:0/2251497829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30101058b0 0x7f301019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.589+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 >> 172.21.15.7:0/2251497829 conn(0x7f30100fc990 msgr2=0x7f3010102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.590+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 shutdown_connections 2024-06-13T06:43:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:43:51.590+0000 7f3016200700 1 -- 172.21.15.7:0/2251497829 wait complete. 2024-06-13T06:43:51.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:51 smithi007 ceph-mon[32077]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:43:51.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:51.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:51.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:51 smithi189 ceph-mon[36616]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:43:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:51 smithi175 ceph-mon[36886]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-13T06:43:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:43:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:52.331 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.0.service 2024-06-13T06:43:52.335 INFO:tasks.cephadm:Deploying osd.1 on smithi175 with /dev/vg_nvme/lv_4... 2024-06-13T06:43:52.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:43:52.578 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:43:52.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:52 smithi007 ceph-mon[32077]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:52.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:52 smithi007 ceph-mon[32077]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:43:52.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:52 smithi007 ceph-mon[32077]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:43:52.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:52 smithi007 ceph-mon[32077]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:43:52.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:52 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:52.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:52 smithi175 ceph-mon[36886]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:52.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:52 smithi175 ceph-mon[36886]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:43:52.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:52 smithi175 ceph-mon[36886]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:43:52.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:52 smithi175 ceph-mon[36886]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:43:52.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:52 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:52 smithi189 ceph-mon[36616]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:53.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:52 smithi189 ceph-mon[36616]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-13T06:43:53.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:52 smithi189 ceph-mon[36616]: osdmap e6: 1 total, 0 up, 1 in 2024-06-13T06:43:53.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:52 smithi189 ceph-mon[36616]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:43:53.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.663 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0[44384]: 2024-06-13T06:43:53.357+0000 7fd1ba000700 -1 osd.0 0 waiting for initial osdmap 2024-06-13T06:43:53.664 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0[44384]: 2024-06-13T06:43:53.400+0000 7fd1af400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:43:53.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:53 smithi175 ceph-mon[36886]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:43:53.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:53 smithi175 ceph-mon[36886]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:43:53.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:53 smithi175 ceph-mon[36886]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' 2024-06-13T06:43:53.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-mon[32077]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:43:53.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-mon[32077]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:43:53.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:53.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:53 smithi007 ceph-mon[32077]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' 2024-06-13T06:43:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:53 smithi189 ceph-mon[36616]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:43:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:53 smithi189 ceph-mon[36616]: osdmap e7: 1 total, 0 up, 1 in 2024-06-13T06:43:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:53 smithi189 ceph-mon[36616]: from='osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948]' entity='osd.0' 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114101 s, 919 MB/s 2024-06-13T06:43:54.432 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-13T06:43:54.997 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] boot 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:43:54.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:54.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] boot 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:43:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: osd.0 [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] boot 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: osdmap e8: 1 total, 1 up, 1 in 2024-06-13T06:43:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-13T06:43:55.231 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:43:56.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:56 smithi175 ceph-mon[36886]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:43:56.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 -- 172.21.15.175:0/2741561089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 msgr2=0x7f5b04102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 --2- 172.21.15.175:0/2741561089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04102910 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5af40099c0 tx=0x7f5af4009cd0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 -- 172.21.15.175:0/2741561089 shutdown_connections 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 --2- 172.21.15.175:0/2741561089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b04104090 0x7f5b0410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 --2- 172.21.15.175:0/2741561089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04102910 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 --2- 172.21.15.175:0/2741561089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 0x7f5b04103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 -- 172.21.15.175:0/2741561089 >> 172.21.15.175:0/2741561089 conn(0x7f5b040fda90 msgr2=0x7f5b040ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 -- 172.21.15.175:0/2741561089 shutdown_connections 2024-06-13T06:43:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.582+0000 7f5b09718700 1 -- 172.21.15.175:0/2741561089 wait complete. 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 Processor -- start 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 -- start start 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 0x7f5b04074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b04104090 0x7f5b04074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410e200 con 0x7f5b04104090 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410e080 con 0x7f5b041024f0 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b09718700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0410e380 con 0x7f5b041036f0 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b02ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:56.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b02ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37032/0 (socket says 172.21.15.175:37032) 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.583+0000 7f5b02ffd700 1 -- 172.21.15.175:0/1464527819 learned_addr learned my addr 172.21.15.175:0/1464527819 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b027fc700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 0x7f5b04074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 -- 172.21.15.175:0/1464527819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 msgr2=0x7f5b04074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 0x7f5b04074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 -- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b04104090 msgr2=0x7f5b04074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b04104090 0x7f5b04074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 -- 172.21.15.175:0/1464527819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b04075670 con 0x7f5b041024f0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b02ffd700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c010 tx=0x7f5af4009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4026070 con 0x7f5b041024f0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af400f460 con 0x7f5b041024f0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4021670 con 0x7f5b041024f0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b041a2a00 con 0x7f5b041024f0 2024-06-13T06:43:56.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.584+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041a2ed0 con 0x7f5b041024f0 2024-06-13T06:43:56.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.585+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b04069180 con 0x7f5b041024f0 2024-06-13T06:43:56.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.586+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f5af4029070 con 0x7f5b041024f0 2024-06-13T06:43:56.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.587+0000 7f5aebfff700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 0x7f5aec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:43:56.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.587+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f5af40ce0c0 con 0x7f5b041024f0 2024-06-13T06:43:56.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.587+0000 7f5b027fc700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 0x7f5aec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:43:56.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.588+0000 7f5b027fc700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 0x7f5aec0a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5af8006fd0 tx=0x7f5af8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:43:56.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.591+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5af405d850 con 0x7f5b041024f0 2024-06-13T06:43:56.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:56 smithi007 ceph-mon[32077]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:43:56.794 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:43:56.795+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5b040628d0 con 0x7f5aec0a2c10 2024-06-13T06:43:56.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:56 smithi189 ceph-mon[36616]: osdmap e9: 1 total, 1 up, 1 in 2024-06-13T06:43:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi007 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 17094M 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi007 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 17094M 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:57.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:43:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:43:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi007 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 17094M 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:43:57.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:43:58.672 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:58 smithi175 ceph-mon[36886]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:58 smithi189 ceph-mon[36616]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:58.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:58 smithi007 ceph-mon[32077]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:43:59.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:59 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/3895873465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:59 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:59 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]': finished 2024-06-13T06:43:59.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:59 smithi175 ceph-mon[36886]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:43:59.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:43:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:43:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:59 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/3895873465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:59 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:59 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]': finished 2024-06-13T06:43:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:59 smithi189 ceph-mon[36616]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:43:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:43:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:43:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:59 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/3895873465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:59 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]: dispatch 2024-06-13T06:43:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:59 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c21f9125-8329-4858-a611-5aeb5e3d6da0"}]': finished 2024-06-13T06:43:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:59 smithi007 ceph-mon[32077]: osdmap e10: 2 total, 1 up, 2 in 2024-06-13T06:43:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:43:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:00.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:00 smithi189 ceph-mon[36616]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:00.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:00 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/1481360886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:00.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:00 smithi175 ceph-mon[36886]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:00.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:00 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/1481360886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:00.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:00 smithi007 ceph-mon[32077]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:00.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:00 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/1481360886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:02.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:02 smithi189 ceph-mon[36616]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:02.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:02 smithi175 ceph-mon[36886]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:02.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:02 smithi007 ceph-mon[32077]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:04.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:04 smithi175 ceph-mon[36886]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:04.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:04 smithi189 ceph-mon[36616]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:04.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:04 smithi007 ceph-mon[32077]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:06 smithi189 ceph-mon[36616]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:44:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:06 smithi175 ceph-mon[36886]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:44:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:06.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:06 smithi007 ceph-mon[32077]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:06.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-13T06:44:06.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:07.805 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:07 smithi175 ceph-mon[36886]: Deploying daemon osd.1 on smithi175 2024-06-13T06:44:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:07 smithi189 ceph-mon[36616]: Deploying daemon osd.1 on smithi175 2024-06-13T06:44:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:07 smithi007 ceph-mon[32077]: Deploying daemon osd.1 on smithi175 2024-06-13T06:44:08.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:08 smithi175 ceph-mon[36886]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:08 smithi189 ceph-mon[36616]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:08.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:08 smithi007 ceph-mon[32077]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:10 smithi189 ceph-mon[36616]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:10.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:10 smithi175 ceph-mon[36886]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:10.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:10 smithi007 ceph-mon[32077]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:12 smithi189 ceph-mon[36616]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:12 smithi175 ceph-mon[36886]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:12 smithi007 ceph-mon[32077]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:15.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:14 smithi175 ceph-mon[36886]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:15.082 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 1 on host 'smithi175' 2024-06-13T06:44:15.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.082+0000 7f5aebfff700 1 -- 172.21.15.175:0/1464527819 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b040628d0 con 0x7f5aec0a2c10 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 msgr2=0x7f5aec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 0x7f5aec0a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5af8006fd0 tx=0x7f5af8008040 comp rx=0 tx=0).stop 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 msgr2=0x7f5b04076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5af400c010 tx=0x7f5af4009f80 comp rx=0 tx=0).stop 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 shutdown_connections 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b04104090 0x7f5b04074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5aec0a2c10 0x7f5aec0a50d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:15.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b041024f0 0x7f5b04076230 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 --2- 172.21.15.175:0/1464527819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b041036f0 0x7f5b04074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 >> 172.21.15.175:0/1464527819 conn(0x7f5b040fda90 msgr2=0x7f5b04106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:44:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.085+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 shutdown_connections 2024-06-13T06:44:15.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:44:15.086+0000 7f5b09718700 1 -- 172.21.15.175:0/1464527819 wait complete. 2024-06-13T06:44:15.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:14 smithi189 ceph-mon[36616]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:15.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:14 smithi007 ceph-mon[32077]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:15.674 DEBUG:teuthology.orchestra.run.smithi175:osd.1> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.1.service 2024-06-13T06:44:15.676 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-06-13T06:44:15.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-13T06:44:15.922 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:16.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:16 smithi189 ceph-mon[36616]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:16 smithi175 ceph-mon[36886]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:16.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:16 smithi007 ceph-mon[32077]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:17.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:17 smithi189 ceph-mon[36616]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-mon[36886]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-13T06:44:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: from='osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: osdmap e11: 2 total, 1 up, 2 in 2024-06-13T06:44:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:17 smithi007 ceph-mon[32077]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:44:17.827 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1[41444]: 2024-06-13T06:44:17.548+0000 7fb0b110a700 -1 osd.1 0 waiting for initial osdmap 2024-06-13T06:44:17.827 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1[41444]: 2024-06-13T06:44:17.552+0000 7fb0a8ef5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:44:17.834 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-13T06:44:17.834 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-13T06:44:17.835 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-06-13T06:44:17.835 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-06-13T06:44:17.835 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011297 s, 928 MB/s 2024-06-13T06:44:17.835 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 2024-06-13T06:44:17.835 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-06-13T06:44:18.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:18 smithi007 ceph-mon[32077]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:44:18.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:18 smithi007 ceph-mon[32077]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:44:18.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:18 smithi007 ceph-mon[32077]: from='osd.1 ' entity='osd.1' 2024-06-13T06:44:18.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:18 smithi189 ceph-mon[36616]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:44:18.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:18 smithi189 ceph-mon[36616]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:44:18.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:18 smithi189 ceph-mon[36616]: from='osd.1 ' entity='osd.1' 2024-06-13T06:44:18.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:18 smithi175 ceph-mon[36886]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:44:18.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:18 smithi175 ceph-mon[36886]: osdmap e12: 2 total, 1 up, 2 in 2024-06-13T06:44:18.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:18 smithi175 ceph-mon[36886]: from='osd.1 ' entity='osd.1' 2024-06-13T06:44:18.450 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:18.455 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-06-13T06:44:18.694 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:44:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:44:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:44:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] boot 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] boot 2024-06-13T06:44:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:44:19.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:44:19.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-13T06:44:19.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: osd.1 [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] boot 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: osdmap e13: 2 total, 2 up, 2 in 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:19.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 -- 172.21.15.189:0/369614015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 msgr2=0x7f2b7c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 --2- 172.21.15.189:0/369614015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c105360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009990 tx=0x7f2b6c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 -- 172.21.15.189:0/369614015 shutdown_connections 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 --2- 172.21.15.189:0/369614015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1081d0 0x7f2b7c10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 --2- 172.21.15.189:0/369614015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c105360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 --2- 172.21.15.189:0/369614015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1058a0 0x7f2b7c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.234+0000 7f2b812aa700 1 -- 172.21.15.189:0/369614015 >> 172.21.15.189:0/369614015 conn(0x7f2b7c0fc980 msgr2=0x7f2b7c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 -- 172.21.15.189:0/369614015 shutdown_connections 2024-06-13T06:44:20.236 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 -- 172.21.15.189:0/369614015 wait complete. 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 Processor -- start 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 -- start start 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.235+0000 7f2b812aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1058a0 0x7f2b7c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1081d0 0x7f2b7c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10e130 con 0x7f2b7c1058a0 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10dfb0 con 0x7f2b7c1081d0 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c10e2b0 con 0x7f2b7c102f70 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:38398/0 (socket says 172.21.15.189:38398) 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 -- 172.21.15.189:0/2042255692 learned_addr learned my addr 172.21.15.189:0/2042255692 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7b7fe700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1081d0 0x7f2b7c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:20.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 -- 172.21.15.189:0/2042255692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1081d0 msgr2=0x7f2b7c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1081d0 0x7f2b7c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 -- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1058a0 msgr2=0x7f2b7c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1058a0 0x7f2b7c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 -- 172.21.15.189:0/2042255692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c1a02e0 con 0x7f2b7c102f70 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b7affd700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c027070 con 0x7f2b7c102f70 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c004150 con 0x7f2b7c102f70 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c1a0570 con 0x7f2b7c102f70 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00f6c0 con 0x7f2b7c102f70 2024-06-13T06:44:20.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.236+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c1a0a40 con 0x7f2b7c102f70 2024-06-13T06:44:20.239 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.237+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c069180 con 0x7f2b7c102f70 2024-06-13T06:44:20.239 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.238+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f2b6c02a070 con 0x7f2b7c102f70 2024-06-13T06:44:20.239 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.239+0000 7f2b63fff700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 0x7f2b640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:20.239 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.239+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f2b6c0ceb00 con 0x7f2b7c102f70 2024-06-13T06:44:20.239 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.239+0000 7f2b7a7fc700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 0x7f2b640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:20.240 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.240+0000 7f2b7a7fc700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 0x7f2b640a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005fd0 tx=0x7f2b70005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:44:20.243 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.244+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b6c013070 con 0x7f2b7c102f70 2024-06-13T06:44:20.243 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:20 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi175 2024-06-13T06:44:20.243 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:20 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-13T06:44:20.243 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:20 smithi189 ceph-mon[36616]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:44:20.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:20 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi175 2024-06-13T06:44:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:20 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-13T06:44:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:20 smithi007 ceph-mon[32077]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:44:20.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:20.447+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2b7c0628d0 con 0x7f2b640a2b90 2024-06-13T06:44:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:20 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi175 2024-06-13T06:44:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:20 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 17088M 2024-06-13T06:44:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:20 smithi175 ceph-mon[36886]: osdmap e14: 2 total, 2 up, 2 in 2024-06-13T06:44:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:21 smithi189 ceph-mon[36616]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:21.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:21 smithi007 ceph-mon[32077]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:21.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:21.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:21.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:21 smithi175 ceph-mon[36886]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:22.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:22 smithi189 ceph-mon[36616]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:22.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:22 smithi007 ceph-mon[32077]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:22.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:22 smithi175 ceph-mon[36886]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/2137929622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]': finished 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:44:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/2137929622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]': finished 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:44:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/2137929622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]: dispatch 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "073fad02-1136-4f2e-bd65-8a2a7ceb5a69"}]': finished 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: osdmap e15: 3 total, 2 up, 3 in 2024-06-13T06:44:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:24.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:24 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/299205283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:24.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:24 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/299205283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:24.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:24 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/299205283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:25 smithi189 ceph-mon[36616]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:25 smithi175 ceph-mon[36886]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:25.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:25 smithi007 ceph-mon[32077]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:27.287 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:27 smithi189 ceph-mon[36616]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:27 smithi175 ceph-mon[36886]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:27.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:27 smithi007 ceph-mon[32077]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:29.287 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:29 smithi189 ceph-mon[36616]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:29.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:29 smithi007 ceph-mon[32077]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:29 smithi175 ceph-mon[36886]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:30.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:44:30.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:30.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:44:30.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:30.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-13T06:44:30.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:31 smithi189 ceph-mon[36616]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:31 smithi189 ceph-mon[36616]: Deploying daemon osd.2 on smithi189 2024-06-13T06:44:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:31 smithi175 ceph-mon[36886]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:31 smithi175 ceph-mon[36886]: Deploying daemon osd.2 on smithi189 2024-06-13T06:44:31.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:31 smithi007 ceph-mon[32077]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:31.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:31 smithi007 ceph-mon[32077]: Deploying daemon osd.2 on smithi189 2024-06-13T06:44:33.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:33 smithi189 ceph-mon[36616]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:33 smithi175 ceph-mon[36886]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:33.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:33 smithi007 ceph-mon[32077]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:35 smithi189 ceph-mon[36616]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:35 smithi175 ceph-mon[36886]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:35.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:35 smithi007 ceph-mon[32077]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:36.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:36.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:36.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:37.564 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:37 smithi189 ceph-mon[36616]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:37 smithi175 ceph-mon[36886]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:37.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:37 smithi007 ceph-mon[32077]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:38.629 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.630+0000 7f2b63fff700 1 -- 172.21.15.189:0/2042255692 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2b7c0628d0 con 0x7f2b640a2b90 2024-06-13T06:44:38.630 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 msgr2=0x7f2b640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 0x7f2b640a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b70005fd0 tx=0x7f2b70005d00 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 msgr2=0x7f2b7c198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000c00 tx=0x7f2b6c0057c0 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 shutdown_connections 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2b640a2b90 0x7f2b640a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7c1081d0 0x7f2b7c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b7c102f70 0x7f2b7c198680 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 --2- 172.21.15.189:0/2042255692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c1058a0 0x7f2b7c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:38.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.632+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 >> 172.21.15.189:0/2042255692 conn(0x7f2b7c0fc980 msgr2=0x7f2b7c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:44:38.633 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.633+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 shutdown_connections 2024-06-13T06:44:38.633 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:44:38.633+0000 7f2b812aa700 1 -- 172.21.15.189:0/2042255692 wait complete. 2024-06-13T06:44:39.268 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.2.service 2024-06-13T06:44:39.270 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-06-13T06:44:39.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:44:39.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:39.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:39 smithi189 ceph-mon[36616]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:39 smithi175 ceph-mon[36886]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:39 smithi007 ceph-mon[32077]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-13T06:44:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:44:40.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:40.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:44:40.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:44:40.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:40.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:40.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:40 smithi189 ceph-mon[36616]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:40 smithi175 ceph-mon[36886]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-13T06:44:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: osdmap e16: 3 total, 2 up, 3 in 2024-06-13T06:44:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: from='osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:41.138 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:40 smithi007 ceph-mon[32077]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:44:41.490 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2[41683]: 2024-06-13T06:44:41.163+0000 7f71c7780700 -1 osd.2 0 waiting for initial osdmap 2024-06-13T06:44:41.491 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2[41683]: 2024-06-13T06:44:41.167+0000 7f71bf56b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:44:41.797 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-mon[36616]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:44:41.797 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-mon[36616]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:44:41.797 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:41.798 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:41.798 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:41 smithi189 ceph-mon[36616]: from='osd.2 ' entity='osd.2' 2024-06-13T06:44:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:41 smithi175 ceph-mon[36886]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:44:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:41 smithi175 ceph-mon[36886]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:44:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:41 smithi175 ceph-mon[36886]: from='osd.2 ' entity='osd.2' 2024-06-13T06:44:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:41 smithi007 ceph-mon[32077]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:44:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:41 smithi007 ceph-mon[32077]: osdmap e17: 3 total, 2 up, 3 in 2024-06-13T06:44:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:41 smithi007 ceph-mon[32077]: from='osd.2 ' entity='osd.2' 2024-06-13T06:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-13T06:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-13T06:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374825 s, 280 MB/s 2024-06-13T06:44:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-13T06:44:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:44:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:44:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] boot 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi189 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:42.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 17088M 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:42.917 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:42.918 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:42.918 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:44:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:44:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] boot 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi189 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 17088M 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:43.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: osd.2 [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] boot 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: osdmap e18: 3 total, 3 up, 3 in 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-13T06:44:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi189 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 17088M 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:44:43.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:43.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-06-13T06:44:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:44:44.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:44 smithi007 ceph-mon[32077]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:44:44.323 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:44.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:44 smithi189 ceph-mon[36616]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:44:44.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:44 smithi175 ceph-mon[36886]: osdmap e19: 3 total, 3 up, 3 in 2024-06-13T06:44:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:45.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:45 smithi007 ceph-mon[32077]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:45.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:45.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:45 smithi007 ceph-mon[32077]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:44:45.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:45 smithi189 ceph-mon[36616]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:45 smithi189 ceph-mon[36616]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:44:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:45 smithi175 ceph-mon[36886]: pgmap v91: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:45 smithi175 ceph-mon[36886]: osdmap e20: 3 total, 3 up, 3 in 2024-06-13T06:44:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T06:44:46.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 -- 172.21.15.7:0/2762882396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81810a3d0 msgr2=0x7fb81810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 --2- 172.21.15.7:0/2762882396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81810a3d0 0x7fb81810c860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb810009f90 tx=0x7fb81000b330 comp rx=0 tx=0).stop 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 -- 172.21.15.7:0/2762882396 shutdown_connections 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 --2- 172.21.15.7:0/2762882396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb81810a3d0 0x7fb81810c860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 --2- 172.21.15.7:0/2762882396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8180764c0 0x7fb818074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 --2- 172.21.15.7:0/2762882396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 0x7fb818075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 -- 172.21.15.7:0/2762882396 >> 172.21.15.7:0/2762882396 conn(0x7fb8180fda60 msgr2=0x7fb8180ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 -- 172.21.15.7:0/2762882396 shutdown_connections 2024-06-13T06:44:46.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.016+0000 7fb820600700 1 -- 172.21.15.7:0/2762882396 wait complete. 2024-06-13T06:44:46.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.017+0000 7fb820600700 1 Processor -- start 2024-06-13T06:44:46.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.017+0000 7fb820600700 1 -- start start 2024-06-13T06:44:46.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 0x7fb81819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:46.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 0x7fb8181a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181102e0 con 0x7fb81810a3d0 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb818110160 con 0x7fb8180764c0 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb820600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb818110460 con 0x7fb818074ef0 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb81d200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb81dc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 0x7fb81819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb81d200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33218/0 (socket says 172.21.15.7:33218) 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb81d200700 1 -- 172.21.15.7:0/1610074618 learned_addr learned my addr 172.21.15.7:0/1610074618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:44:46.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.018+0000 7fb81e600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 0x7fb8181a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 -- 172.21.15.7:0/1610074618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 msgr2=0x7fb81819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 0x7fb81819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 -- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 msgr2=0x7fb8181a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:44:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 0x7fb8181a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 -- 172.21.15.7:0/1610074618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181a24f0 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81e600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 0x7fb8181a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb81d200700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ebf0 tx=0x7fb80400ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400cd40 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb804010910 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8181a27e0 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040189c0 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.019+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8181a2d70 con 0x7fb8180764c0 2024-06-13T06:44:46.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.021+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818069180 con 0x7fb8180764c0 2024-06-13T06:44:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.023+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fb80401f020 con 0x7fb8180764c0 2024-06-13T06:44:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.024+0000 7fb816000700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 0x7fb8000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:44:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.024+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fb8040cddf0 con 0x7fb8180764c0 2024-06-13T06:44:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.024+0000 7fb81dc00700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 0x7fb8000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:44:46.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.025+0000 7fb81dc00700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 0x7fb8000a5150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8180768a0 tx=0x7fb80c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:44:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.028+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb804053850 con 0x7fb8180764c0 2024-06-13T06:44:46.162 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:45 smithi007 sudo[49006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:46.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:45 smithi007 sudo[49006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:46.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:45 smithi007 sudo[49006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:46.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:46 smithi007 sudo[49006]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:46.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.228+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb8040980d0 con 0x7fb8180764c0 2024-06-13T06:44:46.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:44:46.286+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb8180628d0 con 0x7fb8000a2c90 2024-06-13T06:44:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:46 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:46.574 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:46 smithi007 ceph-mon[32077]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:44:46.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:46 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:46.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:46 smithi189 ceph-mon[36616]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:44:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:46 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-13T06:44:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:46 smithi175 ceph-mon[36886]: osdmap e21: 3 total, 3 up, 3 in 2024-06-13T06:44:46.912 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:46 smithi007 sudo[49096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:46.912 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:46 smithi007 sudo[49096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:46.913 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:46 smithi007 sudo[49096]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:46.913 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 06:44:46 smithi007 sudo[49096]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:47.109 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:46 smithi175 sudo[45143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:47.109 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:46 smithi175 sudo[45143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:47.109 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:46 smithi175 sudo[45143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:47.109 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:46 smithi175 sudo[45143]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:47.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:47 smithi175 sudo[45146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:47.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:47 smithi175 sudo[45146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:47.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:47 smithi175 sudo[45146]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:47.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 06:44:47 smithi175 sudo[45146]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: osdmap e22: 3 total, 3 up, 3 in 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:47.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:47.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:47.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: osdmap e22: 3 total, 3 up, 3 in 2024-06-13T06:44:47.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:44:47.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:47.568 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:47.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:47.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:47.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:47.569 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:47.586 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45301]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:47.586 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:47.586 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45301]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:47.586 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45301]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: osdmap e22: 3 total, 3 up, 3 in 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:48.086 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45304]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:48.086 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:48.086 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45304]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:48.086 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 06:44:47 smithi189 sudo[45304]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:48.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:48.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49170]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:48.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:48.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:48.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:48.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 sudo[49200]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:48.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:48.903 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:48 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:48 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:48 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:49.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45308]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:49.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:48 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:44:49.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:48 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 sudo[45311]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45158]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-13T06:44:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45158]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45158]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45158]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45163]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 sudo[45163]: pam_unix(sudo:session): session closed for user root 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-13T06:44:50.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1758852190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]': finished 2024-06-13T06:44:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:44:51.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1758852190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]': finished 2024-06-13T06:44:51.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:44:51.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:44:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-13T06:44:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-13T06:44:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1758852190' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]: dispatch 2024-06-13T06:44:51.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aad01f-267a-43ad-b0d3-acb77db40434"}]': finished 2024-06-13T06:44:51.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: osdmap e23: 4 total, 3 up, 4 in 2024-06-13T06:44:51.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:44:52.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:51 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1035651203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:52.236 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:51 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1035651203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:52.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:51 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1035651203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:44:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:52 smithi189 ceph-mon[36616]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:53.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:52 smithi007 ceph-mon[32077]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:52 smithi175 ceph-mon[36886]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:54 smithi007 ceph-mon[32077]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:54 smithi189 ceph-mon[36616]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:54 smithi175 ceph-mon[36886]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:44:57.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:56 smithi007 ceph-mon[32077]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:44:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:56 smithi189 ceph-mon[36616]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:44:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:56 smithi175 ceph-mon[36886]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-13T06:44:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:58 smithi007 ceph-mon[32077]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:44:59.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:58 smithi189 ceph-mon[36616]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:44:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:58 smithi175 ceph-mon[36886]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-13T06:45:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:45:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:44:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:45:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:44:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:00.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-13T06:45:00.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:44:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:01.191 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:00 smithi007 ceph-mon[32077]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:45:01.191 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:00 smithi007 ceph-mon[32077]: Deploying daemon osd.3 on smithi007 2024-06-13T06:45:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:00 smithi189 ceph-mon[36616]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:45:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:00 smithi189 ceph-mon[36616]: Deploying daemon osd.3 on smithi007 2024-06-13T06:45:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:00 smithi175 ceph-mon[36886]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-13T06:45:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:00 smithi175 ceph-mon[36886]: Deploying daemon osd.3 on smithi007 2024-06-13T06:45:03.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:02 smithi007 ceph-mon[32077]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 818 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-13T06:45:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:02 smithi189 ceph-mon[36616]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 818 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-13T06:45:03.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:02 smithi175 ceph-mon[36886]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 818 B/s rd, 23 KiB/s wr, 2 op/s 2024-06-13T06:45:05.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:04 smithi007 ceph-mon[32077]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:04 smithi189 ceph-mon[36616]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:04 smithi175 ceph-mon[36886]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:06 smithi007 ceph-mon[32077]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:06 smithi189 ceph-mon[36616]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:06 smithi175 ceph-mon[36886]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-13T06:45:09.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:09 smithi007 ceph-mon[32077]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:09.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:09.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:09.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:09 smithi189 ceph-mon[36616]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:09 smithi175 ceph-mon[36886]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:11.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:11 smithi007 ceph-mon[32077]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:11 smithi189 ceph-mon[36616]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:11 smithi175 ceph-mon[36886]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:13.249 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-06-13T06:45:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.248+0000 7fb816000700 1 -- 172.21.15.7:0/1610074618 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8180628d0 con 0x7fb8000a2c90 2024-06-13T06:45:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 msgr2=0x7fb8000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 0x7fb8000a5150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8180768a0 tx=0x7fb80c009260 comp rx=0 tx=0).stop 2024-06-13T06:45:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 msgr2=0x7fb81819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb80400ebf0 tx=0x7fb80400ef00 comp rx=0 tx=0).stop 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 shutdown_connections 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb81810a3d0 0x7fb8181a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb818074ef0 0x7fb81819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8000a2c90 0x7fb8000a5150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 --2- 172.21.15.7:0/1610074618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8180764c0 0x7fb81819add0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.252+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 >> 172.21.15.7:0/1610074618 conn(0x7fb8180fda60 msgr2=0x7fb8181062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.253+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 shutdown_connections 2024-06-13T06:45:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:45:13.253+0000 7fb820600700 1 -- 172.21.15.7:0/1610074618 wait complete. 2024-06-13T06:45:13.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:13 smithi007 ceph-mon[32077]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:13.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:13.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:13.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:13 smithi189 ceph-mon[36616]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:13.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:13.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:13.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:13.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:13 smithi175 ceph-mon[36886]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-13T06:45:14.059 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.3.service 2024-06-13T06:45:14.066 INFO:tasks.cephadm:Deploying osd.4 on smithi175 with /dev/vg_nvme/lv_3... 2024-06-13T06:45:14.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:45:14.311 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:14.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:14.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:14.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:45:14.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:45:14.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:14.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:14.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:14.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:14.414 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T06:45:14.203+0000 7fcab3600700 -1 osd.3 0 waiting for initial osdmap 2024-06-13T06:45:14.414 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 06:45:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T06:45:14.215+0000 7fcaae400700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:45:14.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-13T06:45:14.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: osdmap e24: 4 total, 3 up, 4 in 2024-06-13T06:45:14.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:14.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:45:14.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:14.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:14.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: osdmap e25: 4 total, 3 up, 4 in 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.474 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] boot 2024-06-13T06:45:15.475 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:45:15.475 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: osdmap e25: 4 total, 3 up, 4 in 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] boot 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:45:15.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-13T06:45:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:45:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: osdmap e25: 4 total, 3 up, 4 in 2024-06-13T06:45:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:15.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: osd.3 [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] boot 2024-06-13T06:45:15.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: osdmap e26: 4 total, 4 up, 4 in 2024-06-13T06:45:15.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-13T06:45:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:45:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:45:16.321 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-13T06:45:16.321 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-13T06:45:16.321 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130671 s, 802 MB/s 2024-06-13T06:45:16.322 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-13T06:45:16.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:16 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:45:16.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:16 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:45:16.397 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:16 smithi175 ceph-mon[36886]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:45:16.525 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:16 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:45:16.526 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:16 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:45:16.526 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:16 smithi007 ceph-mon[32077]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:45:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:16 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:45:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:16 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:45:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:16 smithi189 ceph-mon[36616]: osdmap e27: 4 total, 4 up, 4 in 2024-06-13T06:45:16.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-06-13T06:45:17.167 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:45:17.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:17 smithi175 ceph-mon[36886]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:17.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:17 smithi175 ceph-mon[36886]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:45:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:17 smithi189 ceph-mon[36616]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:17 smithi189 ceph-mon[36616]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:45:17.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:17 smithi007 ceph-mon[32077]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:17.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:17 smithi007 ceph-mon[32077]: osdmap e28: 4 total, 4 up, 4 in 2024-06-13T06:45:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- 172.21.15.175:0/1608512264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c100d70 msgr2=0x7fa89c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:18.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- 172.21.15.175:0/1608512264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c100d70 0x7fa89c1031a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa890009990 tx=0x7fa890009ca0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- 172.21.15.175:0/1608512264 shutdown_connections 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- 172.21.15.175:0/1608512264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c1060b0 0x7fa89c106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- 172.21.15.175:0/1608512264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c100d70 0x7fa89c1031a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- 172.21.15.175:0/1608512264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1036e0 0x7fa89c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- 172.21.15.175:0/1608512264 >> 172.21.15.175:0/1608512264 conn(0x7fa89c0fc9b0 msgr2=0x7fa89c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- 172.21.15.175:0/1608512264 shutdown_connections 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- 172.21.15.175:0/1608512264 wait complete. 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 Processor -- start 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- start start 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c100d70 0x7fa89c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:18.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1060b0 0x7fa89c1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c114730 con 0x7fa89c1060b0 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1145b0 con 0x7fa89c1036e0 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.673+0000 7fa8a315b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c1148b0 con 0x7fa89c100d70 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:54018/0 (socket says 172.21.15.175:54018) 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 -- 172.21.15.175:0/440545737 learned_addr learned my addr 172.21.15.175:0/440545737 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa8a0ef7700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c100d70 0x7fa89c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa8a16f8700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1060b0 0x7fa89c1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 -- 172.21.15.175:0/440545737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c100d70 msgr2=0x7fa89c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c100d70 0x7fa89c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.676 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 -- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1060b0 msgr2=0x7fa89c1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1060b0 0x7fa89c1a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 -- 172.21.15.175:0/440545737 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c1a6a10 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa89bfff700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa88800b530 tx=0x7fa88800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888015020 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa888011d10 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c1a6d00 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88800f2e0 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.674+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c1a7260 con 0x7fa89c1036e0 2024-06-13T06:45:18.677 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.675+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c069180 con 0x7fa89c1036e0 2024-06-13T06:45:18.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.676+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa888018020 con 0x7fa89c1036e0 2024-06-13T06:45:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.677+0000 7fa899ffb700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 0x7fa88c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.677+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fa8880cd670 con 0x7fa89c1036e0 2024-06-13T06:45:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.677+0000 7fa8a0ef7700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 0x7fa88c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:18.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.678+0000 7fa8a0ef7700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 0x7fa88c0a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa89000b5c0 tx=0x7fa890005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:18.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.682+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa88801f020 con 0x7fa89c1036e0 2024-06-13T06:45:18.890 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:18.890+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa89c0628d0 con 0x7fa88c0a2c20 2024-06-13T06:45:18.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:18 smithi007 ceph-mon[32077]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:18.999 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:18 smithi007 ceph-mon[32077]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:45:19.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:18 smithi189 ceph-mon[36616]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:19.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:18 smithi189 ceph-mon[36616]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:45:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:18 smithi175 ceph-mon[36886]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:18 smithi175 ceph-mon[36886]: osdmap e29: 4 total, 4 up, 4 in 2024-06-13T06:45:19.807 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:19 smithi175 ceph-mon[36886]: osdmap e30: 4 total, 4 up, 4 in 2024-06-13T06:45:19.807 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:19 smithi175 ceph-mon[36886]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:19.807 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:19.807 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:19.807 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:19 smithi189 ceph-mon[36616]: osdmap e30: 4 total, 4 up, 4 in 2024-06-13T06:45:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:19 smithi189 ceph-mon[36616]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:19 smithi007 ceph-mon[32077]: osdmap e30: 4 total, 4 up, 4 in 2024-06-13T06:45:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:19 smithi007 ceph-mon[32077]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi007 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 8547M 2024-06-13T06:45:21.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:21.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:21.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi007 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 8547M 2024-06-13T06:45:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:21.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:21.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi007 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:21.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 8547M 2024-06-13T06:45:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:21.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:21 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/3108515437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:21 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:21 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]': finished 2024-06-13T06:45:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:21 smithi007 ceph-mon[32077]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:45:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:22.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:21 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/3108515437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:21 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:21 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]': finished 2024-06-13T06:45:22.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:21 smithi189 ceph-mon[36616]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:45:22.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:22.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:21 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/3108515437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:21 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]: dispatch 2024-06-13T06:45:22.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:21 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d14e0c7-6370-4889-8f20-87458638b022"}]': finished 2024-06-13T06:45:22.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:21 smithi175 ceph-mon[36886]: osdmap e31: 5 total, 4 up, 5 in 2024-06-13T06:45:22.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:22 smithi189 ceph-mon[36616]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:45:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:22 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/3593513234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:23.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:22 smithi007 ceph-mon[32077]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:45:23.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:22 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/3593513234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:22 smithi175 ceph-mon[36886]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:45:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:22 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/3593513234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:25.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:24 smithi007 ceph-mon[32077]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:45:25.277 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:24 smithi175 ceph-mon[36886]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:45:25.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:24 smithi189 ceph-mon[36616]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:45:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:26 smithi175 ceph-mon[36886]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:45:27.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:26 smithi007 ceph-mon[32077]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:45:27.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:26 smithi189 ceph-mon[36616]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:45:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:28 smithi175 ceph-mon[36886]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:45:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:28 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:45:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:28 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:28 smithi007 ceph-mon[32077]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:45:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:45:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:28 smithi189 ceph-mon[36616]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:45:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:28 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-13T06:45:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:28 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:30.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:29 smithi175 ceph-mon[36886]: Deploying daemon osd.4 on smithi175 2024-06-13T06:45:30.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:29 smithi007 ceph-mon[32077]: Deploying daemon osd.4 on smithi175 2024-06-13T06:45:30.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:29 smithi189 ceph-mon[36616]: Deploying daemon osd.4 on smithi175 2024-06-13T06:45:31.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:30 smithi007 ceph-mon[32077]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:30 smithi189 ceph-mon[36616]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:30 smithi175 ceph-mon[36886]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:33.299 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:32 smithi175 ceph-mon[36886]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:32 smithi189 ceph-mon[36616]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:33.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:32 smithi007 ceph-mon[32077]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:35 smithi189 ceph-mon[36616]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:35 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:35 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:35 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:35 smithi175 ceph-mon[36886]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:35 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:35 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:35 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:35.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:35 smithi007 ceph-mon[32077]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:35 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:35 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:35 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:37.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:37.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:37.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:38.073 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.073+0000 7fa899ffb700 1 -- 172.21.15.175:0/440545737 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa89c0628d0 con 0x7fa88c0a2c20 2024-06-13T06:45:38.073 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 4 on host 'smithi175' 2024-06-13T06:45:38.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 msgr2=0x7fa88c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:38.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 0x7fa88c0a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa89000b5c0 tx=0x7fa890005fb0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 msgr2=0x7fa89c19f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:38.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa88800b530 tx=0x7fa88800b8f0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.075 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 shutdown_connections 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa89c1060b0 0x7fa89c1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa89c100d70 0x7fa89c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa88c0a2c20 0x7fa88c0a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 --2- 172.21.15.175:0/440545737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa89c1036e0 0x7fa89c19f340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 >> 172.21.15.175:0/440545737 conn(0x7fa89c0fc9b0 msgr2=0x7fa89c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 shutdown_connections 2024-06-13T06:45:38.076 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:45:38.076+0000 7fa8a315b700 1 -- 172.21.15.175:0/440545737 wait complete. 2024-06-13T06:45:38.741 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.4.service 2024-06-13T06:45:38.744 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-06-13T06:45:38.745 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-13T06:45:38.992 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:45:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:39.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:39 smithi189 ceph-mon[36616]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:39.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:39 smithi175 ceph-mon[36886]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:39.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:39.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:39.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:39.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:39.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:39 smithi007 ceph-mon[32077]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-13T06:45:40.247 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-mon[36886]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:45:40.247 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-mon[36886]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:45:40.248 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:40.248 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-mon[36886]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.248 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-mon[36886]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:40 smithi189 ceph-mon[36616]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:45:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:40 smithi189 ceph-mon[36616]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:45:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:40 smithi189 ceph-mon[36616]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:40 smithi189 ceph-mon[36616]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:40 smithi007 ceph-mon[32077]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-13T06:45:40.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:40 smithi007 ceph-mon[32077]: osdmap e32: 5 total, 4 up, 5 in 2024-06-13T06:45:40.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:40.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:40 smithi007 ceph-mon[32077]: from='osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:40 smithi007 ceph-mon[32077]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:45:40.617 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T06:45:40.455+0000 7f8927d6e700 -1 osd.4 0 waiting for initial osdmap 2024-06-13T06:45:40.617 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 06:45:40 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T06:45:40.463+0000 7f891fb59700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113717 s, 922 MB/s 2024-06-13T06:45:40.991 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-06-13T06:45:41.295 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:41.296 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:45:41.296 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: osdmap e33: 5 total, 4 up, 5 in 2024-06-13T06:45:41.296 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.296 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.296 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:41 smithi189 ceph-mon[36616]: from='osd.4 ' entity='osd.4' 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: osdmap e33: 5 total, 4 up, 5 in 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:41 smithi175 ceph-mon[36886]: from='osd.4 ' entity='osd.4' 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: osdmap e33: 5 total, 4 up, 5 in 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:41 smithi007 ceph-mon[32077]: from='osd.4 ' entity='osd.4' 2024-06-13T06:45:41.666 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-06-13T06:45:41.904 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] boot 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.271 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.272 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:42.272 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:42.272 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] boot 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:45:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:42.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:45:42.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:45:42.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: osd.4 [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] boot 2024-06-13T06:45:42.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: osdmap e34: 5 total, 5 up, 5 in 2024-06-13T06:45:42.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-13T06:45:42.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:45:42.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:43.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:43 smithi189 ceph-mon[36616]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:43.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:43 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi175 2024-06-13T06:45:43.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:43 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-13T06:45:43.191 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:43 smithi189 ceph-mon[36616]: osdmap e35: 5 total, 5 up, 5 in 2024-06-13T06:45:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:43 smithi175 ceph-mon[36886]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:43 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi175 2024-06-13T06:45:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:43 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-13T06:45:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:43 smithi175 ceph-mon[36886]: osdmap e35: 5 total, 5 up, 5 in 2024-06-13T06:45:43.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.394+0000 7fe9c460e700 1 -- 172.21.15.189:0/3323441461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102fd0 msgr2=0x7fe9bc109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:43.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.394+0000 7fe9c460e700 1 --2- 172.21.15.189:0/3323441461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc109880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400b0a0 tx=0x7fe9b400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 -- 172.21.15.189:0/3323441461 shutdown_connections 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- 172.21.15.189:0/3323441461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc109880 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- 172.21.15.189:0/3323441461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9bc101430 0x7fe9bc101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- 172.21.15.189:0/3323441461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102630 0x7fe9bc102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 -- 172.21.15.189:0/3323441461 >> 172.21.15.189:0/3323441461 conn(0x7fe9bc0fc9f0 msgr2=0x7fe9bc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 -- 172.21.15.189:0/3323441461 shutdown_connections 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 -- 172.21.15.189:0/3323441461 wait complete. 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 Processor -- start 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 -- start start 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9bc101430 0x7fe9bc06b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.395+0000 7fe9c460e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c460e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10d0c0 con 0x7fe9bc101430 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c460e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10cf40 con 0x7fe9bc102fd0 2024-06-13T06:45:43.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c460e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc10d240 con 0x7fe9bc102630 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47864/0 (socket says 172.21.15.189:47864) 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 -- 172.21.15.189:0/2961774522 learned_addr learned my addr 172.21.15.189:0/2961774522 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c2bab700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 -- 172.21.15.189:0/2961774522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102fd0 msgr2=0x7fe9bc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 -- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9bc101430 msgr2=0x7fe9bc06b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9bc101430 0x7fe9bc06b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 -- 172.21.15.189:0/2961774522 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc1a0380 con 0x7fe9bc102630 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c1ba9700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c970 tx=0x7fe9ac00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac007aa0 con 0x7fe9bc102630 2024-06-13T06:45:43.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9ac004d10 con 0x7fe9bc102630 2024-06-13T06:45:43.399 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc1a0670 con 0x7fe9bc102630 2024-06-13T06:45:43.399 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00f450 con 0x7fe9bc102630 2024-06-13T06:45:43.399 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.396+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc1a0c30 con 0x7fe9bc102630 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.397+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc069180 con 0x7fe9bc102630 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.398+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe9ac004750 con 0x7fe9bc102630 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.399+0000 7fe9b37fe700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 0x7fe9a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.399+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fe9ac0cdd30 con 0x7fe9bc102630 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.399+0000 7fe9c23aa700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 0x7fe9a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:45:43.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.400+0000 7fe9c23aa700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 0x7fe9a80a5160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc102490 tx=0x7fe9b8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:45:43.403 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.403+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe9ac09bd90 con 0x7fe9bc102630 2024-06-13T06:45:43.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:43 smithi007 ceph-mon[32077]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-13T06:45:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:43 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi175 2024-06-13T06:45:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:43 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 8544M 2024-06-13T06:45:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:43 smithi007 ceph-mon[32077]: osdmap e35: 5 total, 5 up, 5 in 2024-06-13T06:45:43.606 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:45:43.606+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe9bc0628d0 con 0x7fe9a80a2ca0 2024-06-13T06:45:44.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:44.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:44.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:44.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:44.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:44.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:44.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:45:44.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:45:44.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:45.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:45 smithi189 ceph-mon[36616]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:45.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:45 smithi189 ceph-mon[36616]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:45 smithi175 ceph-mon[36886]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:45 smithi175 ceph-mon[36886]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:45.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:45 smithi007 ceph-mon[32077]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:45.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:45 smithi007 ceph-mon[32077]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:45:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/307280140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]': finished 2024-06-13T06:45:47.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:45:47.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:45:47.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:47 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/2206222453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/307280140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]': finished 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:45:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:47 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/2206222453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/307280140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]: dispatch 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f01cff3-7564-4832-a4f8-0b2920c1b201"}]': finished 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: osdmap e36: 6 total, 5 up, 6 in 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:45:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/2206222453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:45:49.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:49 smithi189 ceph-mon[36616]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:49.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:49 smithi175 ceph-mon[36886]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:49.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:49 smithi007 ceph-mon[32077]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:51 smithi189 ceph-mon[36616]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:51 smithi175 ceph-mon[36886]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:51.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:51 smithi007 ceph-mon[32077]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:53.241 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:53 smithi189 ceph-mon[36616]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:53 smithi175 ceph-mon[36886]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:53.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:53 smithi007 ceph-mon[32077]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:54.345 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:45:54.345 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:54.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:45:54.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:54.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-13T06:45:54.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:45:55.202 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:55 smithi189 ceph-mon[36616]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:55.203 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:55 smithi189 ceph-mon[36616]: Deploying daemon osd.5 on smithi189 2024-06-13T06:45:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:55 smithi175 ceph-mon[36886]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:55 smithi175 ceph-mon[36886]: Deploying daemon osd.5 on smithi189 2024-06-13T06:45:55.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:55 smithi007 ceph-mon[32077]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:55.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:55 smithi007 ceph-mon[32077]: Deploying daemon osd.5 on smithi189 2024-06-13T06:45:57.351 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:57 smithi189 ceph-mon[36616]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:57 smithi175 ceph-mon[36886]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:57.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:57 smithi007 ceph-mon[32077]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:59 smithi175 ceph-mon[36886]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:45:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:59 smithi007 ceph-mon[32077]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:45:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:59.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:59 smithi189 ceph-mon[36616]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:45:59.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:45:59.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:45:59.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:45:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:01.272 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:01 smithi189 ceph-mon[36616]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:01 smithi175 ceph-mon[36886]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:01.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:01 smithi007 ceph-mon[32077]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:03.146 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.147+0000 7fe9b37fe700 1 -- 172.21.15.189:0/2961774522 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9bc0628d0 con 0x7fe9a80a2ca0 2024-06-13T06:46:03.147 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 msgr2=0x7fe9a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 0x7fe9a80a5160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc102490 tx=0x7fe9b8006d20 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 msgr2=0x7fe9bc19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c970 tx=0x7fe9ac00cc80 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 shutdown_connections 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe9a80a2ca0 0x7fe9a80a5160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9bc102fd0 0x7fe9bc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9bc101430 0x7fe9bc06b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 --2- 172.21.15.189:0/2961774522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9bc102630 0x7fe9bc19d470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 >> 172.21.15.189:0/2961774522 conn(0x7fe9bc0fc9f0 msgr2=0x7fe9bc105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:03.149 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 shutdown_connections 2024-06-13T06:46:03.150 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:46:03.149+0000 7fe9c460e700 1 -- 172.21.15.189:0/2961774522 wait complete. 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-13T06:46:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:03.752 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.5.service 2024-06-13T06:46:03.754 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-06-13T06:46:03.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:46:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-mon[36616]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:46:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-mon[36616]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-mon[36616]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:46:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-mon[36616]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:04 smithi175 ceph-mon[36886]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:46:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:04 smithi175 ceph-mon[36886]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:04 smithi175 ceph-mon[36886]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:46:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:04 smithi175 ceph-mon[36886]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:04 smithi007 ceph-mon[32077]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-13T06:46:04.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:04 smithi007 ceph-mon[32077]: from='osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:04 smithi007 ceph-mon[32077]: osdmap e37: 6 total, 5 up, 6 in 2024-06-13T06:46:04.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:04.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:04 smithi007 ceph-mon[32077]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:46:04.875 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T06:46:04.623+0000 7f01c8569700 -1 osd.5 0 waiting for initial osdmap 2024-06-13T06:46:04.876 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 06:46:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T06:46:04.632+0000 7f01c0354700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: osdmap e38: 6 total, 5 up, 6 in 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: from='osd.5 ' entity='osd.5' 2024-06-13T06:46:05.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] boot 2024-06-13T06:46:05.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:46:05.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:46:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: osdmap e38: 6 total, 5 up, 6 in 2024-06-13T06:46:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: from='osd.5 ' entity='osd.5' 2024-06-13T06:46:05.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] boot 2024-06-13T06:46:05.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:46:05.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: osdmap e38: 6 total, 5 up, 6 in 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: from='osd.5 ' entity='osd.5' 2024-06-13T06:46:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: osd.5 [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] boot 2024-06-13T06:46:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: osdmap e39: 6 total, 6 up, 6 in 2024-06-13T06:46:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-13T06:46:06.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:46:06.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:06.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:46:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:46:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:46:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:07.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:46:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:46:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: osdmap e40: 6 total, 6 up, 6 in 2024-06-13T06:46:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368785 s, 284 MB/s 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-06-13T06:46:07.504 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-13T06:46:07.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:07 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi189 2024-06-13T06:46:07.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:07 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 8544M 2024-06-13T06:46:07.874 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:07 smithi007 ceph-mon[32077]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:46:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:07 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi189 2024-06-13T06:46:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:07 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 8544M 2024-06-13T06:46:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:07 smithi189 ceph-mon[36616]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:46:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:07 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi189 2024-06-13T06:46:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:07 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 8544M 2024-06-13T06:46:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:07 smithi175 ceph-mon[36886]: osdmap e41: 6 total, 6 up, 6 in 2024-06-13T06:46:08.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-06-13T06:46:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:46:09.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:08 smithi189 ceph-mon[36616]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:09.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:08 smithi189 ceph-mon[36616]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:46:09.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:08 smithi007 ceph-mon[32077]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:09.097 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:08 smithi007 ceph-mon[32077]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:46:09.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:08 smithi175 ceph-mon[36886]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:09.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:08 smithi175 ceph-mon[36886]: osdmap e42: 6 total, 6 up, 6 in 2024-06-13T06:46:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:09 smithi189 ceph-mon[36616]: osdmap e43: 6 total, 6 up, 6 in 2024-06-13T06:46:10.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:09 smithi175 ceph-mon[36886]: osdmap e43: 6 total, 6 up, 6 in 2024-06-13T06:46:10.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:09 smithi007 ceph-mon[32077]: osdmap e43: 6 total, 6 up, 6 in 2024-06-13T06:46:10.870 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:10 smithi007 ceph-mon[32077]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.047+0000 7fedd7200700 1 -- 172.21.15.7:0/2710772044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd8074ef0 msgr2=0x7fedd8075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.047+0000 7fedd7200700 1 --2- 172.21.15.7:0/2710772044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd8074ef0 0x7fedd8075370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fedc00098f0 tx=0x7fedc0009c00 comp rx=0 tx=0).stop 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 -- 172.21.15.7:0/2710772044 shutdown_connections 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 --2- 172.21.15.7:0/2710772044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd810ea00 0x7fedd8110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 --2- 172.21.15.7:0/2710772044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 0x7fedd8074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 --2- 172.21.15.7:0/2710772044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd8074ef0 0x7fedd8075370 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 -- 172.21.15.7:0/2710772044 >> 172.21.15.7:0/2710772044 conn(0x7fedd80fd9c0 msgr2=0x7fedd80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 -- 172.21.15.7:0/2710772044 shutdown_connections 2024-06-13T06:46:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.048+0000 7fedd7200700 1 -- 172.21.15.7:0/2710772044 wait complete. 2024-06-13T06:46:11.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 Processor -- start 2024-06-13T06:46:11.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 -- start start 2024-06-13T06:46:11.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 0x7fedd819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd810ea00 0x7fedd81a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8114670 con 0x7fedd8074ef0 2024-06-13T06:46:11.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81144f0 con 0x7fedd810ea00 2024-06-13T06:46:11.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.049+0000 7fedd7200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81147f0 con 0x7fedd80764c0 2024-06-13T06:46:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42628/0 (socket says 172.21.15.7:42628) 2024-06-13T06:46:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 -- 172.21.15.7:0/2465001063 learned_addr learned my addr 172.21.15.7:0/2465001063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:46:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedcfe00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 0x7fedd819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd6800700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd810ea00 0x7fedd81a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 -- 172.21.15.7:0/2465001063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 msgr2=0x7fedd819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 0x7fedd819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 -- 172.21.15.7:0/2465001063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd810ea00 msgr2=0x7fedd81a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd810ea00 0x7fedd81a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.050+0000 7fedd5e00700 1 -- 172.21.15.7:0/2465001063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd81a6940 con 0x7fedd8074ef0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedd5e00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedc800e9c0 tx=0x7fedc800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8004a30 con 0x7fedd8074ef0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc800cca0 con 0x7fedd8074ef0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd81a6c30 con 0x7fedd8074ef0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8005660 con 0x7fedd8074ef0 2024-06-13T06:46:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.051+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a7120 con 0x7fedd8074ef0 2024-06-13T06:46:11.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.052+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd8069180 con 0x7fedd8074ef0 2024-06-13T06:46:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.056+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fedc800c7c0 con 0x7fedd8074ef0 2024-06-13T06:46:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.057+0000 7fedcea00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 0x7fedbc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.058+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fedc80ce590 con 0x7fedd8074ef0 2024-06-13T06:46:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.058+0000 7fedcfe00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 0x7fedbc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.059+0000 7fedcfe00700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 0x7fedbc0a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedc00098c0 tx=0x7fedc0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.061+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fedc8053850 con 0x7fedd8074ef0 2024-06-13T06:46:11.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:10 smithi189 ceph-mon[36616]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:46:11.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:10 smithi175 ceph-mon[36886]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:46:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:11.297+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fedd80628d0 con 0x7fedbc0a2c20 2024-06-13T06:46:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:11 smithi189 ceph-mon[36616]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:11 smithi175 ceph-mon[36886]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:12.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:11 smithi007 ceph-mon[32077]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:12.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:12 smithi007 ceph-mon[32077]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:46:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:12 smithi189 ceph-mon[36616]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:46:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:12 smithi175 ceph-mon[36886]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-13T06:46:15.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:14 smithi189 ceph-mon[36616]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-13T06:46:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:14 smithi175 ceph-mon[36886]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-13T06:46:15.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:14 smithi007 ceph-mon[32077]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-06-13T06:46:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:15 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]: dispatch 2024-06-13T06:46:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:15 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]': finished 2024-06-13T06:46:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:15 smithi189 ceph-mon[36616]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:46:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:15 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]: dispatch 2024-06-13T06:46:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:15 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]': finished 2024-06-13T06:46:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:15 smithi175 ceph-mon[36886]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:46:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:16.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:15 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]: dispatch 2024-06-13T06:46:16.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:15 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3470670573' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "785c5d21-30ac-46c0-9f6c-543778283cb7"}]': finished 2024-06-13T06:46:16.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:15 smithi007 ceph-mon[32077]: osdmap e44: 7 total, 6 up, 7 in 2024-06-13T06:46:16.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:17.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:16 smithi007 ceph-mon[32077]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:46:17.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:16 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1857012810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:17.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:16 smithi189 ceph-mon[36616]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:46:17.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:16 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1857012810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:17.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:16 smithi175 ceph-mon[36886]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-06-13T06:46:17.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:16 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1857012810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:19.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:18 smithi189 ceph-mon[36616]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:46:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:18 smithi175 ceph-mon[36886]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:46:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:18 smithi007 ceph-mon[32077]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-13T06:46:21.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:20 smithi189 ceph-mon[36616]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:20 smithi175 ceph-mon[36886]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:21.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:20 smithi007 ceph-mon[32077]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:22 smithi189 ceph-mon[36616]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:22 smithi175 ceph-mon[36886]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:23.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:22 smithi007 ceph-mon[32077]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:25.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:24 smithi189 ceph-mon[36616]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:25.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:24 smithi175 ceph-mon[36886]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:25.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:24 smithi007 ceph-mon[32077]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:46:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:25 smithi189 ceph-mon[36616]: Deploying daemon osd.6 on smithi007 2024-06-13T06:46:26.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:46:26.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:26.096 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:25 smithi007 ceph-mon[32077]: Deploying daemon osd.6 on smithi007 2024-06-13T06:46:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-13T06:46:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:25 smithi175 ceph-mon[36886]: Deploying daemon osd.6 on smithi007 2024-06-13T06:46:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:26 smithi189 ceph-mon[36616]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:26 smithi175 ceph-mon[36886]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:27.152 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:26 smithi007 ceph-mon[32077]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:28 smithi189 ceph-mon[36616]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:28 smithi175 ceph-mon[36886]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:29.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:28 smithi007 ceph-mon[32077]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:30.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:30 smithi007 ceph-mon[32077]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:30 smithi189 ceph-mon[36616]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:30 smithi175 ceph-mon[36886]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:32 smithi189 ceph-mon[36616]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:32 smithi175 ceph-mon[36886]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:33.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:32 smithi007 ceph-mon[32077]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:34.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:34 smithi007 ceph-mon[32077]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:34.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:34.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:34.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:34 smithi189 ceph-mon[36616]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:34 smithi175 ceph-mon[36886]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:36.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:36 smithi007 ceph-mon[32077]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:36 smithi189 ceph-mon[36616]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:36 smithi175 ceph-mon[36886]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.602+0000 7fedcea00700 1 -- 172.21.15.7:0/2465001063 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fedd80628d0 con 0x7fedbc0a2c20 2024-06-13T06:46:38.602 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-06-13T06:46:38.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 msgr2=0x7fedbc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:38.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 0x7fedbc0a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedc00098c0 tx=0x7fedc0019040 comp rx=0 tx=0).stop 2024-06-13T06:46:38.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 msgr2=0x7fedd819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedc800e9c0 tx=0x7fedc800ecd0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 shutdown_connections 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fedbc0a2c20 0x7fedbc0a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedd810ea00 0x7fedd81a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedd8074ef0 0x7fedd819ecf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 --2- 172.21.15.7:0/2465001063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedd80764c0 0x7fedd819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.605+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 >> 172.21.15.7:0/2465001063 conn(0x7fedd80fd9c0 msgr2=0x7fedd8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.606+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 shutdown_connections 2024-06-13T06:46:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:46:38.606+0000 7fedd7200700 1 -- 172.21.15.7:0/2465001063 wait complete. 2024-06-13T06:46:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:38 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:38 smithi189 ceph-mon[36616]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:38 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:38 smithi189 ceph-mon[36616]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:38 smithi189 ceph-mon[36616]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:38.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:38 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:38 smithi175 ceph-mon[36886]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:38.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:38 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:38 smithi175 ceph-mon[36886]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:38.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:38 smithi175 ceph-mon[36886]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:38.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:38 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:38 smithi007 ceph-mon[32077]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:38 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:38 smithi007 ceph-mon[32077]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:38 smithi007 ceph-mon[32077]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-13T06:46:39.389 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.6.service 2024-06-13T06:46:39.394 INFO:tasks.cephadm:Deploying osd.7 on smithi175 with /dev/vg_nvme/lv_2... 2024-06-13T06:46:39.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:46:39.592 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T06:46:39.566+0000 7fef89800700 -1 osd.6 0 waiting for initial osdmap 2024-06-13T06:46:39.592 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T06:46:39.591+0000 7fef84600700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:46:39.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:39.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:39.640 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: osdmap e45: 7 total, 6 up, 7 in 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:40.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:40 smithi175 ceph-mon[36886]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:40.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:40 smithi175 ceph-mon[36886]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:46:40.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:40 smithi175 ceph-mon[36886]: osdmap e46: 7 total, 6 up, 7 in 2024-06-13T06:46:40.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:40.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:40.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:40 smithi189 ceph-mon[36616]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:40.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:40 smithi189 ceph-mon[36616]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:46:40.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:40 smithi189 ceph-mon[36616]: osdmap e46: 7 total, 6 up, 7 in 2024-06-13T06:46:40.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:40.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:40 smithi007 ceph-mon[32077]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:40 smithi007 ceph-mon[32077]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:46:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:40 smithi007 ceph-mon[32077]: osdmap e46: 7 total, 6 up, 7 in 2024-06-13T06:46:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.610 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:46:41.611 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:46:41.611 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-13T06:46:41.611 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-13T06:46:41.611 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011535 s, 909 MB/s 2024-06-13T06:46:41.611 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] boot 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:46:41.635 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:46:41.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] boot 2024-06-13T06:46:41.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:46:41.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: osd.6 [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] boot 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: osdmap e47: 7 total, 7 up, 7 in 2024-06-13T06:46:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-13T06:46:42.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-06-13T06:46:42.607 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:46:42.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:42 smithi175 ceph-mon[36886]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:42.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:42 smithi175 ceph-mon[36886]: osdmap e48: 7 total, 7 up, 7 in 2024-06-13T06:46:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:42 smithi007 ceph-mon[32077]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:42.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:42 smithi007 ceph-mon[32077]: osdmap e48: 7 total, 7 up, 7 in 2024-06-13T06:46:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:42 smithi189 ceph-mon[36616]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-06-13T06:46:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:42 smithi189 ceph-mon[36616]: osdmap e48: 7 total, 7 up, 7 in 2024-06-13T06:46:44.180 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 -- 172.21.15.175:0/520118912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d41024c0 msgr2=0x7f63d41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 --2- 172.21.15.175:0/520118912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d41024c0 0x7f63d41028e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f63c40099c0 tx=0x7f63c4009cd0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 -- 172.21.15.175:0/520118912 shutdown_connections 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 --2- 172.21.15.175:0/520118912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d4104060 0x7f63d410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 --2- 172.21.15.175:0/520118912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d41024c0 0x7f63d41028e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 --2- 172.21.15.175:0/520118912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 0x7f63d4103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 -- 172.21.15.175:0/520118912 >> 172.21.15.175:0/520118912 conn(0x7f63d40fda60 msgr2=0x7f63d40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 -- 172.21.15.175:0/520118912 shutdown_connections 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.179+0000 7f63dbaab700 1 -- 172.21.15.175:0/520118912 wait complete. 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 Processor -- start 2024-06-13T06:46:44.181 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 -- start start 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41024c0 0x7f63d4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 0x7f63d4198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410e110 con 0x7f63d41024c0 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410df90 con 0x7f63d4104060 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63dbaab700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410e290 con 0x7f63d41036c0 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63da048700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:44.182 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63da048700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59012/0 (socket says 172.21.15.175:59012) 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63da048700 1 -- 172.21.15.175:0/1080217839 learned_addr learned my addr 172.21.15.175:0/1080217839 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63d9046700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 0x7f63d4198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63da048700 1 -- 172.21.15.175:0/1080217839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 msgr2=0x7f63d4198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.180+0000 7f63da048700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 0x7f63d4198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63da048700 1 -- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41024c0 msgr2=0x7f63d4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63da048700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41024c0 0x7f63d4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63da048700 1 -- 172.21.15.175:0/1080217839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d41a03c0 con 0x7f63d4104060 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63da048700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00b970 tx=0x7f63cc00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc019070 con 0x7f63d4104060 2024-06-13T06:46:44.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d41a06b0 con 0x7f63d4104060 2024-06-13T06:46:44.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63cc00ce10 con 0x7f63d4104060 2024-06-13T06:46:44.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0073d0 con 0x7f63d4104060 2024-06-13T06:46:44.184 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.181+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d41a0c70 con 0x7f63d4104060 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.182+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d4069180 con 0x7f63d4104060 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.184+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f63cc01c070 con 0x7f63d4104060 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.184+0000 7f63caffd700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 0x7f63c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.184+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f63cc0cdb40 con 0x7f63d4104060 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.185+0000 7f63d9847700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 0x7f63c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:46:44.185 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.186+0000 7f63d9847700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 0x7f63c00a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63c4003890 tx=0x7f63c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:46:44.187 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.188+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63cc09b7b0 con 0x7f63d4104060 2024-06-13T06:46:44.391 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:46:44.391+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f63d40628d0 con 0x7f63c00a2c20 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:44.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:44.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:44.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:45.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi007 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 5698M 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:45.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.006 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi007 2024-06-13T06:46:46.006 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.006 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.006 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.006 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 5698M 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:46.007 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi007 2024-06-13T06:46:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 5698M 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:46:46.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:46:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:46 smithi189 ceph-mon[36616]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:46 smithi175 ceph-mon[36886]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:47.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:46 smithi007 ceph-mon[32077]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/2791557001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]': finished 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:46:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:47 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/2476031530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/2791557001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]': finished 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:46:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:47 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/2476031530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/2791557001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]: dispatch 2024-06-13T06:46:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41e3e868-f9cf-461e-84ce-4321ccdd9fd8"}]': finished 2024-06-13T06:46:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: osdmap e49: 8 total, 7 up, 8 in 2024-06-13T06:46:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:46:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/2476031530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:46:49.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:49 smithi007 ceph-mon[32077]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:49.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:49 smithi189 ceph-mon[36616]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:49.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:49 smithi175 ceph-mon[36886]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:51 smithi175 ceph-mon[36886]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:51.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:51 smithi007 ceph-mon[32077]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:51 smithi189 ceph-mon[36616]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:53 smithi175 ceph-mon[36886]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:53.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:53 smithi007 ceph-mon[32077]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:53 smithi189 ceph-mon[36616]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:55.332 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:55 smithi175 ceph-mon[36886]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:55.332 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:46:55.332 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:55.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:55 smithi007 ceph-mon[32077]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:55.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:46:55.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:55 smithi189 ceph-mon[36616]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-13T06:46:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:46:56.354 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:56 smithi175 ceph-mon[36886]: Deploying daemon osd.7 on smithi175 2024-06-13T06:46:56.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:56 smithi007 ceph-mon[32077]: Deploying daemon osd.7 on smithi175 2024-06-13T06:46:56.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:56 smithi189 ceph-mon[36616]: Deploying daemon osd.7 on smithi175 2024-06-13T06:46:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:57 smithi175 ceph-mon[36886]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:57.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:57 smithi007 ceph-mon[32077]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:57 smithi189 ceph-mon[36616]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:59.253 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:46:59 smithi175 ceph-mon[36886]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:59.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:46:59 smithi007 ceph-mon[32077]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:46:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:46:59 smithi189 ceph-mon[36616]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:00 smithi175 ceph-mon[36886]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:01.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:01.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:00 smithi007 ceph-mon[32077]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:00 smithi189 ceph-mon[36616]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:02 smithi175 ceph-mon[36886]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:03.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:02 smithi007 ceph-mon[32077]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:02 smithi189 ceph-mon[36616]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:03.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.783+0000 7f63caffd700 1 -- 172.21.15.175:0/1080217839 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f63d40628d0 con 0x7f63c00a2c20 2024-06-13T06:47:03.784 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 7 on host 'smithi175' 2024-06-13T06:47:03.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 msgr2=0x7f63c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:03.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 0x7f63c00a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63c4003890 tx=0x7f63c4000bc0 comp rx=0 tx=0).stop 2024-06-13T06:47:03.785 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 msgr2=0x7f63d419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00b970 tx=0x7f63cc00bc80 comp rx=0 tx=0).stop 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 shutdown_connections 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f63c00a2c20 0x7f63c00a50e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63d4104060 0x7f63d419fd20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41024c0 0x7f63d4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 --2- 172.21.15.175:0/1080217839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63d41036c0 0x7f63d4198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 >> 172.21.15.175:0/1080217839 conn(0x7f63d40fda60 msgr2=0x7f63d4106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 shutdown_connections 2024-06-13T06:47:03.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:47:03.786+0000 7f63dbaab700 1 -- 172.21.15.175:0/1080217839 wait complete. 2024-06-13T06:47:04.102 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.102 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.103 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:04.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:04.433 DEBUG:teuthology.orchestra.run.smithi175:osd.7> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.7.service 2024-06-13T06:47:04.435 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-06-13T06:47:04.435 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-13T06:47:04.680 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:47:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:47:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:47:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:05.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:04 smithi189 ceph-mon[36616]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:05.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:04 smithi175 ceph-mon[36886]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: from='osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: osdmap e50: 8 total, 7 up, 8 in 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:04 smithi007 ceph-mon[32077]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:47:05.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T06:47:05.392+0000 7fa2e6d63700 -1 osd.7 0 waiting for initial osdmap 2024-06-13T06:47:05.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T06:47:05.401+0000 7fa2de34d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:47:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:05 smithi189 ceph-mon[36616]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:47:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:05 smithi189 ceph-mon[36616]: osdmap e51: 8 total, 7 up, 8 in 2024-06-13T06:47:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:05 smithi189 ceph-mon[36616]: from='osd.7 ' entity='osd.7' 2024-06-13T06:47:06.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-mon[36886]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:47:06.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-mon[36886]: osdmap e51: 8 total, 7 up, 8 in 2024-06-13T06:47:06.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:05 smithi175 ceph-mon[36886]: from='osd.7 ' entity='osd.7' 2024-06-13T06:47:06.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:05 smithi007 ceph-mon[32077]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:47:06.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:05 smithi007 ceph-mon[32077]: osdmap e51: 8 total, 7 up, 8 in 2024-06-13T06:47:06.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:06.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:05 smithi007 ceph-mon[32077]: from='osd.7 ' entity='osd.7' 2024-06-13T06:47:06.667 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-13T06:47:06.667 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-13T06:47:06.667 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-06-13T06:47:06.668 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-06-13T06:47:06.668 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114139 s, 919 MB/s 2024-06-13T06:47:06.668 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-06-13T06:47:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:47:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:47:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] boot 2024-06-13T06:47:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:47:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:07.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:06 smithi175 ceph-mon[36886]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] boot 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:06 smithi189 ceph-mon[36616]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-13T06:47:07.373 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: pgmap v194: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: osd.7 [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] boot 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: osdmap e52: 8 total, 8 up, 8 in 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-13T06:47:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:06 smithi007 ceph-mon[32077]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-13T06:47:07.614 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:08.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:08.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:47:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:08.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:08.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:07 smithi007 ceph-mon[32077]: osdmap e53: 8 total, 8 up, 8 in 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:08.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:09.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 -- 172.21.15.189:0/2440183838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 msgr2=0x7f7228103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:09.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 --2- 172.21.15.189:0/2440183838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228103b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7218009990 tx=0x7f7218009ca0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 -- 172.21.15.189:0/2440183838 shutdown_connections 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 --2- 172.21.15.189:0/2440183838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72281040c0 0x7f722810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 --2- 172.21.15.189:0/2440183838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7228102520 0x7f7228102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 --2- 172.21.15.189:0/2440183838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228103b80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 -- 172.21.15.189:0/2440183838 >> 172.21.15.189:0/2440183838 conn(0x7f72280fdaa0 msgr2=0x7f72280fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 -- 172.21.15.189:0/2440183838 shutdown_connections 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.246+0000 7f722da1e700 1 -- 172.21.15.189:0/2440183838 wait complete. 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 Processor -- start 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 -- start start 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7228102520 0x7f72281986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72281040c0 0x7f722819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:09.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f722810e1b0 con 0x7f7228102520 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f722810e030 con 0x7f72281040c0 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f722da1e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f722810e330 con 0x7f7228103720 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:43286/0 (socket says 172.21.15.189:43286) 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 -- 172.21.15.189:0/4192920708 learned_addr learned my addr 172.21.15.189:0/4192920708 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72277fe700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72281040c0 0x7f722819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 -- 172.21.15.189:0/4192920708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72281040c0 msgr2=0x7f722819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72281040c0 0x7f722819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 -- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7228102520 msgr2=0x7f72281986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7228102520 0x7f72281986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.247+0000 7f72267fc700 1 -- 172.21.15.189:0/4192920708 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72281a0340 con 0x7f7228103720 2024-06-13T06:47:09.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f72267fc700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72180057c0 tx=0x7f7218005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:09.250 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218027070 con 0x7f7228103720 2024-06-13T06:47:09.250 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72281a05d0 con 0x7f7228103720 2024-06-13T06:47:09.250 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72180041d0 con 0x7f7228103720 2024-06-13T06:47:09.250 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721800f460 con 0x7f7228103720 2024-06-13T06:47:09.250 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.248+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281a0aa0 con 0x7f7228103720 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.249+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7228069180 con 0x7f7228103720 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.250+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f721802a070 con 0x7f7228103720 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.251+0000 7f722ca1c700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 0x7f72100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.251+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f72180cfcd0 con 0x7f7228103720 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.251+0000 7f7226ffd700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 0x7f72100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:09.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.252+0000 7f7226ffd700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 0x7f72100a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7228103580 tx=0x7f721c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:09.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.254+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7218031020 con 0x7f7228103720 2024-06-13T06:47:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:09 smithi189 ceph-mon[36616]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:09 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi175 2024-06-13T06:47:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:09 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-13T06:47:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:09 smithi189 ceph-mon[36616]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:47:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:09 smithi175 ceph-mon[36886]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:09 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi175 2024-06-13T06:47:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:09 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-13T06:47:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:09 smithi175 ceph-mon[36886]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:47:09.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:09 smithi007 ceph-mon[32077]: pgmap v197: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:09.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:09 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi175 2024-06-13T06:47:09.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:09 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 5696M 2024-06-13T06:47:09.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:09 smithi007 ceph-mon[32077]: osdmap e54: 8 total, 8 up, 8 in 2024-06-13T06:47:09.459 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:09.460+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f72280628d0 con 0x7f72100a2c20 2024-06-13T06:47:10.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:10 smithi189 ceph-mon[36616]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:47:10.319 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:10.320 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:10.320 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:10.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:10 smithi175 ceph-mon[36886]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:47:10.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:10.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:10.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:10 smithi007 ceph-mon[32077]: osdmap e55: 8 total, 8 up, 8 in 2024-06-13T06:47:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:11.308 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:11 smithi189 ceph-mon[36616]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:11.308 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:11 smithi189 ceph-mon[36616]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:11.308 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:11 smithi189 ceph-mon[36616]: osdmap e56: 8 total, 8 up, 8 in 2024-06-13T06:47:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:11 smithi175 ceph-mon[36886]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:11 smithi175 ceph-mon[36886]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:11 smithi175 ceph-mon[36886]: osdmap e56: 8 total, 8 up, 8 in 2024-06-13T06:47:11.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:11 smithi007 ceph-mon[32077]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:11.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:11 smithi007 ceph-mon[32077]: pgmap v200: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:11.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:11 smithi007 ceph-mon[32077]: osdmap e56: 8 total, 8 up, 8 in 2024-06-13T06:47:12.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:12 smithi189 ceph-mon[36616]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-13T06:47:12.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:12 smithi189 ceph-mon[36616]: Cluster is now healthy 2024-06-13T06:47:12.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:12 smithi175 ceph-mon[36886]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-13T06:47:12.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:12 smithi175 ceph-mon[36886]: Cluster is now healthy 2024-06-13T06:47:12.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:12 smithi007 ceph-mon[32077]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-13T06:47:12.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:12 smithi007 ceph-mon[32077]: Cluster is now healthy 2024-06-13T06:47:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/319974830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]': finished 2024-06-13T06:47:13.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:47:13.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:13.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:13 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/198361618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/319974830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]': finished 2024-06-13T06:47:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:47:13.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:13.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:13 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/198361618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/319974830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]: dispatch 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9407bcc-7a87-425c-a20b-dd361583d662"}]': finished 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: osdmap e57: 9 total, 8 up, 9 in 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:13.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:13 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/198361618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:15 smithi189 ceph-mon[36616]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:15.369 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:15 smithi175 ceph-mon[36886]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:15.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:15 smithi007 ceph-mon[32077]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:17 smithi189 ceph-mon[36616]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:17 smithi175 ceph-mon[36886]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:17.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:17 smithi007 ceph-mon[32077]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:19 smithi175 ceph-mon[36886]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:19.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:19 smithi007 ceph-mon[32077]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:19.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:19 smithi189 ceph-mon[36616]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:20.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:47:20.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:20 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:20.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:47:20.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:20 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-13T06:47:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:20 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:21 smithi189 ceph-mon[36616]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:21 smithi189 ceph-mon[36616]: Deploying daemon osd.8 on smithi189 2024-06-13T06:47:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:21 smithi175 ceph-mon[36886]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:21 smithi175 ceph-mon[36886]: Deploying daemon osd.8 on smithi189 2024-06-13T06:47:21.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:21 smithi007 ceph-mon[32077]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:21.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:21 smithi007 ceph-mon[32077]: Deploying daemon osd.8 on smithi189 2024-06-13T06:47:23.274 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:23 smithi189 ceph-mon[36616]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:23 smithi175 ceph-mon[36886]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:23 smithi007 ceph-mon[32077]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:25.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:25 smithi189 ceph-mon[36616]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:25.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:25 smithi175 ceph-mon[36886]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:25.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:25 smithi007 ceph-mon[32077]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:26.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:26 smithi007 ceph-mon[32077]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:26.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:26 smithi189 ceph-mon[36616]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:27.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:26 smithi175 ceph-mon[36886]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:28.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:28 smithi189 ceph-mon[36616]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:28.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:28 smithi007 ceph-mon[32077]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:28 smithi175 ceph-mon[36886]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:29.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.749+0000 7f722ca1c700 1 -- 172.21.15.189:0/4192920708 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72280628d0 con 0x7f72100a2c20 2024-06-13T06:47:29.748 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-06-13T06:47:29.750 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 msgr2=0x7f72100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 0x7f72100a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7228103580 tx=0x7f721c005d20 comp rx=0 tx=0).stop 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 msgr2=0x7f7228198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f72180057c0 tx=0x7f7218005f10 comp rx=0 tx=0).stop 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 shutdown_connections 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72100a2c20 0x7f72100a50e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72281040c0 0x7f722819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:29.751 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7228102520 0x7f72281986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 --2- 172.21.15.189:0/4192920708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228103720 0x7f7228198c20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 >> 172.21.15.189:0/4192920708 conn(0x7f72280fdaa0 msgr2=0x7f72281069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 shutdown_connections 2024-06-13T06:47:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:47:29.751+0000 7f722da1e700 1 -- 172.21.15.189:0/4192920708 wait complete. 2024-06-13T06:47:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-13T06:47:30.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:30.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:47:30.452 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.8.service 2024-06-13T06:47:30.454 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-06-13T06:47:30.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:47:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:31.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T06:47:30.925+0000 7fd3dd8b8700 -1 osd.8 0 waiting for initial osdmap 2024-06-13T06:47:31.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T06:47:30.936+0000 7fd3d96ab700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:31.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:30 smithi189 ceph-mon[36616]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:31.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:30 smithi007 ceph-mon[32077]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: from='osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: osdmap e58: 9 total, 8 up, 9 in 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:30 smithi175 ceph-mon[36886]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:47:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:31 smithi007 ceph-mon[32077]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:47:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:31 smithi007 ceph-mon[32077]: osdmap e59: 9 total, 8 up, 9 in 2024-06-13T06:47:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:31 smithi189 ceph-mon[36616]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:47:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:31 smithi189 ceph-mon[36616]: osdmap e59: 9 total, 8 up, 9 in 2024-06-13T06:47:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:31 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:31 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:32.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:31 smithi175 ceph-mon[36886]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:47:32.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:31 smithi175 ceph-mon[36886]: osdmap e59: 9 total, 8 up, 9 in 2024-06-13T06:47:32.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:32.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.053 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] boot 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.054 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:32 smithi189 ceph-mon[36616]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] boot 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:47:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:32 smithi175 ceph-mon[36886]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: osd.8 [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] boot 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: osdmap e60: 9 total, 9 up, 9 in 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-13T06:47:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:32 smithi007 ceph-mon[32077]: osdmap e61: 9 total, 9 up, 9 in 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0419784 s, 250 MB/s 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-06-13T06:47:33.936 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-06-13T06:47:34.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi189 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 5696M 2024-06-13T06:47:34.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:34.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:34.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.829 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:34 smithi007 ceph-mon[32077]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi189 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 5696M 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:34 smithi189 ceph-mon[36616]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:47:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: pgmap v218: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi189 2024-06-13T06:47:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 5696M 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:47:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:34 smithi175 ceph-mon[36886]: osdmap e62: 9 total, 9 up, 9 in 2024-06-13T06:47:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:47:35.858 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:35 smithi007 ceph-mon[32077]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:47:36.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:35 smithi189 ceph-mon[36616]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:47:36.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:35 smithi175 ceph-mon[36886]: osdmap e63: 9 total, 9 up, 9 in 2024-06-13T06:47:36.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:36 smithi007 ceph-mon[32077]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:36.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:36 smithi007 ceph-mon[32077]: osdmap e64: 9 total, 9 up, 9 in 2024-06-13T06:47:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:36 smithi189 ceph-mon[36616]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:36 smithi189 ceph-mon[36616]: osdmap e64: 9 total, 9 up, 9 in 2024-06-13T06:47:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:36 smithi175 ceph-mon[36886]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:36 smithi175 ceph-mon[36886]: osdmap e64: 9 total, 9 up, 9 in 2024-06-13T06:47:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 -- 172.21.15.7:0/2973257323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 msgr2=0x7f283410cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 --2- 172.21.15.7:0/2973257323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283410cb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282c00b0d0 tx=0x7f282c00b3e0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 -- 172.21.15.7:0/2973257323 shutdown_connections 2024-06-13T06:47:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 --2- 172.21.15.7:0/2973257323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283410cb40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 --2- 172.21.15.7:0/2973257323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 0x7f28341061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 --2- 172.21.15.7:0/2973257323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f2834104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 -- 172.21.15.7:0/2973257323 >> 172.21.15.7:0/2973257323 conn(0x7f28340ffc50 msgr2=0x7f28341020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:47:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 -- 172.21.15.7:0/2973257323 shutdown_connections 2024-06-13T06:47:37.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.502+0000 7f283ac00700 1 -- 172.21.15.7:0/2973257323 wait complete. 2024-06-13T06:47:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.503+0000 7f283ac00700 1 Processor -- start 2024-06-13T06:47:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.503+0000 7f283ac00700 1 -- start start 2024-06-13T06:47:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 0x7f28341a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2834110410 con 0x7f2834105dd0 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2834110290 con 0x7f2834104cb0 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f283ac00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2834110590 con 0x7f2834104310 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57258/0 (socket says 172.21.15.7:57258) 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 -- 172.21.15.7:0/3155328651 learned_addr learned my addr 172.21.15.7:0/3155328651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2838c00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 0x7f28341a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:37.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833400700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 -- 172.21.15.7:0/3155328651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 msgr2=0x7f283419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283419b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 -- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 msgr2=0x7f28341a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 0x7f28341a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.504+0000 7f2833e00700 1 -- 172.21.15.7:0/3155328651 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28341a2660 con 0x7f2834104310 2024-06-13T06:47:37.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f2833400700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f2833e00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282400e9a0 tx=0x7f282400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004220 con 0x7f2834104310 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28240043c0 con 0x7f2834104310 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28341a2950 con 0x7f2834104310 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824003df0 con 0x7f2834104310 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.505+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a2ee0 con 0x7f2834104310 2024-06-13T06:47:37.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.506+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2834069180 con 0x7f2834104310 2024-06-13T06:47:37.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.508+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2824043ae0 con 0x7f2834104310 2024-06-13T06:47:37.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.510+0000 7f2830c00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 0x7f28180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:47:37.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.510+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f28240ce8a0 con 0x7f2834104310 2024-06-13T06:47:37.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.510+0000 7f2833400700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 0x7f28180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:47:37.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.511+0000 7f2833400700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 0x7f28180a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2820009680 tx=0x7f2820009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:47:37.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.512+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f282409c110 con 0x7f2834104310 2024-06-13T06:47:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:47:37.752+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f28340628d0 con 0x7f28180a2ca0 2024-06-13T06:47:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:38.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:47:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:47:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:38 smithi189 ceph-mon[36616]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-13T06:47:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:38 smithi189 ceph-mon[36616]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:39.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:38 smithi007 ceph-mon[32077]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-13T06:47:39.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:38 smithi007 ceph-mon[32077]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:38 smithi175 ceph-mon[36886]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-13T06:47:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:38 smithi175 ceph-mon[36886]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:47:41.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:40 smithi007 ceph-mon[32077]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:47:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:40 smithi189 ceph-mon[36616]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:47:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:40 smithi175 ceph-mon[36886]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-13T06:47:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:42 smithi189 ceph-mon[36616]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-13T06:47:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]: dispatch 2024-06-13T06:47:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]': finished 2024-06-13T06:47:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:42 smithi189 ceph-mon[36616]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:47:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:47:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:42 smithi175 ceph-mon[36886]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-13T06:47:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]: dispatch 2024-06-13T06:47:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]': finished 2024-06-13T06:47:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:42 smithi175 ceph-mon[36886]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:47:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:47:43.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:42 smithi007 ceph-mon[32077]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-13T06:47:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]: dispatch 2024-06-13T06:47:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/519922289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b78e7f03-fad5-401e-a013-def76183c002"}]': finished 2024-06-13T06:47:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:42 smithi007 ceph-mon[32077]: osdmap e65: 10 total, 9 up, 10 in 2024-06-13T06:47:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:47:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:43 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1381596675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:43 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1381596675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:44.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:43 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1381596675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:47:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:44 smithi189 ceph-mon[36616]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:47:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:44 smithi175 ceph-mon[36886]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:47:45.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:44 smithi007 ceph-mon[32077]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-13T06:47:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:46 smithi189 ceph-mon[36616]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-13T06:47:47.115 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:46 smithi007 ceph-mon[32077]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-13T06:47:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:46 smithi175 ceph-mon[36886]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-13T06:47:48.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:48 smithi007 ceph-mon[32077]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:48 smithi189 ceph-mon[36616]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:48 smithi175 ceph-mon[36886]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:51.063 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:50 smithi007 ceph-mon[32077]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:50 smithi189 ceph-mon[36616]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:50 smithi175 ceph-mon[36886]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:52 smithi189 ceph-mon[36616]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:47:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:52 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:52 smithi189 ceph-mon[36616]: Deploying daemon osd.9 on smithi007 2024-06-13T06:47:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:52 smithi175 ceph-mon[36886]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:52 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:47:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:52 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:52 smithi175 ceph-mon[36886]: Deploying daemon osd.9 on smithi007 2024-06-13T06:47:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:52 smithi007 ceph-mon[32077]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:52 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-13T06:47:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:52 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:47:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:52 smithi007 ceph-mon[32077]: Deploying daemon osd.9 on smithi007 2024-06-13T06:47:54.961 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:54 smithi007 ceph-mon[32077]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:54 smithi189 ceph-mon[36616]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:55.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:54 smithi175 ceph-mon[36886]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:57.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:56 smithi189 ceph-mon[36616]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:57.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:56 smithi175 ceph-mon[36886]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:56 smithi007 ceph-mon[32077]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:47:58 smithi189 ceph-mon[36616]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:47:58 smithi175 ceph-mon[36886]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:47:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:47:58 smithi007 ceph-mon[32077]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:00 smithi189 ceph-mon[36616]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:00 smithi175 ceph-mon[36886]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:01.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:00 smithi007 ceph-mon[32077]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:03.107 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:02 smithi007 ceph-mon[32077]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:02 smithi175 ceph-mon[36886]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:02 smithi189 ceph-mon[36616]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:05.452 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-06-13T06:48:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.451+0000 7f2830c00700 1 -- 172.21.15.7:0/3155328651 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f28340628d0 con 0x7f28180a2ca0 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.454+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 msgr2=0x7f28180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.454+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 0x7f28180a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2820009680 tx=0x7f2820009260 comp rx=0 tx=0).stop 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 msgr2=0x7f283419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282400e9a0 tx=0x7f282400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 shutdown_connections 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834105dd0 0x7f28341a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834104310 0x7f283419aaf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f28180a2ca0 0x7f28180a5160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 --2- 172.21.15.7:0/3155328651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2834104cb0 0x7f283419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 >> 172.21.15.7:0/3155328651 conn(0x7f28340ffc50 msgr2=0x7f2834108b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 shutdown_connections 2024-06-13T06:48:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:48:05.455+0000 7f283ac00700 1 -- 172.21.15.7:0/3155328651 wait complete. 2024-06-13T06:48:05.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:05 smithi007 ceph-mon[32077]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:05 smithi007 ceph-mon[32077]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:05 smithi007 ceph-mon[32077]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:05 smithi189 ceph-mon[36616]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:05 smithi189 ceph-mon[36616]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:05 smithi189 ceph-mon[36616]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:05 smithi175 ceph-mon[36886]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:05 smithi175 ceph-mon[36886]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:05 smithi175 ceph-mon[36886]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-13T06:48:06.271 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.9.service 2024-06-13T06:48:06.279 INFO:tasks.cephadm:Deploying osd.10 on smithi175 with /dev/vg_nvme/lv_1... 2024-06-13T06:48:06.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:48:06.528 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:06.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-13T06:48:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: osdmap e66: 10 total, 9 up, 10 in 2024-06-13T06:48:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-06-13T06:48:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:06.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:07.296 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T06:48:07.093+0000 7f9a08600700 -1 osd.9 0 waiting for initial osdmap 2024-06-13T06:48:07.296 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T06:48:07.135+0000 7f99fda00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:48:07.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:07.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:48:07.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: osdmap e67: 10 total, 9 up, 10 in 2024-06-13T06:48:07.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:07 smithi007 ceph-mon[32077]: from='osd.9 ' entity='osd.9' 2024-06-13T06:48:07.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:07.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:48:07.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: osdmap e67: 10 total, 9 up, 10 in 2024-06-13T06:48:07.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:07 smithi175 ceph-mon[36886]: from='osd.9 ' entity='osd.9' 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: osdmap e67: 10 total, 9 up, 10 in 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:07 smithi189 ceph-mon[36616]: from='osd.9 ' entity='osd.9' 2024-06-13T06:48:08.534 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:48:08.534 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:48:08.534 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-06-13T06:48:08.534 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-06-13T06:48:08.534 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132713 s, 790 MB/s 2024-06-13T06:48:08.535 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 2024-06-13T06:48:08.535 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] boot 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:48:08.650 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:08 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] boot 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:48:08.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:08 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: osd.9 [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] boot 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: osdmap e68: 10 total, 10 up, 10 in 2024-06-13T06:48:08.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:08 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-13T06:48:09.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-06-13T06:48:09.526 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T06:48:09.595 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:09 smithi175 ceph-mon[36886]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:09 smithi189 ceph-mon[36616]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:09.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:09 smithi007 ceph-mon[32077]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-13T06:48:10.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:10 smithi175 ceph-mon[36886]: osdmap e69: 10 total, 10 up, 10 in 2024-06-13T06:48:10.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:10 smithi175 ceph-mon[36886]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:10.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:10 smithi007 ceph-mon[32077]: osdmap e69: 10 total, 10 up, 10 in 2024-06-13T06:48:10.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:10 smithi007 ceph-mon[32077]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:10 smithi189 ceph-mon[36616]: osdmap e69: 10 total, 10 up, 10 in 2024-06-13T06:48:10.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:10 smithi189 ceph-mon[36616]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 -- 172.21.15.175:0/903872415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01013f0 msgr2=0x7f09e0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 --2- 172.21.15.175:0/903872415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01013f0 0x7f09e0101810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09d00099c0 tx=0x7f09d0009cd0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 -- 172.21.15.175:0/903872415 shutdown_connections 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 --2- 172.21.15.175:0/903872415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e0102f90 0x7f09e0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 --2- 172.21.15.175:0/903872415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01013f0 0x7f09e0101810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 --2- 172.21.15.175:0/903872415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01025f0 0x7f09e0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 -- 172.21.15.175:0/903872415 >> 172.21.15.175:0/903872415 conn(0x7f09e00fc990 msgr2=0x7f09e00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 -- 172.21.15.175:0/903872415 shutdown_connections 2024-06-13T06:48:11.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.220+0000 7f09e6320700 1 -- 172.21.15.175:0/903872415 wait complete. 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 Processor -- start 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 -- start start 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 0x7f09e01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e0102f90 0x7f09e019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010d0b0 con 0x7f09e0102f90 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010cf30 con 0x7f09e01025f0 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.221+0000 7f09e6320700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010d230 con 0x7f09e01013f0 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:60076/0 (socket says 172.21.15.175:60076) 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 -- 172.21.15.175:0/54112197 learned_addr learned my addr 172.21.15.175:0/54112197 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09dffff700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 0x7f09e01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:11.223 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 -- 172.21.15.175:0/54112197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 msgr2=0x7f09e01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 0x7f09e01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 -- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e0102f90 msgr2=0x7f09e019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e0102f90 0x7f09e019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 -- 172.21.15.175:0/54112197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e01a0520 con 0x7f09e01025f0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09dffff700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 0x7f09e01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09df7fe700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b530 tx=0x7f09d400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4015020 con 0x7f09e01025f0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4011d10 con 0x7f09e01025f0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01a0810 con 0x7f09e01025f0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400f2e0 con 0x7f09e01025f0 2024-06-13T06:48:11.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.222+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01a0dd0 con 0x7f09e01025f0 2024-06-13T06:48:11.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.223+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0069180 con 0x7f09e01025f0 2024-06-13T06:48:11.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.224+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f09d4018020 con 0x7f09e01025f0 2024-06-13T06:48:11.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.225+0000 7f09dd7fa700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 0x7f09c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:11.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.225+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f09d40ce500 con 0x7f09e01025f0 2024-06-13T06:48:11.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.226+0000 7f09dffff700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 0x7f09c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:11.226 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.227+0000 7f09dffff700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 0x7f09c80a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000c00 tx=0x7f09d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:11.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.229+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09d401f020 con 0x7f09e01025f0 2024-06-13T06:48:11.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:11.433+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f09e006a8b0 con 0x7f09c80a2c20 2024-06-13T06:48:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:11 smithi007 ceph-mon[32077]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:11 smithi189 ceph-mon[36616]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:12.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:12.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:12.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:11 smithi175 ceph-mon[36886]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:12.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:12.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi007 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi007 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi007 2024-06-13T06:48:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 4273M 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:12.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:12.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:12.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/1978460587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]': finished 2024-06-13T06:48:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:48:14.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:15.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:15.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/1978460587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:15.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:15.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]': finished 2024-06-13T06:48:15.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:48:15.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/1978460587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]: dispatch 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f61b6ecf-df12-4a81-93f4-b17aac68e6f4"}]': finished 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: osdmap e70: 11 total, 10 up, 11 in 2024-06-13T06:48:15.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:15.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:15 smithi007 ceph-mon[32077]: from='client.? 172.21.15.175:0/2662397342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:15 smithi189 ceph-mon[36616]: from='client.? 172.21.15.175:0/2662397342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:15 smithi175 ceph-mon[36886]: from='client.? 172.21.15.175:0/2662397342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:16.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:16 smithi007 ceph-mon[32077]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:17.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:16 smithi189 ceph-mon[36616]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:17.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:16 smithi175 ceph-mon[36886]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:18.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:18 smithi007 ceph-mon[32077]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:19.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:18 smithi175 ceph-mon[36886]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:19.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:18 smithi189 ceph-mon[36616]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:20.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:20 smithi007 ceph-mon[32077]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:20.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:20 smithi175 ceph-mon[36886]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:21.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:20 smithi189 ceph-mon[36616]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:22 smithi175 ceph-mon[36886]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:48:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:22 smithi175 ceph-mon[36886]: Deploying daemon osd.10 on smithi175 2024-06-13T06:48:22.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:22 smithi007 ceph-mon[32077]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:22.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:48:22.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:22.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:22 smithi007 ceph-mon[32077]: Deploying daemon osd.10 on smithi175 2024-06-13T06:48:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:22 smithi189 ceph-mon[36616]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-13T06:48:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:22 smithi189 ceph-mon[36616]: Deploying daemon osd.10 on smithi175 2024-06-13T06:48:24.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:24 smithi007 ceph-mon[32077]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:24.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:24 smithi175 ceph-mon[36886]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:25.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:24 smithi189 ceph-mon[36616]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:26.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:26 smithi007 ceph-mon[32077]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:27.039 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:26 smithi175 ceph-mon[36886]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:26 smithi189 ceph-mon[36616]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:28 smithi189 ceph-mon[36616]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:28 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:28 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:28 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:28 smithi175 ceph-mon[36886]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:28 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:28 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:28 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:28 smithi007 ceph-mon[32077]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:28 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:30.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:30 smithi175 ceph-mon[36886]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:30.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:30 smithi007 ceph-mon[32077]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:30 smithi189 ceph-mon[36616]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:31.317 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.317+0000 7f09dd7fa700 1 -- 172.21.15.175:0/54112197 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f09e006a8b0 con 0x7f09c80a2c20 2024-06-13T06:48:31.318 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 10 on host 'smithi175' 2024-06-13T06:48:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 msgr2=0x7f09c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:31.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 0x7f09c80a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000c00 tx=0x7f09d0005fd0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 msgr2=0x7f09e0198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b530 tx=0x7f09d400b8f0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 shutdown_connections 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e0102f90 0x7f09e019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09e01013f0 0x7f09e01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09c80a2c20 0x7f09c80a50e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 --2- 172.21.15.175:0/54112197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e01025f0 0x7f09e0198e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 >> 172.21.15.175:0/54112197 conn(0x7f09e00fc990 msgr2=0x7f09e006a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 shutdown_connections 2024-06-13T06:48:31.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-06-13T06:48:31.320+0000 7f09e6320700 1 -- 172.21.15.175:0/54112197 wait complete. 2024-06-13T06:48:31.979 DEBUG:teuthology.orchestra.run.smithi175:osd.10> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.10.service 2024-06-13T06:48:31.981 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-06-13T06:48:31.981 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-13T06:48:32.231 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:48:32.562 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.562 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.562 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.563 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.563 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.563 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:33 smithi189 ceph-mon[36616]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-mon[36886]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: from='osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: osdmap e71: 11 total, 10 up, 11 in 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:33 smithi007 ceph-mon[32077]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-06-13T06:48:34.067 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T06:48:33.735+0000 7f542b37b700 -1 osd.10 0 waiting for initial osdmap 2024-06-13T06:48:34.068 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 06:48:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T06:48:33.748+0000 7f5423166700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:48:34.437 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-13T06:48:34.437 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-13T06:48:34.437 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-06-13T06:48:34.438 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-06-13T06:48:34.438 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115962 s, 904 MB/s 2024-06-13T06:48:34.438 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-06-13T06:48:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:34 smithi189 ceph-mon[36616]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:48:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:34 smithi189 ceph-mon[36616]: osdmap e72: 11 total, 10 up, 11 in 2024-06-13T06:48:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:34 smithi189 ceph-mon[36616]: from='osd.10 ' entity='osd.10' 2024-06-13T06:48:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:34 smithi175 ceph-mon[36886]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:48:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:34 smithi175 ceph-mon[36886]: osdmap e72: 11 total, 10 up, 11 in 2024-06-13T06:48:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:34 smithi175 ceph-mon[36886]: from='osd.10 ' entity='osd.10' 2024-06-13T06:48:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:34 smithi007 ceph-mon[32077]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-06-13T06:48:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:34 smithi007 ceph-mon[32077]: osdmap e72: 11 total, 10 up, 11 in 2024-06-13T06:48:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:34 smithi007 ceph-mon[32077]: from='osd.10 ' entity='osd.10' 2024-06-13T06:48:35.160 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-06-13T06:48:35.401 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] boot 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:48:35.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.525 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:35 smithi189 ceph-mon[36616]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:48:35.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:48:35.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:48:35.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:35.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.543 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] boot 2024-06-13T06:48:35.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:48:35.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:35 smithi175 ceph-mon[36886]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:48:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:48:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:48:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-13T06:48:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: osd.10 [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] boot 2024-06-13T06:48:35.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: osdmap e73: 11 total, 11 up, 11 in 2024-06-13T06:48:35.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-13T06:48:35.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:35 smithi007 ceph-mon[32077]: osdmap e74: 11 total, 11 up, 11 in 2024-06-13T06:48:36.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:36.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi175 2024-06-13T06:48:36.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.793 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi175 2024-06-13T06:48:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi175 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 4272M 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: osdmap e75: 11 total, 11 up, 11 in 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:48:36.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 -- 172.21.15.189:0/1960413179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8102fa0 msgr2=0x7f4bb8109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/1960413179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8102fa0 0x7f4bb8109850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b0a0 tx=0x7f4bb000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 -- 172.21.15.189:0/1960413179 shutdown_connections 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/1960413179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8102fa0 0x7f4bb8109850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/1960413179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb8101400 0x7f4bb8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.016+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/1960413179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102600 0x7f4bb8102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.019 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- 172.21.15.189:0/1960413179 >> 172.21.15.189:0/1960413179 conn(0x7f4bb80fc9c0 msgr2=0x7f4bb80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- 172.21.15.189:0/1960413179 shutdown_connections 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- 172.21.15.189:0/1960413179 wait complete. 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 Processor -- start 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- start start 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb8102600 0x7f4bb8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102fa0 0x7f4bb819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810d0e0 con 0x7f4bb8102600 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810cf60 con 0x7f4bb8102fa0 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.017+0000 7f4bc03ae700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810d260 con 0x7f4bb8101400 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47244/0 (socket says 172.21.15.189:47244) 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 -- 172.21.15.189:0/2523743909 learned_addr learned my addr 172.21.15.189:0/2523743909 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:48:37.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe94b700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102fa0 0x7f4bb819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 -- 172.21.15.189:0/2523743909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102fa0 msgr2=0x7f4bb819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102fa0 0x7f4bb819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 -- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb8102600 msgr2=0x7f4bb8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb8102600 0x7f4bb8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 -- 172.21.15.189:0/2523743909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81a0500 con 0x7f4bb8101400 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bbe14a700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b530 tx=0x7f4bb400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:37.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4015020 con 0x7f4bb8101400 2024-06-13T06:48:37.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4011d10 con 0x7f4bb8101400 2024-06-13T06:48:37.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81a07f0 con 0x7f4bb8101400 2024-06-13T06:48:37.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400f2e0 con 0x7f4bb8101400 2024-06-13T06:48:37.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.018+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a0e00 con 0x7f4bb8101400 2024-06-13T06:48:37.022 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.019+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8069180 con 0x7f4bb8101400 2024-06-13T06:48:37.023 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.020+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4bb4018020 con 0x7f4bb8101400 2024-06-13T06:48:37.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.021+0000 7f4baf7fe700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 0x7f4ba40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:48:37.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.021+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f4bb40ce170 con 0x7f4bb8101400 2024-06-13T06:48:37.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.021+0000 7f4bbd949700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 0x7f4ba40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:48:37.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.022+0000 7f4bbd949700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 0x7f4ba40a51e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80097b0 tx=0x7f4ba8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:48:37.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.026+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4bb401f020 con 0x7f4bb8101400 2024-06-13T06:48:37.231 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:37.231+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4bb80febc0 con 0x7f4ba40a2d20 2024-06-13T06:48:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:37 smithi189 ceph-mon[36616]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:48:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:37 smithi189 ceph-mon[36616]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:38.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:38.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:37 smithi175 ceph-mon[36886]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:48:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:37 smithi175 ceph-mon[36886]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:38.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:37 smithi007 ceph-mon[32077]: osdmap e76: 11 total, 11 up, 11 in 2024-06-13T06:48:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:37 smithi007 ceph-mon[32077]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:48:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-13T06:48:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-13T06:48:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:39.068 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:38 smithi189 ceph-mon[36616]: pgmap v265: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:48:39.068 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:38 smithi189 ceph-mon[36616]: osdmap e77: 11 total, 11 up, 11 in 2024-06-13T06:48:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:38 smithi175 ceph-mon[36886]: pgmap v265: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:48:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:38 smithi175 ceph-mon[36886]: osdmap e77: 11 total, 11 up, 11 in 2024-06-13T06:48:39.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:38 smithi007 ceph-mon[32077]: pgmap v265: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-13T06:48:39.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:38 smithi007 ceph-mon[32077]: osdmap e77: 11 total, 11 up, 11 in 2024-06-13T06:48:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-06-13T06:48:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/3658630939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]': finished 2024-06-13T06:48:41.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:48:41.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:41.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:40 smithi189 ceph-mon[36616]: from='client.? 172.21.15.189:0/1202318626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/3658630939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]': finished 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:40 smithi175 ceph-mon[36886]: from='client.? 172.21.15.189:0/1202318626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/3658630939' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]: dispatch 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2053f9b6-4b88-48c3-8665-7501ea3b4b3b"}]': finished 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: osdmap e78: 12 total, 11 up, 12 in 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:40 smithi007 ceph-mon[32077]: from='client.? 172.21.15.189:0/1202318626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-13T06:48:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:42 smithi189 ceph-mon[36616]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-13T06:48:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:42 smithi175 ceph-mon[36886]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-13T06:48:43.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:42 smithi007 ceph-mon[32077]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-06-13T06:48:45.075 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:44 smithi189 ceph-mon[36616]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:44 smithi175 ceph-mon[36886]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:45.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:44 smithi007 ceph-mon[32077]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:46 smithi189 ceph-mon[36616]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:46 smithi175 ceph-mon[36886]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:47.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:46 smithi007 ceph-mon[32077]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:48:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:48:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:48.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-13T06:48:48.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:48:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:48 smithi189 ceph-mon[36616]: Deploying daemon osd.11 on smithi189 2024-06-13T06:48:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:48 smithi189 ceph-mon[36616]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:48 smithi175 ceph-mon[36886]: Deploying daemon osd.11 on smithi189 2024-06-13T06:48:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:48 smithi175 ceph-mon[36886]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:48 smithi007 ceph-mon[32077]: Deploying daemon osd.11 on smithi189 2024-06-13T06:48:49.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:48 smithi007 ceph-mon[32077]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:51.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:50 smithi189 ceph-mon[36616]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:50 smithi175 ceph-mon[36886]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:51.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:50 smithi007 ceph-mon[32077]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:53.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:52 smithi189 ceph-mon[36616]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:52 smithi175 ceph-mon[36886]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:53.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:52 smithi007 ceph-mon[32077]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:54.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:54 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:54.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:54 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:54.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:54.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:54 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:48:55.704 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:55 smithi189 ceph-mon[36616]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:55 smithi175 ceph-mon[36886]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:55.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:55 smithi007 ceph-mon[32077]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:57.117 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.118+0000 7f4baf7fe700 1 -- 172.21.15.189:0/2523743909 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4bb80febc0 con 0x7f4ba40a2d20 2024-06-13T06:48:57.118 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-06-13T06:48:57.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 msgr2=0x7f4ba40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:57.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 0x7f4ba40a51e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80097b0 tx=0x7f4ba8006d20 comp rx=0 tx=0).stop 2024-06-13T06:48:57.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 msgr2=0x7f4bb81988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:48:57.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b530 tx=0x7f4bb400b840 comp rx=0 tx=0).stop 2024-06-13T06:48:57.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 shutdown_connections 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4ba40a2d20 0x7f4ba40a51e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4bb8102fa0 0x7f4bb819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bb8101400 0x7f4bb81988a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 --2- 172.21.15.189:0/2523743909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb8102600 0x7f4bb8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 >> 172.21.15.189:0/2523743909 conn(0x7f4bb80fc9c0 msgr2=0x7f4bb80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 shutdown_connections 2024-06-13T06:48:57.121 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:48:57.121+0000 7f4bc03ae700 1 -- 172.21.15.189:0/2523743909 wait complete. 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:57 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.793 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.11.service 2024-06-13T06:48:57.795 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-13T06:48:57.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd stat -f json 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:57 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:57.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:57.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-13T06:48:57.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:57 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:58.277 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:48:58.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:58.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:48:58 smithi007 ceph-mon[32077]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:48:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:48:58.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:58.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:48:58 smithi189 ceph-mon[36616]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: from='osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: osdmap e79: 12 total, 11 up, 12 in 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:48:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:48:58 smithi175 ceph-mon[36886]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-06-13T06:48:59.752 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 06:48:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T06:48:59.580+0000 7faa82295700 -1 osd.11 0 waiting for initial osdmap 2024-06-13T06:48:59.752 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 06:48:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T06:48:59.595+0000 7faa7a080700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-13T06:49:00.305 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:00 smithi007 ceph-mon[32077]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:49:00.306 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:00 smithi007 ceph-mon[32077]: osdmap e80: 12 total, 11 up, 12 in 2024-06-13T06:49:00.306 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.306 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.306 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:00 smithi007 ceph-mon[32077]: from='osd.11 ' entity='osd.11' 2024-06-13T06:49:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:00 smithi189 ceph-mon[36616]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:49:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:00 smithi189 ceph-mon[36616]: osdmap e80: 12 total, 11 up, 12 in 2024-06-13T06:49:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:00 smithi189 ceph-mon[36616]: from='osd.11 ' entity='osd.11' 2024-06-13T06:49:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:00 smithi175 ceph-mon[36886]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-06-13T06:49:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:00 smithi175 ceph-mon[36886]: osdmap e80: 12 total, 11 up, 12 in 2024-06-13T06:49:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:00.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:00 smithi175 ceph-mon[36886]: from='osd.11 ' entity='osd.11' 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 -- 172.21.15.7:0/3219913060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4102f50 msgr2=0x7f3af4105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3219913060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4102f50 0x7f3af4105340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009990 tx=0x7f3ae4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 -- 172.21.15.7:0/3219913060 shutdown_connections 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3219913060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af41081b0 0x7f3af410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3219913060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4102f50 0x7f3af4105340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3219913060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 0x7f3af4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 -- 172.21.15.7:0/3219913060 >> 172.21.15.7:0/3219913060 conn(0x7f3af40fc960 msgr2=0x7f3af40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 -- 172.21.15.7:0/3219913060 shutdown_connections 2024-06-13T06:49:00.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.592+0000 7f3af9e00700 1 -- 172.21.15.7:0/3219913060 wait complete. 2024-06-13T06:49:00.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.593+0000 7f3af9e00700 1 Processor -- start 2024-06-13T06:49:00.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.593+0000 7f3af9e00700 1 -- start start 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 0x7f3af4100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af41081b0 0x7f3af4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410e120 con 0x7f3af4105880 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410dfa0 con 0x7f3af4102f50 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af9e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410e2a0 con 0x7f3af41081b0 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48670/0 (socket says 172.21.15.7:48670) 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af2a00700 1 -- 172.21.15.7:0/3356132222 learned_addr learned my addr 172.21.15.7:0/3356132222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.594+0000 7f3af2000700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 0x7f3af4100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:00.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af3400700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af41081b0 0x7f3af4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 -- 172.21.15.7:0/3356132222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af41081b0 msgr2=0x7f3af4101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af41081b0 0x7f3af4101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 -- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 msgr2=0x7f3af4100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 0x7f3af4100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 -- 172.21.15.7:0/3356132222 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4101cf0 con 0x7f3af4102f50 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af2a00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400b2e0 tx=0x7f3ae400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4026070 con 0x7f3af4102f50 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.595+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a2870 con 0x7f3af4102f50 2024-06-13T06:49:00.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.596+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae400f460 con 0x7f3af4102f50 2024-06-13T06:49:00.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.596+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4021670 con 0x7f3af4102f50 2024-06-13T06:49:00.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.596+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a2cc0 con 0x7f3af4102f50 2024-06-13T06:49:00.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.598+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4069180 con 0x7f3af4102f50 2024-06-13T06:49:00.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.600+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3ae4029070 con 0x7f3af4102f50 2024-06-13T06:49:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.601+0000 7f3af8a00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 0x7f3ad80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.601+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3ae40cfa30 con 0x7f3af4102f50 2024-06-13T06:49:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.601+0000 7f3af2000700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 0x7f3ad80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.602+0000 7f3af2000700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 0x7f3ad80a5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009680 tx=0x7f3ae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.605+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ae405d850 con 0x7f3af4102f50 2024-06-13T06:49:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.840+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3af40630a0 con 0x7f3af4102f50 2024-06-13T06:49:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.841+0000 7f3af8a00700 1 -- 172.21.15.7:0/3356132222 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f3ae4013070 con 0x7f3af4102f50 2024-06-13T06:49:00.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.843+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 msgr2=0x7f3ad80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.843+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 0x7f3ad80a5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009680 tx=0x7f3ae0009260 comp rx=0 tx=0).stop 2024-06-13T06:49:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.843+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 msgr2=0x7f3af41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.843+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400b2e0 tx=0x7f3ae400bbb0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 shutdown_connections 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af41081b0 0x7f3af4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ad80a2ca0 0x7f3ad80a5160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af4102f50 0x7f3af41028b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 --2- 172.21.15.7:0/3356132222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3af4105880 0x7f3af4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.844+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 >> 172.21.15.7:0/3356132222 conn(0x7f3af40fc960 msgr2=0x7f3af41039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.845+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 shutdown_connections 2024-06-13T06:49:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:00.845+0000 7f3af9e00700 1 -- 172.21.15.7:0/3356132222 wait complete. 2024-06-13T06:49:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: purged_snaps scrub starts 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: purged_snaps scrub ok 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] boot 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:01 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3356132222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: purged_snaps scrub starts 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: purged_snaps scrub ok 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] boot 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:01 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3356132222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: purged_snaps scrub starts 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: purged_snaps scrub ok 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: osd.11 [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] boot 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: osdmap e81: 12 total, 12 up, 12 in 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-13T06:49:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:01 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3356132222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-13T06:49:01.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1718261340,"num_in_osds":12,"osd_in_since":1718261320,"num_remapped_pgs":0} 2024-06-13T06:49:01.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:49:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: Detected new or changed devices on smithi189 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 4272M 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: Detected new or changed devices on smithi189 2024-06-13T06:49:02.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 4272M 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: Detected new or changed devices on smithi189 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 4272M 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:02.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:03.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:03 smithi007 ceph-mon[32077]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:03.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:03 smithi007 ceph-mon[32077]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:49:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:03 smithi189 ceph-mon[36616]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:03 smithi189 ceph-mon[36616]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:49:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:03 smithi175 ceph-mon[36886]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:03 smithi175 ceph-mon[36886]: osdmap e82: 12 total, 12 up, 12 in 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 -- 172.21.15.7:0/3836449059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 msgr2=0x7fd40010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 --2- 172.21.15.7:0/3836449059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40010aa90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009990 tx=0x7fd3f0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 -- 172.21.15.7:0/3836449059 shutdown_connections 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 --2- 172.21.15.7:0/3836449059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd400111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 --2- 172.21.15.7:0/3836449059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40010aa90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 --2- 172.21.15.7:0/3836449059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd400104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 -- 172.21.15.7:0/3836449059 >> 172.21.15.7:0/3836449059 conn(0x7fd4001001a0 msgr2=0x7fd4001025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 -- 172.21.15.7:0/3836449059 shutdown_connections 2024-06-13T06:49:04.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.409+0000 7fd405a00700 1 -- 172.21.15.7:0/3836449059 wait complete. 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.410+0000 7fd405a00700 1 Processor -- start 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 -- start start 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd40019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001149e0 con 0x7fd40010a6b0 2024-06-13T06:49:04.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd400114860 con 0x7fd4001046b0 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd405a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd400114b60 con 0x7fd400105070 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd3fea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd3ff400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd3f7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd40019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd3fea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44404/0 (socket says 172.21.15.7:44404) 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.411+0000 7fd3fea00700 1 -- 172.21.15.7:0/3812871379 learned_addr learned my addr 172.21.15.7:0/3812871379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 -- 172.21.15.7:0/3812871379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 msgr2=0x7fd40019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd40019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 -- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 msgr2=0x7fd40019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 -- 172.21.15.7:0/3812871379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40019fe10 con 0x7fd4001046b0 2024-06-13T06:49:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3f7e00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd40019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3ff400700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3fea00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000bae0 tx=0x7fd3f000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.412+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0026070 con 0x7fd4001046b0 2024-06-13T06:49:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.413+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4001a00a0 con 0x7fd4001046b0 2024-06-13T06:49:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.413+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f0003d70 con 0x7fd4001046b0 2024-06-13T06:49:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.413+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0021410 con 0x7fd4001046b0 2024-06-13T06:49:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.413+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4001a7980 con 0x7fd4001046b0 2024-06-13T06:49:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.414+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd400069180 con 0x7fd4001046b0 2024-06-13T06:49:04.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.416+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd3f0029070 con 0x7fd4001046b0 2024-06-13T06:49:04.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.417+0000 7fd3f7400700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 0x7fd3e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:04.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.418+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd3f00cfe80 con 0x7fd4001046b0 2024-06-13T06:49:04.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.418+0000 7fd3f7e00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 0x7fd3e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:04.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.418+0000 7fd3f7e00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 0x7fd3e40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd400109cb0 tx=0x7fd3e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.420+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3f0030080 con 0x7fd4001046b0 2024-06-13T06:49:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.653+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd40002dca0 con 0x7fd4001046b0 2024-06-13T06:49:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.655+0000 7fd3f7400700 1 -- 172.21.15.7:0/3812871379 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18399 (secure 0 0 0) 0x7fd3f0013070 con 0x7fd4001046b0 2024-06-13T06:49:04.654 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:04.655 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:49:02.186349+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:49:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 msgr2=0x7fd3e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 0x7fd3e40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd400109cb0 tx=0x7fd3e8009260 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 msgr2=0x7fd40019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000bae0 tx=0x7fd3f000bb10 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 shutdown_connections 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40010a6b0 0x7fd40019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd3e40a2c20 0x7fd3e40a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4001046b0 0x7fd40019ae50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 --2- 172.21.15.7:0/3812871379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd400105070 0x7fd40019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 >> 172.21.15.7:0/3812871379 conn(0x7fd4001001a0 msgr2=0x7fd400100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.658+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 shutdown_connections 2024-06-13T06:49:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:04.659+0000 7fd405a00700 1 -- 172.21.15.7:0/3812871379 wait complete. 2024-06-13T06:49:05.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:05 smithi007 ceph-mon[32077]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:05.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:05 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3812871379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:49:05.453 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:44:44.175120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-13T06:49:05.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-13T06:49:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:05 smithi189 ceph-mon[36616]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:05 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3812871379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:49:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:05 smithi175 ceph-mon[36886]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:05 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3812871379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:49:05.931 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:07.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:07 smithi007 ceph-mon[32077]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:07.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:07 smithi189 ceph-mon[36616]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:07 smithi175 ceph-mon[36886]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:08.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 -- 172.21.15.7:0/3342242481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c0746c0 msgr2=0x7f208c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 --2- 172.21.15.7:0/3342242481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c0746c0 0x7f208c074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f207c009990 tx=0x7f207c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 -- 172.21.15.7:0/3342242481 shutdown_connections 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 --2- 172.21.15.7:0/3342242481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c113660 0x7f208c115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 --2- 172.21.15.7:0/3342242481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c0746c0 0x7f208c074aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 --2- 172.21.15.7:0/3342242481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c074fe0 0x7f208c113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 -- 172.21.15.7:0/3342242481 >> 172.21.15.7:0/3342242481 conn(0x7f208c0795f0 msgr2=0x7f208c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 -- 172.21.15.7:0/3342242481 shutdown_connections 2024-06-13T06:49:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.272+0000 7f2093000700 1 -- 172.21.15.7:0/3342242481 wait complete. 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.273+0000 7f2093000700 1 Processor -- start 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 -- start start 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 0x7f208c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c074fe0 0x7f208c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 0x7f208c1a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f208c118dc0 con 0x7f208c113660 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f208c118c40 con 0x7f208c074fe0 2024-06-13T06:49:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2093000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f208c118f40 con 0x7f208c0746c0 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2091000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 0x7f208c1a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2091000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 0x7f208c1a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41452/0 (socket says 172.21.15.7:41452) 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f2091000700 1 -- 172.21.15.7:0/1476418486 learned_addr learned my addr 172.21.15.7:0/1476418486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f208be00700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 0x7f208c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.274+0000 7f208b400700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c074fe0 0x7f208c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 -- 172.21.15.7:0/1476418486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c074fe0 msgr2=0x7f208c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c074fe0 0x7f208c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 -- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 msgr2=0x7f208c1a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 0x7f208c1a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 -- 172.21.15.7:0/1476418486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c1a4280 con 0x7f208c0746c0 2024-06-13T06:49:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f208be00700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 0x7f208c19f270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f207c006010 tx=0x7f207c00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.275+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c027070 con 0x7f208c0746c0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.276+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c0056d0 con 0x7f208c0746c0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.276+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c1a4510 con 0x7f208c0746c0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.276+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c022d90 con 0x7f208c0746c0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.276+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c1abdf0 con 0x7f208c0746c0 2024-06-13T06:49:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.277+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208c075db0 con 0x7f208c0746c0 2024-06-13T06:49:08.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.279+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f207c02a070 con 0x7f208c0746c0 2024-06-13T06:49:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.280+0000 7f2088c00700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 0x7f20700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.280+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f207c0d02b0 con 0x7f208c0746c0 2024-06-13T06:49:08.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.280+0000 7f208b400700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 0x7f20700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.281+0000 7f208b400700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 0x7f20700a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f208c115580 tx=0x7f2078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.283+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f207c013070 con 0x7f208c0746c0 2024-06-13T06:49:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.511+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f208c1abfc0 con 0x7f208c0746c0 2024-06-13T06:49:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.512+0000 7f2088c00700 1 -- 172.21.15.7:0/1476418486 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f207c09d6a0 con 0x7f208c0746c0 2024-06-13T06:49:08.513 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-13T06:49:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 msgr2=0x7f20700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 0x7f20700a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f208c115580 tx=0x7f2078009260 comp rx=0 tx=0).stop 2024-06-13T06:49:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 msgr2=0x7f208c19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 0x7f208c19f270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f207c006010 tx=0x7f207c00bab0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 shutdown_connections 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f208c113660 0x7f208c1a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f208c0746c0 0x7f208c19f270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20700a2c20 0x7f20700a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 --2- 172.21.15.7:0/1476418486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f208c074fe0 0x7f208c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.516+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 >> 172.21.15.7:0/1476418486 conn(0x7f208c0795f0 msgr2=0x7f208c112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.517+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 shutdown_connections 2024-06-13T06:49:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:08.517+0000 7f2093000700 1 -- 172.21.15.7:0/1476418486 wait complete. 2024-06-13T06:49:09.364 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-06-13T06:49:09.364 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-06-13T06:49:09.364 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-06-13T06:49:09.364 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.b on smithi175 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.e on smithi175 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.h on smithi175 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.k on smithi175 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-06-13T06:49:09.365 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-06-13T06:49:09.365 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-06-13T06:49:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:09 smithi189 ceph-mon[36616]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:09 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1476418486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:49:09.614 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T06:49:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:09 smithi175 ceph-mon[36886]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:09 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1476418486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:49:09.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:09 smithi007 ceph-mon[32077]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:09 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1476418486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:49:11.472 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:11 smithi189 ceph-mon[36616]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:11.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 -- 172.21.15.189:0/3432630674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a41024f0 msgr2=0x7f64a4102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:11.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/3432630674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a41024f0 0x7f64a4102910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6494009990 tx=0x7f6494009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.587 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 -- 172.21.15.189:0/3432630674 shutdown_connections 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/3432630674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a4104090 0x7f64a410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/3432630674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a41024f0 0x7f64a4102910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/3432630674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41036f0 0x7f64a4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 -- 172.21.15.189:0/3432630674 >> 172.21.15.189:0/3432630674 conn(0x7f64a40fda90 msgr2=0x7f64a40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 -- 172.21.15.189:0/3432630674 shutdown_connections 2024-06-13T06:49:11.588 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 -- 172.21.15.189:0/3432630674 wait complete. 2024-06-13T06:49:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.588+0000 7f64a8ffa700 1 Processor -- start 2024-06-13T06:49:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 -- start start 2024-06-13T06:49:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 0x7f64a41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a41036f0 0x7f64a4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64a410e160 con 0x7f64a41036f0 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f64a410dfe0 con 0x7f64a41024f0 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a8ffa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64a410e2e0 con 0x7f64a4104090 2024-06-13T06:49:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54074/0 (socket says 172.21.15.189:54074) 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 -- 172.21.15.189:0/295296409 learned_addr learned my addr 172.21.15.189:0/295296409 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a259c700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 0x7f64a41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 -- 172.21.15.189:0/295296409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 msgr2=0x7f64a41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 0x7f64a41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 -- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a41036f0 msgr2=0x7f64a4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:11.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a41036f0 0x7f64a4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 -- 172.21.15.189:0/295296409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a41a0400 con 0x7f64a4104090 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a259c700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 0x7f64a41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.589+0000 7f64a2d9d700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b9a0 tx=0x7f649c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.590+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c019070 con 0x7f64a4104090 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.590+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649c014440 con 0x7f64a4104090 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.590+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a41a06f0 con 0x7f64a4104090 2024-06-13T06:49:11.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.590+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c0075d0 con 0x7f64a4104090 2024-06-13T06:49:11.594 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.590+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41a0c60 con 0x7f64a4104090 2024-06-13T06:49:11.594 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.591+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a4069180 con 0x7f64a4104090 2024-06-13T06:49:11.595 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.592+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f649c01c070 con 0x7f64a4104090 2024-06-13T06:49:11.595 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.592+0000 7f649bfff700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:11.595 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.592+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f649c0ce020 con 0x7f64a4104090 2024-06-13T06:49:11.596 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.593+0000 7f64a259c700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:11.596 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.594+0000 7f64a259c700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 0x7f64900a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64940039c0 tx=0x7f6494005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:11.596 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.597+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f649c09b4b0 con 0x7f64a4104090 2024-06-13T06:49:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:11 smithi175 ceph-mon[36886]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:11 smithi007 ceph-mon[32077]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:11.800 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.800+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7f64a4062ad0 con 0x7f64900a2c20 2024-06-13T06:49:11.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.817+0000 7f649bfff700 1 -- 172.21.15.189:0/295296409 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f64a4062ad0 con 0x7f64900a2c20 2024-06-13T06:49:11.816 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 msgr2=0x7f64900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 0x7f64900a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64940039c0 tx=0x7f6494005fb0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 msgr2=0x7f64a419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b9a0 tx=0x7f649c00bd60 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 shutdown_connections 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a4104090 0x7f64a419fd60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64a41024f0 0x7f64a41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 --2- 172.21.15.189:0/295296409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a41036f0 0x7f64a4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:11.819 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.819+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 >> 172.21.15.189:0/295296409 conn(0x7f64a40fda90 msgr2=0x7f64a4106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:11.820 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.820+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 shutdown_connections 2024-06-13T06:49:11.820 INFO:teuthology.orchestra.run.smithi189.stderr:2024-06-13T06:49:11.820+0000 7f64a8ffa700 1 -- 172.21.15.189:0/295296409 wait complete. 2024-06-13T06:49:12.514 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.a.service 2024-06-13T06:49:12.522 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.d.service 2024-06-13T06:49:12.567 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.g.service 2024-06-13T06:49:12.611 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.j.service 2024-06-13T06:49:12.658 DEBUG:teuthology.orchestra.run.smithi175:mds.b> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.b.service 2024-06-13T06:49:12.661 DEBUG:teuthology.orchestra.run.smithi175:mds.e> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.e.service 2024-06-13T06:49:12.704 DEBUG:teuthology.orchestra.run.smithi175:mds.h> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.h.service 2024-06-13T06:49:12.747 DEBUG:teuthology.orchestra.run.smithi175:mds.k> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.k.service 2024-06-13T06:49:12.790 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.c.service 2024-06-13T06:49:12.796 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.f.service 2024-06-13T06:49:12.839 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.i.service 2024-06-13T06:49:12.883 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.l.service 2024-06-13T06:49:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:12 smithi007 ceph-mon[32077]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:12 smithi007 ceph-mon[32077]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:12 smithi007 ceph-mon[32077]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-06-13T06:49:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:12.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:12 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:12.926 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-13T06:49:12.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:12 smithi189 ceph-mon[36616]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:12 smithi189 ceph-mon[36616]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:12 smithi189 ceph-mon[36616]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-06-13T06:49:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:12 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:12 smithi175 ceph-mon[36886]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:12 smithi175 ceph-mon[36886]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:49:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:12 smithi175 ceph-mon[36886]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-06-13T06:49:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:12 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.076 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:13 smithi007 ceph-mon[32077]: Deploying daemon mds.l on smithi189 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:13 smithi189 ceph-mon[36616]: Deploying daemon mds.l on smithi189 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:13 smithi175 ceph-mon[36886]: Deploying daemon mds.l on smithi189 2024-06-13T06:49:15.309 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:15 smithi007 ceph-mon[32077]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:15.552 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:15 smithi189 ceph-mon[36616]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:15.552 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 systemd[1]: Starting Ceph mds.l for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:15.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:15 smithi175 ceph-mon[36886]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:15.826 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 podman[66577]: 2024-06-13 06:49:15.455776825 +0000 UTC m=+0.016924522 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:15.826 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 podman[66577]: 2024-06-13 06:49:15.575124828 +0000 UTC m=+0.136272518 container create 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-13T06:49:15.826 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 podman[66577]: 2024-06-13 06:49:15.827973935 +0000 UTC m=+0.389121631 container init 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T06:49:15.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 -- 172.21.15.7:0/1407624197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 msgr2=0x7f885810f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:15.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 --2- 172.21.15.7:0/1407624197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885810f340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f885000b0a0 tx=0x7f885000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 -- 172.21.15.7:0/1407624197 shutdown_connections 2024-06-13T06:49:15.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 --2- 172.21.15.7:0/1407624197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885810f340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 --2- 172.21.15.7:0/1407624197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f8858101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 --2- 172.21.15.7:0/1407624197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 0x7f88581070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 -- 172.21.15.7:0/1407624197 >> 172.21.15.7:0/1407624197 conn(0x7f8858076640 msgr2=0x7f8858076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 -- 172.21.15.7:0/1407624197 shutdown_connections 2024-06-13T06:49:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.912+0000 7f885cc00700 1 -- 172.21.15.7:0/1407624197 wait complete. 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.913+0000 7f885cc00700 1 Processor -- start 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.913+0000 7f885cc00700 1 -- start start 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.913+0000 7f885cc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f88581a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.913+0000 7f885cc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 0x7f88581a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f885cc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f885cc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8858112790 con 0x7f8858107790 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f885cc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8858112610 con 0x7f88581018d0 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f885cc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8858112910 con 0x7f8858100f20 2024-06-13T06:49:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41372/0 (socket says 172.21.15.7:41372) 2024-06-13T06:49:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 -- 172.21.15.7:0/625930788 learned_addr learned my addr 172.21.15.7:0/625930788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857400700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f88581a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8856a00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 0x7f88581a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:15.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 -- 172.21.15.7:0/625930788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 msgr2=0x7f88581a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:15.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f88581a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 -- 172.21.15.7:0/625930788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 msgr2=0x7f88581a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 0x7f88581a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857e00700 1 -- 172.21.15.7:0/625930788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885819b3f0 con 0x7f8858107790 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.914+0000 7f8857400700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f88581a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f8857e00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f885000b070 tx=0x7f8850004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850021070 con 0x7f8858107790 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8850003c00 con 0x7f8858107790 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850004740 con 0x7f8858107790 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885819b680 con 0x7f8858107790 2024-06-13T06:49:15.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.915+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88581a79c0 con 0x7f8858107790 2024-06-13T06:49:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.917+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8858069d00 con 0x7f8858107790 2024-06-13T06:49:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.919+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8850007980 con 0x7f8858107790 2024-06-13T06:49:15.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.920+0000 7f884fe00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 0x7f883c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:15.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.920+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f885000f020 con 0x7f8858107790 2024-06-13T06:49:15.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.921+0000 7f8857400700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 0x7f883c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.921+0000 7f8857400700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 0x7f883c0a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8848011490 tx=0x7f8848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:15.925+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88500d6080 con 0x7f8858107790 2024-06-13T06:49:16.086 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 podman[66577]: 2024-06-13 06:49:15.830639428 +0000 UTC m=+0.391787116 container start 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-13T06:49:16.086 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 ceph-mds[66596]: -- 172.21.15.189:0/348880224 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e9cbcfa40 con 0x560e9cba2800 2024-06-13T06:49:16.086 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l[66592]: starting mds.l at 2024-06-13T06:49:16.086 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 bash[66577]: 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc 2024-06-13T06:49:16.086 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 06:49:15 smithi189 systemd[1]: Started Ceph mds.l for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:16.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.219+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f885819bfd0 con 0x7f8858107790 2024-06-13T06:49:16.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.220+0000 7f884fe00700 1 -- 172.21.15.7:0/625930788 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f885002a020 con 0x7f8858107790 2024-06-13T06:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-13T06:49:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 msgr2=0x7f883c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 0x7f883c0a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8848011490 tx=0x7f8848009260 comp rx=0 tx=0).stop 2024-06-13T06:49:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 msgr2=0x7f885819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f885000b070 tx=0x7f8850004c60 comp rx=0 tx=0).stop 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 shutdown_connections 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8858107790 0x7f885819ab10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8858100f20 0x7f88581a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f883c0a2ca0 0x7f883c0a5160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 --2- 172.21.15.7:0/625930788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88581018d0 0x7f88581a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 >> 172.21.15.7:0/625930788 conn(0x7f8858076640 msgr2=0x7f885810d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 shutdown_connections 2024-06-13T06:49:16.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:16.223+0000 7f885cc00700 1 -- 172.21.15.7:0/625930788 wait complete. 2024-06-13T06:49:16.225 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-06-13T06:49:16.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6832/1184948673,v1:172.21.15.189:6833/1184948673] up:boot 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: fsmap 1 up:standby 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: Deploying daemon mds.k on smithi175 2024-06-13T06:49:16.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:16 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/625930788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6832/1184948673,v1:172.21.15.189:6833/1184948673] up:boot 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: fsmap 1 up:standby 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: Deploying daemon mds.k on smithi175 2024-06-13T06:49:16.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:16 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/625930788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6832/1184948673,v1:172.21.15.189:6833/1184948673] up:boot 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: fsmap 1 up:standby 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: Deploying daemon mds.k on smithi175 2024-06-13T06:49:17.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:16 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/625930788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:18.367 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 systemd[1]: Starting Ceph mds.k for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 podman[66019]: 2024-06-13 06:49:18.359609073 +0000 UTC m=+0.017545050 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 podman[66019]: 2024-06-13 06:49:18.461911661 +0000 UTC m=+0.119847681 container create ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 podman[66019]: 2024-06-13 06:49:18.673186906 +0000 UTC m=+0.331122888 container init ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 podman[66019]: 2024-06-13 06:49:18.676139222 +0000 UTC m=+0.334075200 container start ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mds[66038]: -- 172.21.15.175:0/1487924600 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6289eda40 con 0x55d6289c0800 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k[66034]: starting mds.k at 2024-06-13T06:49:18.764 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 bash[66019]: ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c 2024-06-13T06:49:19.029 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 06:49:18 smithi175 systemd[1]: Started Ceph mds.k for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mon[36886]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:18 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:18 smithi007 ceph-mon[32077]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:18 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:18 smithi189 ceph-mon[36616]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:18 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6832/587777641,v1:172.21.15.175:6833/587777641] up:boot 2024-06-13T06:49:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: fsmap 2 up:standby 2024-06-13T06:49:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:20.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:49:20.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:19 smithi007 ceph-mon[32077]: Deploying daemon mds.j on smithi007 2024-06-13T06:49:20.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:20.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6832/587777641,v1:172.21.15.175:6833/587777641] up:boot 2024-06-13T06:49:20.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: fsmap 2 up:standby 2024-06-13T06:49:20.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:20.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:49:20.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:19 smithi189 ceph-mon[36616]: Deploying daemon mds.j on smithi007 2024-06-13T06:49:20.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:20.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6832/587777641,v1:172.21.15.175:6833/587777641] up:boot 2024-06-13T06:49:20.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: fsmap 2 up:standby 2024-06-13T06:49:20.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:20.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-13T06:49:20.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:19 smithi175 ceph-mon[36886]: Deploying daemon mds.j on smithi007 2024-06-13T06:49:21.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:20 smithi175 ceph-mon[36886]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:21.131 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:20 smithi007 ceph-mon[32077]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:20 smithi189 ceph-mon[36616]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:21.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 -- 172.21.15.7:0/2648779264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102510 msgr2=0x7f1cec102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/2648779264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102510 0x7f1cec102990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009990 tx=0x7f1cd8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 -- 172.21.15.7:0/2648779264 shutdown_connections 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/2648779264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102ed0 0x7f1cec10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/2648779264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 0x7f1cec108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/2648779264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102510 0x7f1cec102990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 -- 172.21.15.7:0/2648779264 >> 172.21.15.7:0/2648779264 conn(0x7f1cec079560 msgr2=0x7f1cec079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 -- 172.21.15.7:0/2648779264 shutdown_connections 2024-06-13T06:49:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.754+0000 7f1cf0e00700 1 -- 172.21.15.7:0/2648779264 wait complete. 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 Processor -- start 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 -- start start 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102510 0x7f1cec19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 0x7f1cec19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1127f0 con 0x7f1cec108520 2024-06-13T06:49:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112670 con 0x7f1cec102ed0 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.755+0000 7f1cf0e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112970 con 0x7f1cec102510 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36758/0 (socket says 172.21.15.7:36758) 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 -- 172.21.15.7:0/1855776232 learned_addr learned my addr 172.21.15.7:0/1855776232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1cea800700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 0x7f1cec19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102510 0x7f1cec19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 -- 172.21.15.7:0/1855776232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102510 msgr2=0x7f1cec19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102510 0x7f1cec19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 -- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 msgr2=0x7f1cec19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 0x7f1cec19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 -- 172.21.15.7:0/1855776232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec19fe30 con 0x7f1cec102ed0 2024-06-13T06:49:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.756+0000 7f1ce9400700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000c00 tx=0x7f1cd800ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.757+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8027070 con 0x7f1cec102ed0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.757+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd8003d70 con 0x7f1cec102ed0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.757+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1a00c0 con 0x7f1cec102ed0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.757+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800f460 con 0x7f1cec102ed0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.757+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a79a0 con 0x7f1cec102ed0 2024-06-13T06:49:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.758+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec069180 con 0x7f1cec102ed0 2024-06-13T06:49:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.760+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1cd802a070 con 0x7f1cec102ed0 2024-06-13T06:49:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.761+0000 7f1ce2a00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 0x7f1cd00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.761+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1cd80d02d0 con 0x7f1cec102ed0 2024-06-13T06:49:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.761+0000 7f1ce9e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 0x7f1cd00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.762+0000 7f1ce9e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 0x7f1cd00a5160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009be0 tx=0x7f1cdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:21.763+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1cd8013070 con 0x7f1cec102ed0 2024-06-13T06:49:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.046+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cec1a7b70 con 0x7f1cec102ed0 2024-06-13T06:49:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.047+0000 7f1ce2a00700 1 -- 172.21.15.7:0/1855776232 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f1cd809d6c0 con 0x7f1cec102ed0 2024-06-13T06:49:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:22.048 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-13T06:49:22.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 msgr2=0x7f1cd00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 0x7f1cd00a5160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009be0 tx=0x7f1cdc009260 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 msgr2=0x7f1cec19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8000c00 tx=0x7f1cd800ba90 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 shutdown_connections 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cec108520 0x7f1cec19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1cec102510 0x7f1cec19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1cd00a2ca0 0x7f1cd00a5160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 --2- 172.21.15.7:0/1855776232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1cec102ed0 0x7f1cec19b360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 >> 172.21.15.7:0/1855776232 conn(0x7f1cec079560 msgr2=0x7f1cec0747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 shutdown_connections 2024-06-13T06:49:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:22.051+0000 7f1cf0e00700 1 -- 172.21.15.7:0/1855776232 wait complete. 2024-06-13T06:49:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-06-13T06:49:23.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:22 smithi007 ceph-mon[32077]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:23.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:22 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1855776232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:23.216 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:22 smithi189 ceph-mon[36616]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:23.217 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:22 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1855776232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:22 smithi175 ceph-mon[36886]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:22 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1855776232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:24.414 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 systemd[1]: Starting Ceph mds.j for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:24.741 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 podman[69613]: 2024-06-13 06:49:24.361929916 +0000 UTC m=+0.046071083 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:24.741 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 podman[69613]: 2024-06-13 06:49:24.453450612 +0000 UTC m=+0.137591790 container create 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-13T06:49:24.741 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 podman[69613]: 2024-06-13 06:49:24.741439822 +0000 UTC m=+0.425581010 container init 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-13T06:49:24.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:25.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:24 smithi007 ceph-mon[32077]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:25.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 podman[69613]: 2024-06-13 06:49:24.753294628 +0000 UTC m=+0.437435787 container start 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-06-13T06:49:25.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 ceph-mds[69632]: -- 172.21.15.7:0/2558326315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9a669ba40 con 0x55a9a666e800 2024-06-13T06:49:25.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j[69628]: starting mds.j at 2024-06-13T06:49:25.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 bash[69613]: 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb 2024-06-13T06:49:25.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 06:49:24 smithi007 systemd[1]: Started Ceph mds.j for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:25.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:24 smithi189 ceph-mon[36616]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:25.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:24 smithi175 ceph-mon[36886]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:26.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6834/3083618837,v1:172.21.15.7:6835/3083618837] up:boot 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: fsmap 3 up:standby 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:26.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:25 smithi007 ceph-mon[32077]: Deploying daemon mds.d on smithi007 2024-06-13T06:49:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6834/3083618837,v1:172.21.15.7:6835/3083618837] up:boot 2024-06-13T06:49:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: fsmap 3 up:standby 2024-06-13T06:49:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:26.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:25 smithi189 ceph-mon[36616]: Deploying daemon mds.d on smithi007 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6834/3083618837,v1:172.21.15.7:6835/3083618837] up:boot 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: fsmap 3 up:standby 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:26.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:25 smithi175 ceph-mon[36886]: Deploying daemon mds.d on smithi007 2024-06-13T06:49:27.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:26 smithi007 ceph-mon[32077]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:27.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:26 smithi189 ceph-mon[36616]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:27.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:26 smithi175 ceph-mon[36886]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 -- 172.21.15.7:0/3912504467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 msgr2=0x7f9a6410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 --2- 172.21.15.7:0/3912504467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410f320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b0a0 tx=0x7f9a5c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 -- 172.21.15.7:0/3912504467 shutdown_connections 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 --2- 172.21.15.7:0/3912504467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410f320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 --2- 172.21.15.7:0/3912504467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a64108500 0x7f9a641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.776+0000 7f9a6c200700 1 --2- 172.21.15.7:0/3912504467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a6406a170 0x7f9a6406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.777+0000 7f9a6c200700 1 -- 172.21.15.7:0/3912504467 >> 172.21.15.7:0/3912504467 conn(0x7f9a64078340 msgr2=0x7f9a64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.777+0000 7f9a6c200700 1 -- 172.21.15.7:0/3912504467 shutdown_connections 2024-06-13T06:49:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.777+0000 7f9a6c200700 1 -- 172.21.15.7:0/3912504467 wait complete. 2024-06-13T06:49:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.778+0000 7f9a6c200700 1 Processor -- start 2024-06-13T06:49:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.778+0000 7f9a6c200700 1 -- start start 2024-06-13T06:49:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 0x7f9a6410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 0x7f9a641a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641127b0 con 0x7f9a6406a170 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a64112630 con 0x7f9a64108500 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6c200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a64112930 con 0x7f9a6406ab30 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a69800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 0x7f9a6410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a6a200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 0x7f9a641a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a68e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a69800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 0x7f9a6410e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57396/0 (socket says 172.21.15.7:57396) 2024-06-13T06:49:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.779+0000 7f9a69800700 1 -- 172.21.15.7:0/2860875933 learned_addr learned my addr 172.21.15.7:0/2860875933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 -- 172.21.15.7:0/2860875933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 msgr2=0x7f9a6410ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 -- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 msgr2=0x7f9a6410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 0x7f9a6410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 -- 172.21.15.7:0/2860875933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a641a4270 con 0x7f9a64108500 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a68e00700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.780+0000 7f9a6a200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 0x7f9a641a3b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b070 tx=0x7f9a5c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.781+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c021070 con 0x7f9a64108500 2024-06-13T06:49:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.781+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a5c007b80 con 0x7f9a64108500 2024-06-13T06:49:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.781+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c005040 con 0x7f9a64108500 2024-06-13T06:49:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.781+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a641a4500 con 0x7f9a64108500 2024-06-13T06:49:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.781+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a641abe10 con 0x7f9a64108500 2024-06-13T06:49:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.782+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a64069180 con 0x7f9a64108500 2024-06-13T06:49:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.784+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9a5c024070 con 0x7f9a64108500 2024-06-13T06:49:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.786+0000 7f9a62000700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 0x7f9a4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.786+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9a5c00f020 con 0x7f9a64108500 2024-06-13T06:49:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.786+0000 7f9a69800700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 0x7f9a4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.787+0000 7f9a69800700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 0x7f9a4c0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009ca0 tx=0x7f9a58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:28.788+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a5c02b020 con 0x7f9a64108500 2024-06-13T06:49:29.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:28 smithi007 ceph-mon[32077]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:29.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.082+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a641a05f0 con 0x7f9a64108500 2024-06-13T06:49:29.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.083+0000 7f9a62000700 1 -- 172.21.15.7:0/2860875933 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f9a5c007d30 con 0x7f9a64108500 2024-06-13T06:49:29.083 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:29.083 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 msgr2=0x7f9a4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 0x7f9a4c0a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009ca0 tx=0x7f9a58009260 comp rx=0 tx=0).stop 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 msgr2=0x7f9a641a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 0x7f9a641a3b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b070 tx=0x7f9a5c004c80 comp rx=0 tx=0).stop 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 shutdown_connections 2024-06-13T06:49:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2c20 0x7f9a4c0a50e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64108500 0x7f9a641a3b60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a6406a170 0x7f9a6410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 --2- 172.21.15.7:0/2860875933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6406ab30 0x7f9a6410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 >> 172.21.15.7:0/2860875933 conn(0x7f9a64078340 msgr2=0x7f9a640ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 shutdown_connections 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:29.086+0000 7f9a6c200700 1 -- 172.21.15.7:0/2860875933 wait complete. 2024-06-13T06:49:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-06-13T06:49:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:28 smithi189 ceph-mon[36616]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:29.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:28 smithi175 ceph-mon[36886]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:29.414 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 podman[70053]: 2024-06-13 06:49:29.095221312 +0000 UTC m=+0.033281363 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:29.726 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 podman[70053]: 2024-06-13 06:49:29.470157356 +0000 UTC m=+0.408217458 container create 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-13T06:49:29.977 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 podman[70053]: 2024-06-13 06:49:29.825694368 +0000 UTC m=+0.763754491 container init 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T06:49:29.978 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 podman[70053]: 2024-06-13 06:49:29.837895199 +0000 UTC m=+0.775955312 container start 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T06:49:29.978 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 ceph-mds[70088]: -- 172.21.15.7:0/3296757406 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c69c55a40 con 0x557c69c29000 2024-06-13T06:49:29.978 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d[70084]: starting mds.d at 2024-06-13T06:49:29.978 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 bash[70053]: 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f 2024-06-13T06:49:30.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:29 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2860875933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:29 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2860875933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:30.378 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:29 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2860875933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:30.378 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 06:49:29 smithi007 systemd[1]: Started Ceph mds.d for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:30 smithi175 ceph-mon[36886]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6836/3062011119,v1:172.21.15.7:6837/3062011119] up:boot 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: fsmap 4 up:standby 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:31.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:31 smithi175 ceph-mon[36886]: Deploying daemon mds.e on smithi175 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6836/3062011119,v1:172.21.15.7:6837/3062011119] up:boot 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: fsmap 4 up:standby 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:31.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:31.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:31.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:30 smithi189 ceph-mon[36616]: Deploying daemon mds.e on smithi175 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:30 smithi007 ceph-mon[32077]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:30 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6836/3062011119,v1:172.21.15.7:6837/3062011119] up:boot 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:30 smithi007 ceph-mon[32077]: fsmap 4 up:standby 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:31.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:31 smithi007 ceph-mon[32077]: Deploying daemon mds.e on smithi175 2024-06-13T06:49:32.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:33 smithi175 ceph-mon[36886]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:33 smithi007 ceph-mon[32077]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:33.264 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:33 smithi189 ceph-mon[36616]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6834/1289787148,v1:172.21.15.175:6835/1289787148] up:boot 2024-06-13T06:49:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: fsmap 5 up:standby 2024-06-13T06:49:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:49:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:34.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:34.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:34.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:34.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:34 smithi189 ceph-mon[36616]: Deploying daemon mds.f on smithi189 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6834/1289787148,v1:172.21.15.175:6835/1289787148] up:boot 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: fsmap 5 up:standby 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:34.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:34.344 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:34.344 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:34 smithi007 ceph-mon[32077]: Deploying daemon mds.f on smithi189 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6834/1289787148,v1:172.21.15.175:6835/1289787148] up:boot 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: fsmap 5 up:standby 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:34 smithi175 ceph-mon[36886]: Deploying daemon mds.f on smithi189 2024-06-13T06:49:35.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.212+0000 7f0ec2e00700 1 -- 172.21.15.7:0/2237645408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc10ec00 msgr2=0x7f0ebc111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:35.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.212+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/2237645408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc111070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009f90 tx=0x7f0eb400b330 comp rx=0 tx=0).stop 2024-06-13T06:49:35.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.212+0000 7f0ec2e00700 1 -- 172.21.15.7:0/2237645408 shutdown_connections 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.212+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/2237645408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc111070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.213+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/2237645408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.213+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/2237645408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.213+0000 7f0ec2e00700 1 -- 172.21.15.7:0/2237645408 >> 172.21.15.7:0/2237645408 conn(0x7f0ebc0fdf10 msgr2=0x7f0ebc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.213+0000 7f0ec2e00700 1 -- 172.21.15.7:0/2237645408 shutdown_connections 2024-06-13T06:49:35.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.213+0000 7f0ec2e00700 1 -- 172.21.15.7:0/2237645408 wait complete. 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 Processor -- start 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 -- start start 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc114930 con 0x7f0ebc0748c0 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.214+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1147b0 con 0x7f0ebc10ec00 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec2e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc114ab0 con 0x7f0ebc0751e0 2024-06-13T06:49:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38084/0 (socket says 172.21.15.7:38084) 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53916/0 (socket says 172.21.15.7:53916) 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 -- 172.21.15.7:0/340200145 learned_addr learned my addr 172.21.15.7:0/340200145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec2400700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 -- 172.21.15.7:0/340200145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc10ec00 msgr2=0x7f0ebc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 -- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 msgr2=0x7f0ebc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 -- 172.21.15.7:0/340200145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc19fd20 con 0x7f0ebc0751e0 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.215+0000 7f0ec1000700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ec10 tx=0x7f0ea800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.216+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800cd40 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.216+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8004500 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.216+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc1a0010 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.216+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8010690 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.216+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1a7890 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.217+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc069180 con 0x7f0ebc0751e0 2024-06-13T06:49:35.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.219+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0ea8003f10 con 0x7f0ebc0751e0 2024-06-13T06:49:35.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.220+0000 7f0eba000700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 0x7f0ea40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:35.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.220+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0ea80cef10 con 0x7f0ebc0751e0 2024-06-13T06:49:35.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.220+0000 7f0ec1a00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 0x7f0ea40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.221+0000 7f0ec1a00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 0x7f0ea40a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009c60 tx=0x7f0eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:35.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.224+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ea809c270 con 0x7f0ebc0751e0 2024-06-13T06:49:35.293 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:35 smithi189 ceph-mon[36616]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:35 smithi175 ceph-mon[36886]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:35.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:35 smithi007 ceph-mon[32077]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:35.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.506+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ebc19c070 con 0x7f0ebc0751e0 2024-06-13T06:49:35.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.507+0000 7f0eba000700 1 -- 172.21.15.7:0/340200145 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f0ea809bb70 con 0x7f0ebc0751e0 2024-06-13T06:49:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-13T06:49:35.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 msgr2=0x7f0ea40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:35.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 0x7f0ea40a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009c60 tx=0x7f0eb0009260 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 msgr2=0x7f0ebc19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ec10 tx=0x7f0ea800ef20 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 shutdown_connections 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea40a2ca0 0x7f0ea40a5160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ebc10ec00 0x7f0ebc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ebc0748c0 0x7f0ebc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 --2- 172.21.15.7:0/340200145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc0751e0 0x7f0ebc19b2a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.510+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 >> 172.21.15.7:0/340200145 conn(0x7f0ebc0fdf10 msgr2=0x7f0ebc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.511+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 shutdown_connections 2024-06-13T06:49:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:35.511+0000 7f0ec2e00700 1 -- 172.21.15.7:0/340200145 wait complete. 2024-06-13T06:49:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-06-13T06:49:35.586 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 podman[66879]: 2024-06-13 06:49:35.540814074 +0000 UTC m=+0.143683668 container create bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 podman[66879]: 2024-06-13 06:49:35.794108376 +0000 UTC m=+0.396977936 container init bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 podman[66879]: 2024-06-13 06:49:35.797082155 +0000 UTC m=+0.399951711 container start bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 ceph-mds[66898]: -- 172.21.15.189:0/789911152 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fda283a40 con 0x556fda256c00 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f[66894]: starting mds.f at 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 bash[66879]: bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc 2024-06-13T06:49:36.041 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 06:49:35 smithi189 systemd[1]: Started Ceph mds.f for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:36.302 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/340200145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:36.302 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.303 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.303 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.303 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:36.303 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:36.303 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/340200145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/340200145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:37 smithi189 ceph-mon[36616]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:37 smithi189 ceph-mon[36616]: Deploying daemon mds.c on smithi189 2024-06-13T06:49:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:37 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6834/4153094559,v1:172.21.15.189:6835/4153094559] up:boot 2024-06-13T06:49:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:37 smithi189 ceph-mon[36616]: fsmap 6 up:standby 2024-06-13T06:49:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:49:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:37 smithi175 ceph-mon[36886]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:37 smithi175 ceph-mon[36886]: Deploying daemon mds.c on smithi189 2024-06-13T06:49:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:37 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6834/4153094559,v1:172.21.15.189:6835/4153094559] up:boot 2024-06-13T06:49:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:37 smithi175 ceph-mon[36886]: fsmap 6 up:standby 2024-06-13T06:49:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:49:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:37 smithi007 ceph-mon[32077]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:37 smithi007 ceph-mon[32077]: Deploying daemon mds.c on smithi189 2024-06-13T06:49:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:37 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6834/4153094559,v1:172.21.15.189:6835/4153094559] up:boot 2024-06-13T06:49:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:37 smithi007 ceph-mon[32077]: fsmap 6 up:standby 2024-06-13T06:49:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-13T06:49:38.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:38.563 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 systemd[1]: Starting Ceph mds.c for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:38.563 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 podman[67182]: 2024-06-13 06:49:38.46672483 +0000 UTC m=+0.017726554 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:38.836 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 podman[67182]: 2024-06-13 06:49:38.597839373 +0000 UTC m=+0.148841091 container create 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T06:49:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:39.272 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:39 smithi007 ceph-mon[32077]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:39.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:39 smithi189 ceph-mon[36616]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 podman[67182]: 2024-06-13 06:49:38.892752465 +0000 UTC m=+0.443754197 container init 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 podman[67182]: 2024-06-13 06:49:38.89527219 +0000 UTC m=+0.446273913 container start 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 ceph-mds[67201]: -- 172.21.15.189:0/1658004967 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563942727a40 con 0x5639426fac00 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: starting mds.c at 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 bash[67182]: 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 2024-06-13T06:49:39.337 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 06:49:38 smithi189 systemd[1]: Started Ceph mds.c for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:39 smithi175 ceph-mon[36886]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6836/4202454996,v1:172.21.15.189:6837/4202454996] up:boot 2024-06-13T06:49:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: fsmap 7 up:standby 2024-06-13T06:49:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:49:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:40.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:40.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:40.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:40 smithi189 ceph-mon[36616]: Deploying daemon mds.h on smithi175 2024-06-13T06:49:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6836/4202454996,v1:172.21.15.189:6837/4202454996] up:boot 2024-06-13T06:49:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: fsmap 7 up:standby 2024-06-13T06:49:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:49:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:40.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:40.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:40.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:40 smithi175 ceph-mon[36886]: Deploying daemon mds.h on smithi175 2024-06-13T06:49:40.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6836/4202454996,v1:172.21.15.189:6837/4202454996] up:boot 2024-06-13T06:49:40.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: fsmap 7 up:standby 2024-06-13T06:49:40.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T06:49:40.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:40.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:40.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:40.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:40.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:40 smithi007 ceph-mon[32077]: Deploying daemon mds.h on smithi175 2024-06-13T06:49:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 -- 172.21.15.7:0/2948040590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714810a650 msgr2=0x7f714810aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 --2- 172.21.15.7:0/2948040590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714810a650 0x7f714810aa30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f713c009990 tx=0x7f713c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 -- 172.21.15.7:0/2948040590 shutdown_connections 2024-06-13T06:49:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 --2- 172.21.15.7:0/2948040590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 0x7f7148111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 --2- 172.21.15.7:0/2948040590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714810a650 0x7f714810aa30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 --2- 172.21.15.7:0/2948040590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71481045c0 0x7f7148104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 -- 172.21.15.7:0/2948040590 >> 172.21.15.7:0/2948040590 conn(0x7f71481000b0 msgr2=0x7f71481024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 -- 172.21.15.7:0/2948040590 shutdown_connections 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.204+0000 7f714ea00700 1 -- 172.21.15.7:0/2948040590 wait complete. 2024-06-13T06:49:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.205+0000 7f714ea00700 1 Processor -- start 2024-06-13T06:49:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 -- start start 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 0x7f71481a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714810a650 0x7f714819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71481148f0 con 0x7f7148104f80 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114770 con 0x7f71481045c0 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.206+0000 7f714ea00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7148114a70 con 0x7f714810a650 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57090/0 (socket says 172.21.15.7:57090) 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 -- 172.21.15.7:0/3257530882 learned_addr learned my addr 172.21.15.7:0/3257530882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714cc00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 0x7f71481a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714e000700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714810a650 0x7f714819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 -- 172.21.15.7:0/3257530882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714810a650 msgr2=0x7f714819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714810a650 0x7f714819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 -- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 msgr2=0x7f71481a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 0x7f71481a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.207+0000 7f714d600700 1 -- 172.21.15.7:0/3257530882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819b210 con 0x7f71481045c0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f714d600700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c00bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c026070 con 0x7f71481045c0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c003d70 con 0x7f71481045c0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819b4a0 con 0x7f71481045c0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00fdc0 con 0x7f71481045c0 2024-06-13T06:49:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.208+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a77c0 con 0x7f71481045c0 2024-06-13T06:49:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.209+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7148069180 con 0x7f71481045c0 2024-06-13T06:49:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.211+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f713c029070 con 0x7f71481045c0 2024-06-13T06:49:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.212+0000 7f7146000700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 0x7f71300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.213+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f713c0cfff0 con 0x7f71481045c0 2024-06-13T06:49:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.213+0000 7f714cc00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 0x7f71300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.213+0000 7f714cc00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 0x7f71300a5160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7148109c60 tx=0x7f7134009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.215+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f713c013070 con 0x7f71481045c0 2024-06-13T06:49:41.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:41 smithi007 ceph-mon[32077]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:41.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.491+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71481a7990 con 0x7f71481045c0 2024-06-13T06:49:41.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.492+0000 7f7146000700 1 -- 172.21.15.7:0/3257530882 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f713c09d400 con 0x7f71481045c0 2024-06-13T06:49:41.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:41.492 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-13T06:49:41.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 msgr2=0x7f71300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 0x7f71300a5160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7148109c60 tx=0x7f7134009260 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 msgr2=0x7f71481a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c00bb30 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 shutdown_connections 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714810a650 0x7f714819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71300a2ca0 0x7f71300a5160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71481045c0 0x7f71481a08f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 --2- 172.21.15.7:0/3257530882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7148104f80 0x7f71481a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.495+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 >> 172.21.15.7:0/3257530882 conn(0x7f71481000b0 msgr2=0x7f71481016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.496+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 shutdown_connections 2024-06-13T06:49:41.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:41.496+0000 7f714ea00700 1 -- 172.21.15.7:0/3257530882 wait complete. 2024-06-13T06:49:41.496 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-06-13T06:49:41.535 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:41 smithi175 ceph-mon[36886]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:41.535 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 systemd[1]: Starting Ceph mds.h for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:41 smithi189 ceph-mon[36616]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:41.830 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 podman[66640]: 2024-06-13 06:49:41.438681357 +0000 UTC m=+0.017425051 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:41.831 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 podman[66640]: 2024-06-13 06:49:41.564060645 +0000 UTC m=+0.142804324 container create 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-13T06:49:41.831 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 podman[66640]: 2024-06-13 06:49:41.83187201 +0000 UTC m=+0.410615702 container init 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-13T06:49:42.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 podman[66640]: 2024-06-13 06:49:41.83465326 +0000 UTC m=+0.413396941 container start 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-06-13T06:49:42.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 ceph-mds[66659]: -- 172.21.15.175:0/4259757451 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e7a68ba40 con 0x564e7a65f000 2024-06-13T06:49:42.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: starting mds.h at 2024-06-13T06:49:42.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 bash[66640]: 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 2024-06-13T06:49:42.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 06:49:41 smithi175 systemd[1]: Started Ceph mds.h for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3257530882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:42.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:42 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:42.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3257530882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:42.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:42.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:42.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:42 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:42.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3257530882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:42.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:42.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:42.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:42.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:42 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:43 smithi189 ceph-mon[36616]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:43 smithi189 ceph-mon[36616]: Deploying daemon mds.i on smithi189 2024-06-13T06:49:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:43 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6836/2460722721,v1:172.21.15.175:6837/2460722721] up:boot 2024-06-13T06:49:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:43 smithi189 ceph-mon[36616]: fsmap 8 up:standby 2024-06-13T06:49:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:43 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:49:43.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:43 smithi007 ceph-mon[32077]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:43 smithi007 ceph-mon[32077]: Deploying daemon mds.i on smithi189 2024-06-13T06:49:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:43 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6836/2460722721,v1:172.21.15.175:6837/2460722721] up:boot 2024-06-13T06:49:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:43 smithi007 ceph-mon[32077]: fsmap 8 up:standby 2024-06-13T06:49:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:43 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:49:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:43 smithi175 ceph-mon[36886]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:43 smithi175 ceph-mon[36886]: Deploying daemon mds.i on smithi189 2024-06-13T06:49:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:43 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6836/2460722721,v1:172.21.15.175:6837/2460722721] up:boot 2024-06-13T06:49:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:43 smithi175 ceph-mon[36886]: fsmap 8 up:standby 2024-06-13T06:49:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:43 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T06:49:44.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:44.566 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 systemd[1]: Starting Ceph mds.i for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:44.831 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 podman[67498]: 2024-06-13 06:49:44.470358637 +0000 UTC m=+0.017698995 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:44.831 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 podman[67498]: 2024-06-13 06:49:44.579786005 +0000 UTC m=+0.127126360 container create cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T06:49:44.831 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 podman[67498]: 2024-06-13 06:49:44.832447287 +0000 UTC m=+0.379787651 container init cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc) 2024-06-13T06:49:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:45.086 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 podman[67498]: 2024-06-13 06:49:44.835866559 +0000 UTC m=+0.383206922 container start cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-13T06:49:45.086 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 ceph-mds[67517]: -- 172.21.15.189:0/1847734697 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0049dba40 con 0x55e0049aec00 2024-06-13T06:49:45.087 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: starting mds.i at 2024-06-13T06:49:45.087 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 bash[67498]: cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 2024-06-13T06:49:45.087 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 06:49:44 smithi189 systemd[1]: Started Ceph mds.i for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:45.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:45.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:45 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:45.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:45.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:45 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:45.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:45 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:46 smithi189 ceph-mon[36616]: Deploying daemon mds.g on smithi007 2024-06-13T06:49:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:46 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6838/3291148051,v1:172.21.15.189:6839/3291148051] up:boot 2024-06-13T06:49:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:46 smithi189 ceph-mon[36616]: fsmap 9 up:standby 2024-06-13T06:49:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:46 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:49:46.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:46 smithi007 ceph-mon[32077]: Deploying daemon mds.g on smithi007 2024-06-13T06:49:46.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:46 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6838/3291148051,v1:172.21.15.189:6839/3291148051] up:boot 2024-06-13T06:49:46.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:46 smithi007 ceph-mon[32077]: fsmap 9 up:standby 2024-06-13T06:49:46.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:46 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:49:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:46 smithi175 ceph-mon[36886]: Deploying daemon mds.g on smithi007 2024-06-13T06:49:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:46 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6838/3291148051,v1:172.21.15.189:6839/3291148051] up:boot 2024-06-13T06:49:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:46 smithi175 ceph-mon[36886]: fsmap 9 up:standby 2024-06-13T06:49:46.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:46 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T06:49:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:47 smithi007 ceph-mon[32077]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:47 smithi189 ceph-mon[36616]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:47 smithi175 ceph-mon[36886]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.824+0000 7f96cde00700 1 -- 172.21.15.7:0/2513004903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c810c9b0 msgr2=0x7f96c810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.824+0000 7f96cde00700 1 --2- 172.21.15.7:0/2513004903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c810c9b0 0x7f96c810ee70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96c000b0a0 tx=0x7f96c000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 -- 172.21.15.7:0/2513004903 shutdown_connections 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 --2- 172.21.15.7:0/2513004903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c810c9b0 0x7f96c810ee70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 --2- 172.21.15.7:0/2513004903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103580 0x7f96c8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 --2- 172.21.15.7:0/2513004903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8103ea0 0x7f96c810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 -- 172.21.15.7:0/2513004903 >> 172.21.15.7:0/2513004903 conn(0x7f96c80fcdb0 msgr2=0x7f96c80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 -- 172.21.15.7:0/2513004903 shutdown_connections 2024-06-13T06:49:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.825+0000 7f96cde00700 1 -- 172.21.15.7:0/2513004903 wait complete. 2024-06-13T06:49:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.826+0000 7f96cde00700 1 Processor -- start 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.826+0000 7f96cde00700 1 -- start start 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 0x7f96c819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 0x7f96c819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8112750 con 0x7f96c8103ea0 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96c81125d0 con 0x7f96c8103580 2024-06-13T06:49:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cde00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96c81128d0 con 0x7f96c810c9b0 2024-06-13T06:49:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96c7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 0x7f96c819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57100/0 (socket says 172.21.15.7:57100) 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cca00700 1 -- 172.21.15.7:0/782050519 learned_addr learned my addr 172.21.15.7:0/782050519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cd400700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 0x7f96c819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.827+0000 7f96cca00700 1 -- 172.21.15.7:0/782050519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 msgr2=0x7f96c819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cca00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 0x7f96c819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cca00700 1 -- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 msgr2=0x7f96c819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cca00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 0x7f96c819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cca00700 1 -- 172.21.15.7:0/782050519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c819fe20 con 0x7f96c8103580 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96c7e00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 0x7f96c819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cd400700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 0x7f96c819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cca00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00e9a0 tx=0x7f96bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00ca10 con 0x7f96c8103580 2024-06-13T06:49:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc00cbb0 con 0x7f96c8103580 2024-06-13T06:49:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c81a0110 con 0x7f96c8103580 2024-06-13T06:49:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc018680 con 0x7f96c8103580 2024-06-13T06:49:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.828+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c81a7990 con 0x7f96c8103580 2024-06-13T06:49:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.829+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c8069180 con 0x7f96c8103580 2024-06-13T06:49:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.831+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f96bc010430 con 0x7f96c8103580 2024-06-13T06:49:47.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.833+0000 7f96c5600700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 0x7f96b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:47.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.833+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f96bc0cf240 con 0x7f96c8103580 2024-06-13T06:49:47.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.833+0000 7f96c7e00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 0x7f96b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.834+0000 7f96c7e00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 0x7f96b00a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f96c80ff850 tx=0x7f96b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:47.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:47.836+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f96bc09c5a0 con 0x7f96c8103580 2024-06-13T06:49:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.119+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96c81a7b60 con 0x7f96c8103580 2024-06-13T06:49:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.120+0000 7f96c5600700 1 -- 172.21.15.7:0/782050519 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f96bc09bea0 con 0x7f96c8103580 2024-06-13T06:49:48.119 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:48.119 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-13T06:49:48.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 msgr2=0x7f96b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:48.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 0x7f96b00a50e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f96c80ff850 tx=0x7f96b4009260 comp rx=0 tx=0).stop 2024-06-13T06:49:48.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 msgr2=0x7f96c819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:48.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00e9a0 tx=0x7f96bc00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 shutdown_connections 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c810c9b0 0x7f96c819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96b00a2c20 0x7f96b00a50e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c8103580 0x7f96c819ae10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 --2- 172.21.15.7:0/782050519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8103ea0 0x7f96c819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.123+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 >> 172.21.15.7:0/782050519 conn(0x7f96c80fcdb0 msgr2=0x7f96c810ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.124+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 shutdown_connections 2024-06-13T06:49:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:48.124+0000 7f96cde00700 1 -- 172.21.15.7:0/782050519 wait complete. 2024-06-13T06:49:48.124 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-06-13T06:49:48.354 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:48 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/782050519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:48.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:48 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/782050519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:48.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:48 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/782050519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:49.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:49 smithi007 ceph-mon[32077]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:49 smithi189 ceph-mon[36616]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:49 smithi175 ceph-mon[36886]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:49.914 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:49 smithi007 podman[70852]: 2024-06-13 06:49:49.694375081 +0000 UTC m=+0.173826526 container create 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-13T06:49:50.413 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:49 smithi007 podman[70852]: 2024-06-13 06:49:49.973932597 +0000 UTC m=+0.453384102 container init 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-06-13T06:49:50.413 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:49 smithi007 podman[70852]: 2024-06-13 06:49:49.985853882 +0000 UTC m=+0.465305360 container start 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-13T06:49:50.414 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:50 smithi007 ceph-mds[70871]: -- 172.21.15.7:0/3341472853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563da53cfa40 con 0x563da53a2c00 2024-06-13T06:49:50.414 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: starting mds.g at 2024-06-13T06:49:50.414 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:50 smithi007 bash[70852]: 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c 2024-06-13T06:49:50.414 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 06:49:50 smithi007 systemd[1]: Started Ceph mds.g for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:51.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:51.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:51.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:boot 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: fsmap 10 up:standby 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:51.563 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:51 smithi007 ceph-mon[32077]: Deploying daemon mds.a on smithi007 2024-06-13T06:49:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:boot 2024-06-13T06:49:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: fsmap 10 up:standby 2024-06-13T06:49:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:51.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:51 smithi189 ceph-mon[36616]: Deploying daemon mds.a on smithi007 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:boot 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: fsmap 10 up:standby 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:51.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:51.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:51.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:51 smithi175 ceph-mon[36886]: Deploying daemon mds.a on smithi007 2024-06-13T06:49:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:53.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:53 smithi007 ceph-mon[32077]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:53.413 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:53 smithi007 systemd[1]: Starting Ceph mds.a for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:53.443 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:53 smithi175 ceph-mon[36886]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:53 smithi189 ceph-mon[36616]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:54.164 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 podman[71252]: 2024-06-13 06:49:54.044985394 +0000 UTC m=+0.033220404 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:54.164 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 podman[71252]: 2024-06-13 06:49:54.160652137 +0000 UTC m=+0.148887188 container create 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-13T06:49:54.664 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 podman[71252]: 2024-06-13 06:49:54.540629421 +0000 UTC m=+0.528864479 container init 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-13T06:49:54.664 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 podman[71252]: 2024-06-13 06:49:54.552976107 +0000 UTC m=+0.541211130 container start 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-13T06:49:54.664 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 ceph-mds[71289]: -- 172.21.15.7:0/1500592890 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b910b5fa40 con 0x55b910b32c00 2024-06-13T06:49:54.664 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: starting mds.a at 2024-06-13T06:49:55.063 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 bash[71252]: 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e 2024-06-13T06:49:55.063 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 06:49:54 smithi007 systemd[1]: Started Ceph mds.a for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 -- 172.21.15.7:0/1663118010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad810d8a0 msgr2=0x7f6ad810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 --2- 172.21.15.7:0/1663118010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad810d8a0 0x7f6ad810dc80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009990 tx=0x7f6ac8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 -- 172.21.15.7:0/1663118010 shutdown_connections 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 --2- 172.21.15.7:0/1663118010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad8109390 0x7f6ad8115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 --2- 172.21.15.7:0/1663118010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad810d8a0 0x7f6ad810dc80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 --2- 172.21.15.7:0/1663118010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad81089d0 0x7f6ad8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 -- 172.21.15.7:0/1663118010 >> 172.21.15.7:0/1663118010 conn(0x7f6ad8078220 msgr2=0x7f6ad81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 -- 172.21.15.7:0/1663118010 shutdown_connections 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.063+0000 7f6adca00700 1 -- 172.21.15.7:0/1663118010 wait complete. 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.065+0000 7f6adca00700 1 Processor -- start 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.065+0000 7f6adca00700 1 -- start start 2024-06-13T06:49:55.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad8109390 0x7f6ad819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad810d8a0 0x7f6ad81a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8118d00 con 0x7f6ad810d8a0 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8118b80 con 0x7f6ad81089d0 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6adca00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8118e80 con 0x7f6ad8109390 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54058/0 (socket says 172.21.15.7:54058) 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 -- 172.21.15.7:0/3280488569 learned_addr learned my addr 172.21.15.7:0/3280488569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7e00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad810d8a0 0x7f6ad81a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad6a00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad8109390 0x7f6ad819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 -- 172.21.15.7:0/3280488569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad8109390 msgr2=0x7f6ad819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad8109390 0x7f6ad819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 -- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad810d8a0 msgr2=0x7f6ad81a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad810d8a0 0x7f6ad81a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.066+0000 7f6ad7400700 1 -- 172.21.15.7:0/3280488569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad81a4170 con 0x7f6ad81089d0 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6ad7400700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8027070 con 0x7f6ad81089d0 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ac80056d0 con 0x7f6ad81089d0 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81a4400 con 0x7f6ad81089d0 2024-06-13T06:49:55.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8022e10 con 0x7f6ad81089d0 2024-06-13T06:49:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.067+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81abce0 con 0x7f6ad81089d0 2024-06-13T06:49:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.069+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8069180 con 0x7f6ad81089d0 2024-06-13T06:49:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.070+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6ac802a070 con 0x7f6ad81089d0 2024-06-13T06:49:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.072+0000 7f6acfe00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 0x7f6abc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:49:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.072+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6ac80cfd50 con 0x7f6ad81089d0 2024-06-13T06:49:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.072+0000 7f6ad6a00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 0x7f6abc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:49:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.073+0000 7f6ad6a00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 0x7f6abc0a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ad81084d0 tx=0x7f6ac4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:49:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.076+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ac805e850 con 0x7f6ad81089d0 2024-06-13T06:49:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.360+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad81a04c0 con 0x7f6ad81089d0 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:55.360 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:55 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.361+0000 7f6acfe00700 1 -- 172.21.15.7:0/3280488569 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f6ac8013070 con 0x7f6ad81089d0 2024-06-13T06:49:55.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:49:55.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.364+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 msgr2=0x7f6abc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.364+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 0x7f6abc0a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ad81084d0 tx=0x7f6ac4009260 comp rx=0 tx=0).stop 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.364+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 msgr2=0x7f6ad819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.364+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac800bab0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 shutdown_connections 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ad810d8a0 0x7f6ad81a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6abc0a2ca0 0x7f6abc0a5160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad81089d0 0x7f6ad819f1b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 --2- 172.21.15.7:0/3280488569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ad8109390 0x7f6ad819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:49:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 >> 172.21.15.7:0/3280488569 conn(0x7f6ad8078220 msgr2=0x7f6ad8078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:49:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 shutdown_connections 2024-06-13T06:49:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:49:55.365+0000 7f6adca00700 1 -- 172.21.15.7:0/3280488569 wait complete. 2024-06-13T06:49:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-06-13T06:49:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:55.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:55.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:55 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:55.600 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:55.600 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.600 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.600 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:55.600 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-13T06:49:55.601 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-13T06:49:55.601 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:55 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:49:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:56 smithi189 ceph-mon[36616]: Deploying daemon mds.b on smithi175 2024-06-13T06:49:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:56 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:boot 2024-06-13T06:49:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:56 smithi189 ceph-mon[36616]: fsmap 11 up:standby 2024-06-13T06:49:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:56 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:49:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:56 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3280488569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:56.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:56 smithi175 ceph-mon[36886]: Deploying daemon mds.b on smithi175 2024-06-13T06:49:56.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:56 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:boot 2024-06-13T06:49:56.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:56 smithi175 ceph-mon[36886]: fsmap 11 up:standby 2024-06-13T06:49:56.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:56 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:49:56.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:56 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3280488569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:56 smithi007 ceph-mon[32077]: Deploying daemon mds.b on smithi175 2024-06-13T06:49:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:56 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:boot 2024-06-13T06:49:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:56 smithi007 ceph-mon[32077]: fsmap 11 up:standby 2024-06-13T06:49:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:56 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T06:49:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:56 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3280488569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:49:57.208 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 systemd[1]: Starting Ceph mds.b for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T06:49:57.473 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:57 smithi175 ceph-mon[36886]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:57.473 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 podman[66960]: 2024-06-13 06:49:57.252653923 +0000 UTC m=+0.017776345 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T06:49:57.473 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 podman[66960]: 2024-06-13 06:49:57.352241379 +0000 UTC m=+0.117363791 container create 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-13T06:49:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:57 smithi189 ceph-mon[36616]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:57.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:57 smithi007 ceph-mon[32077]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:57.867 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 podman[66960]: 2024-06-13 06:49:57.596726002 +0000 UTC m=+0.361848430 container init 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T06:49:57.867 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 podman[66960]: 2024-06-13 06:49:57.599239125 +0000 UTC m=+0.364361543 container start 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T06:49:57.868 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 ceph-mds[66979]: -- 172.21.15.175:0/3670334374 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d840641a40 con 0x55d840614800 2024-06-13T06:49:57.868 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: starting mds.b at 2024-06-13T06:49:57.868 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 bash[66960]: 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec 2024-06-13T06:49:57.868 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:49:57 smithi175 systemd[1]: Started Ceph mds.b for b9295806-294f-11ef-bc9c-c7b262605968. 2024-06-13T06:49:58.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:49:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:boot 2024-06-13T06:49:59.072 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: fsmap 12 up:standby 2024-06-13T06:49:59.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:49:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:59.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:boot 2024-06-13T06:49:59.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: fsmap 12 up:standby 2024-06-13T06:49:59.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:49:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:49:59.116 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:boot 2024-06-13T06:49:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: fsmap 12 up:standby 2024-06-13T06:49:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:49:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T06:50:00.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:00.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:00 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T06:50:00.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:00.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:00.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:00 smithi175 ceph-mon[36886]: overall HEALTH_OK 2024-06-13T06:50:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:01.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:01.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:01.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:01.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:00 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T06:50:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.680+0000 7f9c9b600700 1 -- 172.21.15.7:0/3125128670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94102e80 msgr2=0x7f9c9410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.680+0000 7f9c9b600700 1 --2- 172.21.15.7:0/3125128670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94102e80 0x7f9c9410f330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009f90 tx=0x7f9c8c00b330 comp rx=0 tx=0).stop 2024-06-13T06:50:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 -- 172.21.15.7:0/3125128670 shutdown_connections 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 --2- 172.21.15.7:0/3125128670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c94102e80 0x7f9c9410f330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 --2- 172.21.15.7:0/3125128670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941084f0 0x7f9c941088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 --2- 172.21.15.7:0/3125128670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c941024e0 0x7f9c94102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 -- 172.21.15.7:0/3125128670 >> 172.21.15.7:0/3125128670 conn(0x7f9c94079530 msgr2=0x7f9c94079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 -- 172.21.15.7:0/3125128670 shutdown_connections 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.681+0000 7f9c9b600700 1 -- 172.21.15.7:0/3125128670 wait complete. 2024-06-13T06:50:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.683+0000 7f9c9b600700 1 Processor -- start 2024-06-13T06:50:01.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.683+0000 7f9c9b600700 1 -- start start 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.683+0000 7f9c9b600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.683+0000 7f9c9b600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c9b600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 0x7f9c94075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c9b600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94112790 con 0x7f9c94102e80 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c9b600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94112610 con 0x7f9c941024e0 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c9b600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94112910 con 0x7f9c941084f0 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c93e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47764/0 (socket says 172.21.15.7:47764) 2024-06-13T06:50:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50610/0 (socket says 172.21.15.7:50610) 2024-06-13T06:50:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c93e00700 1 -- 172.21.15.7:0/659381463 learned_addr learned my addr 172.21.15.7:0/659381463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:01.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c99600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 0x7f9c94075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 -- 172.21.15.7:0/659381463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 msgr2=0x7f9c94075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 0x7f9c94075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 -- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 msgr2=0x7f9c94074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.684+0000 7f9c98c00700 1 -- 172.21.15.7:0/659381463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c941a7150 con 0x7f9c941024e0 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c99600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 0x7f9c94075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c93e00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c98c00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800e9f0 tx=0x7f9c8800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:01.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800cb70 con 0x7f9c941024e0 2024-06-13T06:50:01.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c8800ee90 con 0x7f9c941024e0 2024-06-13T06:50:01.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941a7440 con 0x7f9c941024e0 2024-06-13T06:50:01.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88003dc0 con 0x7f9c941024e0 2024-06-13T06:50:01.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.685+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941a7980 con 0x7f9c941024e0 2024-06-13T06:50:01.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.687+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94069180 con 0x7f9c941024e0 2024-06-13T06:50:01.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.688+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9c88004000 con 0x7f9c941024e0 2024-06-13T06:50:01.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.690+0000 7f9c91600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 0x7f9c7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.690+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9c880ced30 con 0x7f9c941024e0 2024-06-13T06:50:01.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.690+0000 7f9c93e00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 0x7f9c7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:01.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.691+0000 7f9c93e00700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 0x7f9c7c0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009750 tx=0x7f9c80009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:01.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.694+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c8809c090 con 0x7f9c941024e0 2024-06-13T06:50:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.994+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c94075cc0 con 0x7f9c941024e0 2024-06-13T06:50:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.996+0000 7f9c91600700 1 -- 172.21.15.7:0/659381463 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f9c8809b990 con 0x7f9c941024e0 2024-06-13T06:50:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:01.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 msgr2=0x7f9c7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 0x7f9c7c0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009750 tx=0x7f9c80009310 comp rx=0 tx=0).stop 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 msgr2=0x7f9c94074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800e9f0 tx=0x7f9c8800ed00 comp rx=0 tx=0).stop 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 shutdown_connections 2024-06-13T06:50:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c941084f0 0x7f9c94075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c7c0a2c20 0x7f9c7c0a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c941024e0 0x7f9c94074790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 --2- 172.21.15.7:0/659381463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c94102e80 0x7f9c94074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 >> 172.21.15.7:0/659381463 conn(0x7f9c94079530 msgr2=0x7f9c94100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 shutdown_connections 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:01.999+0000 7f9c9b600700 1 -- 172.21.15.7:0/659381463 wait complete. 2024-06-13T06:50:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-06-13T06:50:02.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:02 smithi007 ceph-mon[32077]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:02.880 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:02 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/659381463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:02.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-13T06:50:03.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:02 smithi189 ceph-mon[36616]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:03.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:02 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/659381463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:02 smithi175 ceph-mon[36886]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:02 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/659381463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:03.417 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:04.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.607 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:04.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:04.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:04.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:50:05.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:05 smithi007 ceph-mon[32077]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:05 smithi189 ceph-mon[36616]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:05 smithi175 ceph-mon[36886]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 -- 172.21.15.7:0/2632038122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c074fe0 msgr2=0x7fad7c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 --2- 172.21.15.7:0/2632038122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c074fe0 0x7fad7c10e9b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad68009990 tx=0x7fad68009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 -- 172.21.15.7:0/2632038122 shutdown_connections 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 --2- 172.21.15.7:0/2632038122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 0x7fad7c1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 --2- 172.21.15.7:0/2632038122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c0746c0 0x7fad7c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 --2- 172.21.15.7:0/2632038122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c074fe0 0x7fad7c10e9b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 -- 172.21.15.7:0/2632038122 >> 172.21.15.7:0/2632038122 conn(0x7fad7c0795f0 msgr2=0x7fad7c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 -- 172.21.15.7:0/2632038122 shutdown_connections 2024-06-13T06:50:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.649+0000 7fad82c00700 1 -- 172.21.15.7:0/2632038122 wait complete. 2024-06-13T06:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.650+0000 7fad82c00700 1 Processor -- start 2024-06-13T06:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 -- start start 2024-06-13T06:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c074fe0 0x7fad7c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 0x7fad7c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c114a50 con 0x7fad7c10eef0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c1148d0 con 0x7fad7c0746c0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad82c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c114bd0 con 0x7fad7c074fe0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad7be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad7be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad7be00700 1 -- 172.21.15.7:0/1390209944 learned_addr learned my addr 172.21.15.7:0/1390209944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.651+0000 7fad80c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 0x7fad7c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7b400700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c074fe0 0x7fad7c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 -- 172.21.15.7:0/1390209944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c074fe0 msgr2=0x7fad7c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c074fe0 0x7fad7c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 -- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 msgr2=0x7fad7c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 0x7fad7c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 -- 172.21.15.7:0/1390209944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c19fea0 con 0x7fad7c0746c0 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad7be00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00ef30 tx=0x7fad6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0092e0 con 0x7fad7c0746c0 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6c004bb0 con 0x7fad7c0746c0 2024-06-13T06:50:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.652+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004280 con 0x7fad7c0746c0 2024-06-13T06:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.653+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c1a0130 con 0x7fad7c0746c0 2024-06-13T06:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.653+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1a7a10 con 0x7fad7c0746c0 2024-06-13T06:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.654+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c069180 con 0x7fad7c0746c0 2024-06-13T06:50:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.656+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fad6c01b030 con 0x7fad7c0746c0 2024-06-13T06:50:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.657+0000 7fad78c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 0x7fad600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.657+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fad6c0ce7d0 con 0x7fad7c0746c0 2024-06-13T06:50:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.657+0000 7fad7b400700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 0x7fad600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.658+0000 7fad7b400700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 0x7fad600a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b2e0 tx=0x7fad6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.661+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad6c022020 con 0x7fad7c0746c0 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.925+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fad7c1a7be0 con 0x7fad7c0746c0 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.925+0000 7fad78c00700 1 -- 172.21.15.7:0/1390209944 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fad6c010040 con 0x7fad7c0746c0 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T06:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-06-13T06:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-06-13T06:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-06-13T06:50:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 msgr2=0x7fad600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 0x7fad600a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b2e0 tx=0x7fad6801a040 comp rx=0 tx=0).stop 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 msgr2=0x7fad7c19aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00ef30 tx=0x7fad6c00c3a0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 shutdown_connections 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c10eef0 0x7fad7c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2ca0 0x7fad600a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad7c0746c0 0x7fad7c19aee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 --2- 172.21.15.7:0/1390209944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad7c074fe0 0x7fad7c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.929+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 >> 172.21.15.7:0/1390209944 conn(0x7fad7c0795f0 msgr2=0x7fad7c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.930+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 shutdown_connections 2024-06-13T06:50:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:05.930+0000 7fad82c00700 1 -- 172.21.15.7:0/1390209944 wait complete. 2024-06-13T06:50:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:06 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1390209944' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:50:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:06 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1390209944' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:50:06.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:06 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1390209944' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-13T06:50:07.212 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-13T06:50:07.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-13T06:50:07.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:07 smithi007 ceph-mon[32077]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:07 smithi175 ceph-mon[36886]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:07.656 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:07 smithi189 ceph-mon[36616]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:09.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:09.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:09.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:09.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:50:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:50:09.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:50:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 -- 172.21.15.7:0/878304135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 msgr2=0x7f22c810f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 --2- 172.21.15.7:0/878304135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c810f360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b0a0 tx=0x7f22c000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 -- 172.21.15.7:0/878304135 shutdown_connections 2024-06-13T06:50:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 --2- 172.21.15.7:0/878304135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c810f360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 --2- 172.21.15.7:0/878304135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8108520 0x7f22c8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 --2- 172.21.15.7:0/878304135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8102510 0x7f22c8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 -- 172.21.15.7:0/878304135 >> 172.21.15.7:0/878304135 conn(0x7f22c8079640 msgr2=0x7f22c8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 -- 172.21.15.7:0/878304135 shutdown_connections 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.839+0000 7f22cd200700 1 -- 172.21.15.7:0/878304135 wait complete. 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.840+0000 7f22cd200700 1 Processor -- start 2024-06-13T06:50:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 -- start start 2024-06-13T06:50:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 0x7f22c819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22c81127f0 con 0x7f22c8102510 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8112670 con 0x7f22c8108520 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22cd200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22c8112970 con 0x7f22c8102ed0 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22c5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22c5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47800/0 (socket says 172.21.15.7:47800) 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22c5e00700 1 -- 172.21.15.7:0/2483134573 learned_addr learned my addr 172.21.15.7:0/2483134573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.841+0000 7f22bfe00700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 0x7f22c819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 -- 172.21.15.7:0/2483134573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 msgr2=0x7f22c819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 -- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 msgr2=0x7f22c819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 -- 172.21.15.7:0/2483134573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c819fd50 con 0x7f22c8108520 2024-06-13T06:50:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c5e00700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.842+0000 7f22c6800700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 0x7f22c819f610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b070 tx=0x7f22c0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.843+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0021070 con 0x7f22c8108520 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.843+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c0007b80 con 0x7f22c8108520 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.843+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0005040 con 0x7f22c8108520 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.843+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c819ffe0 con 0x7f22c8108520 2024-06-13T06:50:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.843+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c81a78c0 con 0x7f22c8108520 2024-06-13T06:50:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.845+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a4005330 con 0x7f22c8108520 2024-06-13T06:50:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.846+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f22c0024070 con 0x7f22c8108520 2024-06-13T06:50:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.848+0000 7f22bea00700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 0x7f22ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.848+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22c000f020 con 0x7f22c8108520 2024-06-13T06:50:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.848+0000 7f22c5e00700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 0x7f22ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.849+0000 7f22c5e00700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 0x7f22ac0a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22b8011630 tx=0x7f22b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:09.852+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22c002b020 con 0x7f22c8108520 2024-06-13T06:50:10.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.085+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f22a4005620 con 0x7f22c8108520 2024-06-13T06:50:10.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.830+0000 7f22bea00700 1 -- 172.21.15.7:0/2483134573 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f22c009d1d0 con 0x7f22c8108520 2024-06-13T06:50:10.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.837+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 msgr2=0x7f22ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:10.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.837+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 0x7f22ac0a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f22b8011630 tx=0x7f22b8009260 comp rx=0 tx=0).stop 2024-06-13T06:50:10.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.838+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 msgr2=0x7f22c819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:10.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.838+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 0x7f22c819f610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b070 tx=0x7f22c0004c80 comp rx=0 tx=0).stop 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.839+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 shutdown_connections 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.839+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22ac0a2c20 0x7f22ac0a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c8108520 0x7f22c819f610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c8102510 0x7f22c819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 --2- 172.21.15.7:0/2483134573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c8102ed0 0x7f22c819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 >> 172.21.15.7:0/2483134573 conn(0x7f22c8079640 msgr2=0x7f22c8074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 shutdown_connections 2024-06-13T06:50:10.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:10.840+0000 7f22cd200700 1 -- 172.21.15.7:0/2483134573 wait complete. 2024-06-13T06:50:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-06-13T06:50:11.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:10 smithi175 ceph-mon[36886]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:11.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:10 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2483134573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:10 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:10 smithi007 ceph-mon[32077]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:11.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:10 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2483134573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:10 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:10 smithi189 ceph-mon[36616]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:10 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2483134573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:10 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-13T06:50:11.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-13T06:50:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:11 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:50:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:11 smithi007 ceph-mon[32077]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:50:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:11 smithi007 ceph-mon[32077]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:50:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:11 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:50:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:11 smithi175 ceph-mon[36886]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:50:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:11 smithi175 ceph-mon[36886]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:50:12.230 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:12.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:11 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-13T06:50:12.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:11 smithi189 ceph-mon[36616]: osdmap e83: 12 total, 12 up, 12 in 2024-06-13T06:50:12.238 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:11 smithi189 ceph-mon[36616]: osdmap e84: 12 total, 12 up, 12 in 2024-06-13T06:50:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:12 smithi175 ceph-mon[36886]: pgmap v319: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:12 smithi175 ceph-mon[36886]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:12 smithi175 ceph-mon[36886]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:50:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:12 smithi007 ceph-mon[32077]: pgmap v319: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:12 smithi007 ceph-mon[32077]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:12 smithi007 ceph-mon[32077]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:50:13.253 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:12 smithi189 ceph-mon[36616]: pgmap v319: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:13.253 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:12 smithi189 ceph-mon[36616]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:13.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:12 smithi189 ceph-mon[36616]: osdmap e85: 12 total, 12 up, 12 in 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 -- 172.21.15.7:0/259879875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c0746c0 msgr2=0x7fdd5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 --2- 172.21.15.7:0/259879875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c074aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009990 tx=0x7fdd4c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 -- 172.21.15.7:0/259879875 shutdown_connections 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 --2- 172.21.15.7:0/259879875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 --2- 172.21.15.7:0/259879875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c074aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 --2- 172.21.15.7:0/259879875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 -- 172.21.15.7:0/259879875 >> 172.21.15.7:0/259879875 conn(0x7fdd5c0795e0 msgr2=0x7fdd5c0799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.611+0000 7fdd63000700 1 -- 172.21.15.7:0/259879875 shutdown_connections 2024-06-13T06:50:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.612+0000 7fdd63000700 1 -- 172.21.15.7:0/259879875 wait complete. 2024-06-13T06:50:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.613+0000 7fdd63000700 1 Processor -- start 2024-06-13T06:50:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.613+0000 7fdd63000700 1 -- start start 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c114a90 con 0x7fdd5c074fe0 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c114910 con 0x7fdd5c0746c0 2024-06-13T06:50:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd63000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c114c10 con 0x7fdd5c10eef0 2024-06-13T06:50:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd53400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd5be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd61000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd53400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36522/0 (socket says 172.21.15.7:36522) 2024-06-13T06:50:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd53400700 1 -- 172.21.15.7:0/4135569556 learned_addr learned my addr 172.21.15.7:0/4135569556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd61000700 1 -- 172.21.15.7:0/4135569556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0746c0 msgr2=0x7fdd5c19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd61000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd61000700 1 -- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 msgr2=0x7fdd5c19b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.614+0000 7fdd61000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd61000700 1 -- 172.21.15.7:0/4135569556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c19ffc0 con 0x7fdd5c10eef0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd53400700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd61000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c19f880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c06ad20 tx=0x7fdd5400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540092e0 con 0x7fdd5c10eef0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5400c940 con 0x7fdd5c10eef0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5c1a0250 con 0x7fdd5c10eef0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540045e0 con 0x7fdd5c10eef0 2024-06-13T06:50:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.615+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5c1a7b30 con 0x7fdd5c10eef0 2024-06-13T06:50:14.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.616+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5c075de0 con 0x7fdd5c10eef0 2024-06-13T06:50:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.618+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdd5401b030 con 0x7fdd5c10eef0 2024-06-13T06:50:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.620+0000 7fdd59600700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 0x7fdd400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.620+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fdd540ce570 con 0x7fdd5c10eef0 2024-06-13T06:50:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.620+0000 7fdd5be00700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 0x7fdd400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.621+0000 7fdd5be00700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 0x7fdd400a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c006010 tx=0x7fdd4c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.624+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdd54022020 con 0x7fdd5c10eef0 2024-06-13T06:50:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:14.855+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fdd5c02dca0 con 0x7fdd5c10eef0 2024-06-13T06:50:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.459+0000 7fdd59600700 1 -- 172.21.15.7:0/4135569556 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7fdd54010040 con 0x7fdd5c10eef0 2024-06-13T06:50:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.466+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 msgr2=0x7fdd400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.466+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 0x7fdd400a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c006010 tx=0x7fdd4c00b260 comp rx=0 tx=0).stop 2024-06-13T06:50:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.466+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 msgr2=0x7fdd5c19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.466+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c19f880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c06ad20 tx=0x7fdd5400c3a0 comp rx=0 tx=0).stop 2024-06-13T06:50:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.467+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 shutdown_connections 2024-06-13T06:50:15.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.467+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c10eef0 0x7fdd5c19f880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.467+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdd400a2ca0 0x7fdd400a5160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.467+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0746c0 0x7fdd5c19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.467+0000 7fdd63000700 1 --2- 172.21.15.7:0/4135569556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd5c074fe0 0x7fdd5c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.468+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 >> 172.21.15.7:0/4135569556 conn(0x7fdd5c0795e0 msgr2=0x7fdd5c1009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.469+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 shutdown_connections 2024-06-13T06:50:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:15.469+0000 7fdd63000700 1 -- 172.21.15.7:0/4135569556 wait complete. 2024-06-13T06:50:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-06-13T06:50:15.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:15 smithi007 ceph-mon[32077]: pgmap v322: 33 pgs: 1 creating+activating, 6 creating+peering, 11 unknown, 15 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:15 smithi189 ceph-mon[36616]: pgmap v322: 33 pgs: 1 creating+activating, 6 creating+peering, 11 unknown, 15 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:15 smithi175 ceph-mon[36886]: pgmap v322: 33 pgs: 1 creating+activating, 6 creating+peering, 11 unknown, 15 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:16.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-13T06:50:16.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:16 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4135569556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:16 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:16 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:50:16.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:16 smithi007 ceph-mon[32077]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:50:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:16 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4135569556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:16 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:16 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:50:16.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:16 smithi189 ceph-mon[36616]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:50:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:16 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4135569556' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:16 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-13T06:50:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:16 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-13T06:50:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:16 smithi175 ceph-mon[36886]: osdmap e86: 12 total, 12 up, 12 in 2024-06-13T06:50:16.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:17.566 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:17 smithi007 ceph-mon[32077]: pgmap v324: 97 pgs: 64 unknown, 1 creating+activating, 6 creating+peering, 26 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:17.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:17 smithi007 ceph-mon[32077]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:50:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:17 smithi189 ceph-mon[36616]: pgmap v324: 97 pgs: 64 unknown, 1 creating+activating, 6 creating+peering, 26 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:17 smithi189 ceph-mon[36616]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:50:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:17 smithi175 ceph-mon[36886]: pgmap v324: 97 pgs: 64 unknown, 1 creating+activating, 6 creating+peering, 26 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:17 smithi175 ceph-mon[36886]: osdmap e87: 12 total, 12 up, 12 in 2024-06-13T06:50:18.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:18 smithi007 ceph-mon[32077]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:50:18.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:18 smithi007 ceph-mon[32077]: pgmap v327: 97 pgs: 41 unknown, 15 creating+peering, 41 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:18.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:18 smithi189 ceph-mon[36616]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:50:18.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:18 smithi189 ceph-mon[36616]: pgmap v327: 97 pgs: 41 unknown, 15 creating+peering, 41 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:18.869 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:18 smithi175 ceph-mon[36886]: osdmap e88: 12 total, 12 up, 12 in 2024-06-13T06:50:18.869 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:18 smithi175 ceph-mon[36886]: pgmap v327: 97 pgs: 41 unknown, 15 creating+peering, 41 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 -- 172.21.15.7:0/343252758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c10ebe0 msgr2=0x7f828c111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 --2- 172.21.15.7:0/343252758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c10ebe0 0x7f828c111050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f828400b0a0 tx=0x7f828400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 -- 172.21.15.7:0/343252758 shutdown_connections 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 --2- 172.21.15.7:0/343252758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c10ebe0 0x7f828c111050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 --2- 172.21.15.7:0/343252758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 0x7f828c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 --2- 172.21.15.7:0/343252758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c0751e0 0x7f828c10e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 -- 172.21.15.7:0/343252758 >> 172.21.15.7:0/343252758 conn(0x7f828c0fdeb0 msgr2=0x7f828c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 -- 172.21.15.7:0/343252758 shutdown_connections 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.120+0000 7f8291400700 1 -- 172.21.15.7:0/343252758 wait complete. 2024-06-13T06:50:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 Processor -- start 2024-06-13T06:50:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 -- start start 2024-06-13T06:50:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 0x7f828c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 0x7f828c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 0x7f828c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1148e0 con 0x7f828c0751e0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f828c114760 con 0x7f828c10ebe0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f8291400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f828c114a60 con 0x7f828c0748c0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.122+0000 7f828b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 0x7f828c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f828b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 0x7f828c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36550/0 (socket says 172.21.15.7:36550) 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f828b400700 1 -- 172.21.15.7:0/626449167 learned_addr learned my addr 172.21.15.7:0/626449167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 0x7f828c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f828be00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 0x7f828c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 -- 172.21.15.7:0/626449167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 msgr2=0x7f828c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 0x7f828c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 -- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 msgr2=0x7f828c1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 0x7f828c1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 -- 172.21.15.7:0/626449167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c19b400 con 0x7f828c10ebe0 2024-06-13T06:50:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.123+0000 7f8290a00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 0x7f828c19ab20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f828400b0a0 tx=0x7f8284004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.124+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284021070 con 0x7f828c10ebe0 2024-06-13T06:50:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.124+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8284007b80 con 0x7f828c10ebe0 2024-06-13T06:50:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.124+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c19b690 con 0x7f828c10ebe0 2024-06-13T06:50:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.124+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284005040 con 0x7f828c10ebe0 2024-06-13T06:50:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.124+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c1a7800 con 0x7f828c10ebe0 2024-06-13T06:50:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.125+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828c075f70 con 0x7f828c10ebe0 2024-06-13T06:50:19.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.127+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8284024070 con 0x7f828c10ebe0 2024-06-13T06:50:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.129+0000 7f8288c00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 0x7f82700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:19.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.129+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f82840cfd90 con 0x7f828c10ebe0 2024-06-13T06:50:19.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.129+0000 7f828be00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 0x7f82700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:19.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.130+0000 7f828be00700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 0x7f82700a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f827c009c60 tx=0x7f827c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.131+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8284007d30 con 0x7f828c10ebe0 2024-06-13T06:50:19.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.410+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f828c02dca0 con 0x7f828c10ebe0 2024-06-13T06:50:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.502+0000 7f8288c00700 1 -- 172.21.15.7:0/626449167 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f828409ce90 con 0x7f828c10ebe0 2024-06-13T06:50:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.506+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 msgr2=0x7f82700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 0x7f82700a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f827c009c60 tx=0x7f827c009260 comp rx=0 tx=0).stop 2024-06-13T06:50:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 msgr2=0x7f828c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 0x7f828c19ab20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f828400b0a0 tx=0x7f8284004d00 comp rx=0 tx=0).stop 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 shutdown_connections 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f82700a2c20 0x7f82700a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828c10ebe0 0x7f828c19ab20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0748c0 0x7f828c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 --2- 172.21.15.7:0/626449167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f828c0751e0 0x7f828c1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.507+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 >> 172.21.15.7:0/626449167 conn(0x7f828c0fdeb0 msgr2=0x7f828c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.510+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 shutdown_connections 2024-06-13T06:50:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:19.510+0000 7f8291400700 1 -- 172.21.15.7:0/626449167 wait complete. 2024-06-13T06:50:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-06-13T06:50:19.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:19 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/626449167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:19 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:19 smithi007 ceph-mon[32077]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-13T06:50:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:19 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/626449167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:19 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:19 smithi189 ceph-mon[36616]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-13T06:50:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:19 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/626449167' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:19 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-13T06:50:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:19 smithi175 ceph-mon[36886]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-13T06:50:20.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:50:20.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:50:20.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-13T06:50:20.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-13T06:50:20.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: pgmap v329: 97 pgs: 25 unknown, 15 creating+peering, 57 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:20.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:20.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:active 2024-06-13T06:50:20.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:20 smithi007 ceph-mon[32077]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:20.521 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-13T06:50:20.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: pgmap v329: 97 pgs: 25 unknown, 15 creating+peering, 57 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:20.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:20.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:active 2024-06-13T06:50:20.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:20 smithi189 ceph-mon[36616]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: osdmap e89: 12 total, 12 up, 12 in 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: pgmap v329: 97 pgs: 25 unknown, 15 creating+peering, 57 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] up:active 2024-06-13T06:50:20.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:20 smithi175 ceph-mon[36886]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:21.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:21 smithi007 ceph-mon[32077]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:50:21.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:21 smithi007 ceph-mon[32077]: Cluster is now healthy 2024-06-13T06:50:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:21 smithi189 ceph-mon[36616]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:50:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:21 smithi189 ceph-mon[36616]: Cluster is now healthy 2024-06-13T06:50:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:21 smithi175 ceph-mon[36886]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-13T06:50:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:21 smithi175 ceph-mon[36886]: Cluster is now healthy 2024-06-13T06:50:22.616 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:22 smithi007 ceph-mon[32077]: pgmap v330: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-13T06:50:22.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:22 smithi189 ceph-mon[36616]: pgmap v330: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-13T06:50:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:22 smithi175 ceph-mon[36886]: pgmap v330: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-06-13T06:50:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 -- 172.21.15.7:0/317593807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104102740 msgr2=0x7f3104102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 --2- 172.21.15.7:0/317593807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104102740 0x7f3104102b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30f800b0d0 tx=0x7f30f800b3e0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 -- 172.21.15.7:0/317593807 shutdown_connections 2024-06-13T06:50:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 --2- 172.21.15.7:0/317593807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 0x7f310410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 --2- 172.21.15.7:0/317593807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104102740 0x7f3104102b20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 --2- 172.21.15.7:0/317593807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31041030f0 0x7f3104107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 -- 172.21.15.7:0/317593807 >> 172.21.15.7:0/317593807 conn(0x7f31040795f0 msgr2=0x7f3104079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 -- 172.21.15.7:0/317593807 shutdown_connections 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.184+0000 7f310c200700 1 -- 172.21.15.7:0/317593807 wait complete. 2024-06-13T06:50:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.185+0000 7f310c200700 1 Processor -- start 2024-06-13T06:50:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 -- start start 2024-06-13T06:50:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 0x7f3104074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 0x7f3104074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 0x7f3104075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3104112760 con 0x7f31041030f0 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31041125e0 con 0x7f3104107840 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f310c200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31041128e0 con 0x7f3104102740 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f3108e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 0x7f3104074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.186+0000 7f3109800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 0x7f3104074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3108e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 0x7f3104074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38202/0 (socket says 172.21.15.7:38202) 2024-06-13T06:50:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3108e00700 1 -- 172.21.15.7:0/187594706 learned_addr learned my addr 172.21.15.7:0/187594706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:23.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f310a200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 0x7f3104075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:23.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 -- 172.21.15.7:0/187594706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 msgr2=0x7f3104075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 0x7f3104075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 -- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 msgr2=0x7f3104074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 0x7f3104074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 -- 172.21.15.7:0/187594706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041a7120 con 0x7f3104102740 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.187+0000 7f3109800700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 0x7f3104074790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30f800d010 tx=0x7f30f8007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:23.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.188+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8003c00 con 0x7f3104102740 2024-06-13T06:50:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.188+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8022070 con 0x7f3104102740 2024-06-13T06:50:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.188+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041a73b0 con 0x7f3104102740 2024-06-13T06:50:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.188+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f801b6c0 con 0x7f3104102740 2024-06-13T06:50:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.188+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a7800 con 0x7f3104102740 2024-06-13T06:50:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.189+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310410aa80 con 0x7f3104102740 2024-06-13T06:50:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.191+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f30f801b860 con 0x7f3104102740 2024-06-13T06:50:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.192+0000 7f3102000700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 0x7f30ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.193+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f30f800f020 con 0x7f3104102740 2024-06-13T06:50:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.193+0000 7f3108e00700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 0x7f30ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.193+0000 7f3108e00700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 0x7f30ec0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3104100f60 tx=0x7f30f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:23.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.195+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f30f809cf70 con 0x7f3104102740 2024-06-13T06:50:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.437+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f310410a460 con 0x7f3104102740 2024-06-13T06:50:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.504+0000 7f3102000700 1 -- 172.21.15.7:0/187594706 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7f30f809c870 con 0x7f3104102740 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 msgr2=0x7f30ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 0x7f30ec0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3104100f60 tx=0x7f30f0009260 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 msgr2=0x7f3104074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 0x7f3104074790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30f800d010 tx=0x7f30f8007a20 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 shutdown_connections 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30ec0a2c20 0x7f30ec0a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3104107840 0x7f3104075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3104102740 0x7f3104074790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 --2- 172.21.15.7:0/187594706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31041030f0 0x7f3104074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.508+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 >> 172.21.15.7:0/187594706 conn(0x7f31040795f0 msgr2=0x7f3104101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.509+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 shutdown_connections 2024-06-13T06:50:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:23.509+0000 7f310c200700 1 -- 172.21.15.7:0/187594706 wait complete. 2024-06-13T06:50:23.510 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:50:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T06:50:23.223+0000 7f97abd43700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-13T06:50:23.788 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:23 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/187594706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:23.789 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:23 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:23 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/187594706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:23 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:23.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:23 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/187594706' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:23.892 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:23 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-13T06:50:24.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-13T06:50:24.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:24 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:50:24.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:24 smithi007 ceph-mon[32077]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:50:24.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:24 smithi007 ceph-mon[32077]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-13T06:50:24.830 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:24.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:24 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:50:24.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:24 smithi189 ceph-mon[36616]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:50:24.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:24 smithi189 ceph-mon[36616]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-13T06:50:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:24 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-13T06:50:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:24 smithi175 ceph-mon[36886]: osdmap e90: 12 total, 12 up, 12 in 2024-06-13T06:50:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:24 smithi175 ceph-mon[36886]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-13T06:50:26.720 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:26 smithi007 ceph-mon[32077]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 -- 172.21.15.7:0/70754437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 msgr2=0x7f5ff810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 --2- 172.21.15.7:0/70754437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff810f3a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b0d0 tx=0x7f5ff000b3e0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 -- 172.21.15.7:0/70754437 shutdown_connections 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 --2- 172.21.15.7:0/70754437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff810f3a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 --2- 172.21.15.7:0/70754437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff81027a0 0x7f5ff8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 --2- 172.21.15.7:0/70754437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff8103150 0x7f5ff81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.015+0000 7f6000400700 1 -- 172.21.15.7:0/70754437 >> 172.21.15.7:0/70754437 conn(0x7f5ff8079560 msgr2=0x7f5ff8079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.016+0000 7f6000400700 1 -- 172.21.15.7:0/70754437 shutdown_connections 2024-06-13T06:50:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.016+0000 7f6000400700 1 -- 172.21.15.7:0/70754437 wait complete. 2024-06-13T06:50:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.017+0000 7f6000400700 1 Processor -- start 2024-06-13T06:50:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.017+0000 7f6000400700 1 -- start start 2024-06-13T06:50:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.017+0000 7f6000400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.017+0000 7f6000400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 0x7f5ff8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f6000400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:27.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f6000400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8112810 con 0x7f5ff8107790 2024-06-13T06:50:27.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f6000400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8112690 con 0x7f5ff81027a0 2024-06-13T06:50:27.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f6000400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8112990 con 0x7f5ff8103150 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffe400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffd000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 0x7f5ff8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36630/0 (socket says 172.21.15.7:36630) 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 -- 172.21.15.7:0/1358243137 learned_addr learned my addr 172.21.15.7:0/1358243137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 -- 172.21.15.7:0/1358243137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 msgr2=0x7f5ff8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 0x7f5ff8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 -- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 msgr2=0x7f5ff8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffda00700 1 -- 172.21.15.7:0/1358243137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81a7070 con 0x7f5ff81027a0 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffd000700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 0x7f5ff8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.018+0000 7f5ffe400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f5ffda00700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d950 tx=0x7f5fec00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec016070 con 0x7f5ff81027a0 2024-06-13T06:50:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec00fd70 con 0x7f5ff81027a0 2024-06-13T06:50:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00e610 con 0x7f5ff81027a0 2024-06-13T06:50:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81a7360 con 0x7f5ff81027a0 2024-06-13T06:50:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.019+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a7850 con 0x7f5ff81027a0 2024-06-13T06:50:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.020+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8050a00 con 0x7f5ff81027a0 2024-06-13T06:50:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.022+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5fec019070 con 0x7f5ff81027a0 2024-06-13T06:50:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.024+0000 7f5ff6000700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 0x7f5fe00a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.024+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5fec0cead0 con 0x7f5ff81027a0 2024-06-13T06:50:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.024+0000 7f5ffd000700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 0x7f5fe00a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.025+0000 7f5ffd000700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 0x7f5fe00a5200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8108a70 tx=0x7f5fe4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.026+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5fec09b9a0 con 0x7f5ff81027a0 2024-06-13T06:50:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:26 smithi189 ceph-mon[36616]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:26 smithi175 ceph-mon[36886]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:27.258+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5ff81a7bf0 con 0x7f5ff81027a0 2024-06-13T06:50:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:27 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1358243137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:27 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:27 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1358243137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:27 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:27 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1358243137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:27 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-13T06:50:28.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.696+0000 7f5ff6000700 1 -- 172.21.15.7:0/1358243137 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7f5fec09b2a0 con 0x7f5ff81027a0 2024-06-13T06:50:28.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.705+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 msgr2=0x7f5fe00a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.705+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 0x7f5fe00a5200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8108a70 tx=0x7f5fe4009260 comp rx=0 tx=0).stop 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.705+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 msgr2=0x7f5ff8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.705+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00d950 tx=0x7f5fec00dc60 comp rx=0 tx=0).stop 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 shutdown_connections 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ff8107790 0x7f5ff8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5fe00a2d40 0x7f5fe00a5200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff81027a0 0x7f5ff8074790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 --2- 172.21.15.7:0/1358243137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff8103150 0x7f5ff8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.706+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 >> 172.21.15.7:0/1358243137 conn(0x7f5ff8079560 msgr2=0x7f5ff81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:28.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.709+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 shutdown_connections 2024-06-13T06:50:28.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:28.709+0000 7f6000400700 1 -- 172.21.15.7:0/1358243137 wait complete. 2024-06-13T06:50:28.713 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-06-13T06:50:29.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:28 smithi007 ceph-mon[32077]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:29.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:28 smithi007 ceph-mon[32077]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:50:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:28 smithi189 ceph-mon[36616]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:28 smithi189 ceph-mon[36616]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:50:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:28 smithi175 ceph-mon[36886]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-13T06:50:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:28 smithi175 ceph-mon[36886]: osdmap e91: 12 total, 12 up, 12 in 2024-06-13T06:50:29.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-13T06:50:29.954 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:29 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:50:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:29 smithi189 ceph-mon[36616]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:50:30.107 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:29 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:50:30.108 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:29 smithi007 ceph-mon[32077]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:50:30.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:29 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-13T06:50:30.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:29 smithi175 ceph-mon[36886]: osdmap e92: 12 total, 12 up, 12 in 2024-06-13T06:50:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:30 smithi189 ceph-mon[36616]: pgmap v337: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:30 smithi189 ceph-mon[36616]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:30 smithi189 ceph-mon[36616]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:50:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:30 smithi175 ceph-mon[36886]: pgmap v337: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:30 smithi175 ceph-mon[36886]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:30 smithi175 ceph-mon[36886]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:50:31.151 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:30 smithi007 ceph-mon[32077]: pgmap v337: 129 pgs: 1 creating+peering, 31 unknown, 97 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:31.151 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:30 smithi007 ceph-mon[32077]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-13T06:50:31.151 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:30 smithi007 ceph-mon[32077]: osdmap e93: 12 total, 12 up, 12 in 2024-06-13T06:50:31.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:31 smithi007 ceph-mon[32077]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:50:32.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:31 smithi189 ceph-mon[36616]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:50:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:31 smithi175 ceph-mon[36886]: osdmap e94: 12 total, 12 up, 12 in 2024-06-13T06:50:32.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 -- 172.21.15.7:0/293379675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa700105070 msgr2=0x7fa700111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:32.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 --2- 172.21.15.7:0/293379675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa700105070 0x7fa700111580 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00b0a0 tx=0x7fa6fc00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 -- 172.21.15.7:0/293379675 shutdown_connections 2024-06-13T06:50:32.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 --2- 172.21.15.7:0/293379675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa700105070 0x7fa700111580 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 --2- 172.21.15.7:0/293379675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 --2- 172.21.15.7:0/293379675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7001046b0 0x7fa700104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 -- 172.21.15.7:0/293379675 >> 172.21.15.7:0/293379675 conn(0x7fa7001001a0 msgr2=0x7fa7001025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 -- 172.21.15.7:0/293379675 shutdown_connections 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.128+0000 7fa708800700 1 -- 172.21.15.7:0/293379675 wait complete. 2024-06-13T06:50:32.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.129+0000 7fa708800700 1 Processor -- start 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.129+0000 7fa708800700 1 -- start start 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7001046b0 0x7fa70019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa700114a40 con 0x7fa70010a6b0 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7001148c0 con 0x7fa700105070 2024-06-13T06:50:32.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa708800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa700114bc0 con 0x7fa7001046b0 2024-06-13T06:50:32.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa706800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:32.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:32.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56500/0 (socket says 172.21.15.7:56500) 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa706800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58290/0 (socket says 172.21.15.7:58290) 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 -- 172.21.15.7:0/3408127783 learned_addr learned my addr 172.21.15.7:0/3408127783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705e00700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7001046b0 0x7fa70019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 -- 172.21.15.7:0/3408127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7001046b0 msgr2=0x7fa70019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7001046b0 0x7fa70019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 -- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 msgr2=0x7fa70019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.130+0000 7fa705400700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa705400700 1 -- 172.21.15.7:0/3408127783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70019fdf0 con 0x7fa700105070 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa705400700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9a0 tx=0x7fa6f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000ca10 con 0x7fa700105070 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f000cbb0 con 0x7fa700105070 2024-06-13T06:50:32.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7001a00e0 con 0x7fa700105070 2024-06-13T06:50:32.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0018680 con 0x7fa700105070 2024-06-13T06:50:32.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.131+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7001a7960 con 0x7fa700105070 2024-06-13T06:50:32.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.133+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa700069180 con 0x7fa700105070 2024-06-13T06:50:32.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.135+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa6f0010430 con 0x7fa700105070 2024-06-13T06:50:32.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.136+0000 7fa6faa00700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 0x7fa6e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:32.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.137+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7fa6f00cf440 con 0x7fa700105070 2024-06-13T06:50:32.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.137+0000 7fa705e00700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 0x7fa6e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:32.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.138+0000 7fa705e00700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 0x7fa6e80a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005950 tx=0x7fa6f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.141+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa6f009c120 con 0x7fa700105070 2024-06-13T06:50:32.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.368+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa70002de60 con 0x7fa700105070 2024-06-13T06:50:32.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.756+0000 7fa6faa00700 1 -- 172.21.15.7:0/3408127783 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7fa6f009ba20 con 0x7fa700105070 2024-06-13T06:50:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.761+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 msgr2=0x7fa6e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.761+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 0x7fa6e80a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005950 tx=0x7fa6f400b410 comp rx=0 tx=0).stop 2024-06-13T06:50:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.761+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 msgr2=0x7fa70019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.761+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9a0 tx=0x7fa6f000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 shutdown_connections 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70010a6b0 0x7fa70019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7001046b0 0x7fa70019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6e80a2ca0 0x7fa6e80a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 --2- 172.21.15.7:0/3408127783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa700105070 0x7fa70019b370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 >> 172.21.15.7:0/3408127783 conn(0x7fa7001001a0 msgr2=0x7fa7001006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 shutdown_connections 2024-06-13T06:50:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:32.762+0000 7fa708800700 1 -- 172.21.15.7:0/3408127783 wait complete. 2024-06-13T06:50:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-06-13T06:50:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:32 smithi189 ceph-mon[36616]: pgmap v340: 129 pgs: 1 creating+peering, 16 unknown, 112 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3408127783' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:32 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:32 smithi175 ceph-mon[36886]: pgmap v340: 129 pgs: 1 creating+peering, 16 unknown, 112 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3408127783' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:32 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:32 smithi007 ceph-mon[32077]: pgmap v340: 129 pgs: 1 creating+peering, 16 unknown, 112 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:33.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3408127783' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:32 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-13T06:50:33.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-13T06:50:33.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:33 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:50:33.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:33 smithi007 ceph-mon[32077]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:50:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:34.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:33 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:50:34.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:33 smithi189 ceph-mon[36616]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:50:34.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:33 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-13T06:50:34.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:33 smithi175 ceph-mon[36886]: osdmap e95: 12 total, 12 up, 12 in 2024-06-13T06:50:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:34 smithi189 ceph-mon[36616]: pgmap v342: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:35.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:34 smithi007 ceph-mon[32077]: pgmap v342: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:34 smithi175 ceph-mon[36886]: pgmap v342: 129 pgs: 1 creating+peering, 128 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 -- 172.21.15.7:0/3490230892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 msgr2=0x7f1d5006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 --2- 172.21.15.7:0/3490230892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5006b040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d400098f0 tx=0x7f1d40009c00 comp rx=0 tx=0).stop 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 -- 172.21.15.7:0/3490230892 shutdown_connections 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 --2- 172.21.15.7:0/3490230892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d501077c0 0x7f1d5010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 --2- 172.21.15.7:0/3490230892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5006b040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 --2- 172.21.15.7:0/3490230892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d5006b610 0x7f1d50107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 -- 172.21.15.7:0/3490230892 >> 172.21.15.7:0/3490230892 conn(0x7f1d500781d0 msgr2=0x7f1d500785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 -- 172.21.15.7:0/3490230892 shutdown_connections 2024-06-13T06:50:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.366+0000 7f1d4f200700 1 -- 172.21.15.7:0/3490230892 wait complete. 2024-06-13T06:50:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.368+0000 7f1d4f200700 1 Processor -- start 2024-06-13T06:50:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.368+0000 7f1d4f200700 1 -- start start 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.368+0000 7f1d4f200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.368+0000 7f1d4f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.368+0000 7f1d4f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d501077c0 0x7f1d501989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501126f0 con 0x7f1d5006b610 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4f200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50112570 con 0x7f1d5006ac60 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50112870 con 0x7f1d501077c0 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58314/0 (socket says 172.21.15.7:58314) 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 -- 172.21.15.7:0/228156635 learned_addr learned my addr 172.21.15.7:0/228156635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4de00700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4e800700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d501077c0 0x7f1d501989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 -- 172.21.15.7:0/228156635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d501077c0 msgr2=0x7f1d501989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d501077c0 0x7f1d501989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 -- 172.21.15.7:0/228156635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 msgr2=0x7f1d5019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4d400700 1 -- 172.21.15.7:0/228156635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d50199150 con 0x7f1d5006b610 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.369+0000 7f1d4de00700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d4d400700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e940 tx=0x7f1d3c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00c9e0 con 0x7f1d5006b610 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d50199440 con 0x7f1d5006b610 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c00cb80 con 0x7f1d5006b610 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c018680 con 0x7f1d5006b610 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.370+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a56b0 con 0x7f1d5006b610 2024-06-13T06:50:36.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.371+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d50069180 con 0x7f1d5006b610 2024-06-13T06:50:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.375+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1d3c0188e0 con 0x7f1d5006b610 2024-06-13T06:50:36.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.376+0000 7f1d46a00700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 0x7f1d340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:36.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.376+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f1d3c0cf570 con 0x7f1d5006b610 2024-06-13T06:50:36.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.376+0000 7f1d4de00700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 0x7f1d340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:36.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.377+0000 7f1d4de00700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 0x7f1d340a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d50107290 tx=0x7f1d40019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:36.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.379+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d3c09c2b0 con 0x7f1d5006b610 2024-06-13T06:50:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.664+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1d50199ed0 con 0x7f1d5006b610 2024-06-13T06:50:36.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.768+0000 7f1d46a00700 1 -- 172.21.15.7:0/228156635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f1d3c09bbb0 con 0x7f1d5006b610 2024-06-13T06:50:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.773+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 msgr2=0x7f1d340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.773+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 0x7f1d340a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d50107290 tx=0x7f1d40019040 comp rx=0 tx=0).stop 2024-06-13T06:50:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.773+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 msgr2=0x7f1d5019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.773+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e940 tx=0x7f1d3c00ec50 comp rx=0 tx=0).stop 2024-06-13T06:50:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 shutdown_connections 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d501077c0 0x7f1d501989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d340a2ca0 0x7f1d340a5160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d5006ac60 0x7f1d5019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 --2- 172.21.15.7:0/228156635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d5006b610 0x7f1d5019ed20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 >> 172.21.15.7:0/228156635 conn(0x7f1d500781d0 msgr2=0x7f1d500ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 shutdown_connections 2024-06-13T06:50:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:36.775+0000 7f1d4f200700 1 -- 172.21.15.7:0/228156635 wait complete. 2024-06-13T06:50:36.776 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-06-13T06:50:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:36 smithi189 ceph-mon[36616]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:36 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:50:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:36 smithi175 ceph-mon[36886]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:36 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:50:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:36 smithi007 ceph-mon[32077]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:36 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-13T06:50:37.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:50:37.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:37 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:50:37.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:37 smithi007 ceph-mon[32077]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:50:37.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:37 smithi007 ceph-mon[32077]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:37 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:50:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:37 smithi189 ceph-mon[36616]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:50:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:37 smithi189 ceph-mon[36616]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:37 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/228156635' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-13T06:50:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:37 smithi175 ceph-mon[36886]: osdmap e96: 12 total, 12 up, 12 in 2024-06-13T06:50:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:37 smithi175 ceph-mon[36886]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:50:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:38 smithi007 ceph-mon[32077]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:38 smithi007 ceph-mon[32077]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:50:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:38 smithi007 ceph-mon[32077]: Cluster is now healthy 2024-06-13T06:50:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:38 smithi189 ceph-mon[36616]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:38 smithi189 ceph-mon[36616]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:50:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:38 smithi189 ceph-mon[36616]: Cluster is now healthy 2024-06-13T06:50:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:38 smithi175 ceph-mon[36886]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:38 smithi175 ceph-mon[36886]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-13T06:50:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:38 smithi175 ceph-mon[36886]: Cluster is now healthy 2024-06-13T06:50:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 -- 172.21.15.7:0/1023995602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 msgr2=0x7f02e810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 --2- 172.21.15.7:0/1023995602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e810aa90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009990 tx=0x7f02d8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 -- 172.21.15.7:0/1023995602 shutdown_connections 2024-06-13T06:50:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 --2- 172.21.15.7:0/1023995602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e8105070 0x7f02e8111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 --2- 172.21.15.7:0/1023995602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e810aa90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 --2- 172.21.15.7:0/1023995602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e81046b0 0x7f02e8104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 -- 172.21.15.7:0/1023995602 >> 172.21.15.7:0/1023995602 conn(0x7f02e81001a0 msgr2=0x7f02e81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 -- 172.21.15.7:0/1023995602 shutdown_connections 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.430+0000 7f02ee800700 1 -- 172.21.15.7:0/1023995602 wait complete. 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.431+0000 7f02ee800700 1 Processor -- start 2024-06-13T06:50:40.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 -- start start 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 0x7f02e819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 0x7f02e819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8114a40 con 0x7f02e810a6b0 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02e81148c0 con 0x7f02e81046b0 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.432+0000 7f02ee800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8114bc0 con 0x7f02e8105070 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 0x7f02e819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 0x7f02e819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58340/0 (socket says 172.21.15.7:58340) 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 0x7f02e819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56552/0 (socket says 172.21.15.7:56552) 2024-06-13T06:50:40.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7e00700 1 -- 172.21.15.7:0/567572131 learned_addr learned my addr 172.21.15.7:0/567572131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:40.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 -- 172.21.15.7:0/567572131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 msgr2=0x7f02e819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:40.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 0x7f02e819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 -- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 msgr2=0x7f02e819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02dea00700 1 -- 172.21.15.7:0/567572131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e819fe60 con 0x7f02e8105070 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.433+0000 7f02e7e00700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02dea00700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 0x7f02e819b390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d750 tx=0x7f02d000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0014030 con 0x7f02e8105070 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e81a0150 con 0x7f02e8105070 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d0004500 con 0x7f02e8105070 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005400 con 0x7f02e8105070 2024-06-13T06:50:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.434+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81a79d0 con 0x7f02e8105070 2024-06-13T06:50:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.435+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e8069180 con 0x7f02e8105070 2024-06-13T06:50:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.437+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f02d0004750 con 0x7f02e8105070 2024-06-13T06:50:40.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.439+0000 7f02e4c00700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 0x7f02c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.439+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f02d00cebc0 con 0x7f02e8105070 2024-06-13T06:50:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.439+0000 7f02e7400700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 0x7f02c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.440+0000 7f02e7400700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 0x7f02c80a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.443+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02d009b880 con 0x7f02e8105070 2024-06-13T06:50:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.728+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02e802dca0 con 0x7f02e8105070 2024-06-13T06:50:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.729+0000 7f02e4c00700 1 -- 172.21.15.7:0/567572131 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11797 (secure 0 0 0) 0x7f02d009b180 con 0x7f02e8105070 2024-06-13T06:50:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:40.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:50:36.667450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.732+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 msgr2=0x7f02c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.732+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 0x7f02c80a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02d8006010 tx=0x7f02d800b260 comp rx=0 tx=0).stop 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 msgr2=0x7f02e819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 0x7f02e819b390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d750 tx=0x7f02d000da60 comp rx=0 tx=0).stop 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 shutdown_connections 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e810a6b0 0x7f02e819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02c80a2c20 0x7f02c80a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02e81046b0 0x7f02e819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 --2- 172.21.15.7:0/567572131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02e8105070 0x7f02e819b390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 >> 172.21.15.7:0/567572131 conn(0x7f02e81001a0 msgr2=0x7f02e81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 shutdown_connections 2024-06-13T06:50:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:40.733+0000 7f02ee800700 1 -- 172.21.15.7:0/567572131 wait complete. 2024-06-13T06:50:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-06-13T06:50:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:40 smithi189 ceph-mon[36616]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:40 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/567572131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:40 smithi175 ceph-mon[36886]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:40 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/567572131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:40 smithi007 ceph-mon[32077]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:41.137 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:40 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/567572131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:41.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:50:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:43.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:42 smithi007 ceph-mon[32077]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:42 smithi189 ceph-mon[36616]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:42 smithi175 ceph-mon[36886]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 -- 172.21.15.7:0/2109016633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07681084f0 msgr2=0x7f07681088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 --2- 172.21.15.7:0/2109016633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07681084f0 0x7f07681088d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0758009990 tx=0x7f0758009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 -- 172.21.15.7:0/2109016633 shutdown_connections 2024-06-13T06:50:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 --2- 172.21.15.7:0/2109016633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f076806b3c0 0x7f076810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 --2- 172.21.15.7:0/2109016633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07681084f0 0x7f07681088d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 --2- 172.21.15.7:0/2109016633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 -- 172.21.15.7:0/2109016633 >> 172.21.15.7:0/2109016633 conn(0x7f0768078300 msgr2=0x7f0768078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 -- 172.21.15.7:0/2109016633 shutdown_connections 2024-06-13T06:50:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.254+0000 7f076d000700 1 -- 172.21.15.7:0/2109016633 wait complete. 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.255+0000 7f076d000700 1 Processor -- start 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 -- start start 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076806b3c0 0x7f076819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07681084f0 0x7f0768198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07681127a0 con 0x7f07681084f0 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0768112620 con 0x7f076806b3c0 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f076d000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0768112920 con 0x7f076806aa00 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f0765e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f075fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076806b3c0 0x7f076819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f0765e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38452/0 (socket says 172.21.15.7:38452) 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.256+0000 7f0765e00700 1 -- 172.21.15.7:0/4110658738 learned_addr learned my addr 172.21.15.7:0/4110658738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0766800700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07681084f0 0x7f0768198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 -- 172.21.15.7:0/4110658738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076806b3c0 msgr2=0x7f076819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076806b3c0 0x7f076819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 -- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07681084f0 msgr2=0x7f0768198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07681084f0 0x7f0768198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 -- 172.21.15.7:0/4110658738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0768199240 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.257+0000 7f0765e00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f075800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.258+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758027070 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.258+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07580056d0 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.258+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07681994d0 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.258+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758022e10 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.258+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07681a57a0 con 0x7f076806aa00 2024-06-13T06:50:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.259+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f076810a9d0 con 0x7f076806aa00 2024-06-13T06:50:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.261+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f075802a070 con 0x7f076806aa00 2024-06-13T06:50:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.262+0000 7f075ea00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 0x7f074c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.263+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f07580cfef0 con 0x7f076806aa00 2024-06-13T06:50:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.263+0000 7f075fe00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 0x7f074c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.263+0000 7f075fe00700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 0x7f074c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07680788e0 tx=0x7f0750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.267+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0758013070 con 0x7f076806aa00 2024-06-13T06:50:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.497+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0768199fc0 con 0x7f076806aa00 2024-06-13T06:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.499+0000 7f075ea00700 1 -- 172.21.15.7:0/4110658738 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24071 (secure 0 0 0) 0x7f075809cc40 con 0x7f076806aa00 2024-06-13T06:50:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:44.500 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:50:36.667248+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:50:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 msgr2=0x7f074c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 0x7f074c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07680788e0 tx=0x7f0750009260 comp rx=0 tx=0).stop 2024-06-13T06:50:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 msgr2=0x7f076819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:44.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f075800bab0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 shutdown_connections 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07681084f0 0x7f0768198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076806aa00 0x7f076819e8d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f074c0a2c20 0x7f074c0a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 --2- 172.21.15.7:0/4110658738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f076806b3c0 0x7f076819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 >> 172.21.15.7:0/4110658738 conn(0x7f0768078300 msgr2=0x7f0768100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 shutdown_connections 2024-06-13T06:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:44.503+0000 7f076d000700 1 -- 172.21.15.7:0/4110658738 wait complete. 2024-06-13T06:50:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:44 smithi189 ceph-mon[36616]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:44 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4110658738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:44 smithi175 ceph-mon[36886]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:44 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4110658738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:45.157 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:44 smithi007 ceph-mon[32077]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:45.157 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:44 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4110658738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:45.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:50:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:46.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:46 smithi007 ceph-mon[32077]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:46 smithi189 ceph-mon[36616]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:46 smithi175 ceph-mon[36886]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 -- 172.21.15.7:0/1266531328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4105070 msgr2=0x7efec4111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 --2- 172.21.15.7:0/1266531328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4105070 0x7efec4111580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efebc009f90 tx=0x7efebc00b330 comp rx=0 tx=0).stop 2024-06-13T06:50:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 -- 172.21.15.7:0/1266531328 shutdown_connections 2024-06-13T06:50:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 --2- 172.21.15.7:0/1266531328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4105070 0x7efec4111580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 --2- 172.21.15.7:0/1266531328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec410a6b0 0x7efec410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 --2- 172.21.15.7:0/1266531328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec4104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 -- 172.21.15.7:0/1266531328 >> 172.21.15.7:0/1266531328 conn(0x7efec41001c0 msgr2=0x7efec41025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:48.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 -- 172.21.15.7:0/1266531328 shutdown_connections 2024-06-13T06:50:48.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.330+0000 7efec8c00700 1 -- 172.21.15.7:0/1266531328 wait complete. 2024-06-13T06:50:48.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.331+0000 7efec8c00700 1 Processor -- start 2024-06-13T06:50:48.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.331+0000 7efec8c00700 1 -- start start 2024-06-13T06:50:48.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec4105070 0x7efec419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efec4114a40 con 0x7efec410a6b0 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efec41148c0 con 0x7efec41046b0 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec8c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efec4114bc0 con 0x7efec4105070 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec1e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38692/0 (socket says 172.21.15.7:38692) 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec2800700 1 -- 172.21.15.7:0/3403909334 learned_addr learned my addr 172.21.15.7:0/3403909334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec1400700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec4105070 0x7efec419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec2800700 1 -- 172.21.15.7:0/3403909334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec4105070 msgr2=0x7efec419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.332+0000 7efec2800700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec4105070 0x7efec419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec2800700 1 -- 172.21.15.7:0/3403909334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 msgr2=0x7efec419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:48.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec2800700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec2800700 1 -- 172.21.15.7:0/3403909334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec419fe60 con 0x7efec410a6b0 2024-06-13T06:50:48.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec1e00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:48.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec2800700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efebc009550 tx=0x7efebc00cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc021070 con 0x7efec410a6b0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc007ac0 con 0x7efec410a6b0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a00f0 con 0x7efec410a6b0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc0052a0 con 0x7efec410a6b0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.333+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a79d0 con 0x7efec410a6b0 2024-06-13T06:50:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.334+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4069180 con 0x7efec410a6b0 2024-06-13T06:50:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.337+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efebc024070 con 0x7efec410a6b0 2024-06-13T06:50:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.338+0000 7efebaa00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 0x7efea80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.338+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7efebc00f020 con 0x7efec410a6b0 2024-06-13T06:50:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.338+0000 7efec1e00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 0x7efea80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.339+0000 7efec1e00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 0x7efea80a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009fd0 tx=0x7efeb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.341+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efebc02b020 con 0x7efec410a6b0 2024-06-13T06:50:48.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.578+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efec402dca0 con 0x7efec410a6b0 2024-06-13T06:50:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.580+0000 7efebaa00700 1 -- 172.21.15.7:0/3403909334 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24071 (secure 0 0 0) 0x7efebc09d8d0 con 0x7efec410a6b0 2024-06-13T06:50:48.580 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:48.580 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:50:36.667248+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:50:48.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.583+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 msgr2=0x7efea80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:48.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.583+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 0x7efea80a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009fd0 tx=0x7efeb4009260 comp rx=0 tx=0).stop 2024-06-13T06:50:48.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.583+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 msgr2=0x7efec419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:48.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.583+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efebc009550 tx=0x7efebc00cfa0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 shutdown_connections 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec410a6b0 0x7efec419f720 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efea80a2c20 0x7efea80a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efec41046b0 0x7efec419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 --2- 172.21.15.7:0/3403909334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efec4105070 0x7efec419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 >> 172.21.15.7:0/3403909334 conn(0x7efec41001c0 msgr2=0x7efec41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 shutdown_connections 2024-06-13T06:50:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:48.584+0000 7efec8c00700 1 -- 172.21.15.7:0/3403909334 wait complete. 2024-06-13T06:50:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:48 smithi007 ceph-mon[32077]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:48 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3403909334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:48 smithi189 ceph-mon[36616]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:48 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3403909334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:48 smithi175 ceph-mon[36886]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:48 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3403909334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:49.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:50:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:51.106 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:50 smithi007 ceph-mon[32077]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:50 smithi175 ceph-mon[36886]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:50 smithi189 ceph-mon[36616]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 -- 172.21.15.7:0/3783322656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 msgr2=0x7f1d98103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/3783322656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d98103960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009990 tx=0x7f1d88009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 -- 172.21.15.7:0/3783322656 shutdown_connections 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/3783322656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d9810c9b0 0x7f1d9810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/3783322656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d98103960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/3783322656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d98103ea0 0x7f1d9810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 -- 172.21.15.7:0/3783322656 >> 172.21.15.7:0/3783322656 conn(0x7f1d980fcd90 msgr2=0x7f1d980ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 -- 172.21.15.7:0/3783322656 shutdown_connections 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.339+0000 7f1d9dc00700 1 -- 172.21.15.7:0/3783322656 wait complete. 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.341+0000 7f1d9dc00700 1 Processor -- start 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.341+0000 7f1d9dc00700 1 -- start start 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.341+0000 7f1d9dc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.341+0000 7f1d9dc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d98103ea0 0x7f1d9819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d9dc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9810c9b0 0x7f1d98198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d9dc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98112740 con 0x7f1d98103580 2024-06-13T06:50:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d9dc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981125c0 con 0x7f1d98103ea0 2024-06-13T06:50:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d9dc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981128c0 con 0x7f1d9810c9b0 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42232/0 (socket says 172.21.15.7:42232) 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 -- 172.21.15.7:0/4029053740 learned_addr learned my addr 172.21.15.7:0/4029053740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97400700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d98103ea0 0x7f1d9819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d9d200700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9810c9b0 0x7f1d98198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 -- 172.21.15.7:0/4029053740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9810c9b0 msgr2=0x7f1d98198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9810c9b0 0x7f1d98198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 -- 172.21.15.7:0/4029053740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d98103ea0 msgr2=0x7f1d9819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d98103ea0 0x7f1d9819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.342+0000 7f1d97e00700 1 -- 172.21.15.7:0/4029053740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981991c0 con 0x7f1d98103580 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d97e00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d8800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88027070 con 0x7f1d98103580 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d880056d0 con 0x7f1d98103580 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88022e10 con 0x7f1d98103580 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98199450 con 0x7f1d98103580 2024-06-13T06:50:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.343+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981a5720 con 0x7f1d98103580 2024-06-13T06:50:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.345+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98069180 con 0x7f1d98103580 2024-06-13T06:50:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.346+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1d8802a070 con 0x7f1d98103580 2024-06-13T06:50:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.348+0000 7f1d94c00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 0x7f1d7c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.348+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f1d880cfea0 con 0x7f1d98103580 2024-06-13T06:50:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.348+0000 7f1d97400700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 0x7f1d7c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.349+0000 7f1d97400700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 0x7f1d7c0a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d981008b0 tx=0x7f1d84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.352+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d88013070 con 0x7f1d98103580 2024-06-13T06:50:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.620+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9802dca0 con 0x7f1d98103580 2024-06-13T06:50:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.622+0000 7f1d94c00700 1 -- 172.21.15.7:0/4029053740 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11797 (secure 0 0 0) 0x7f1d8809cbf0 con 0x7f1d98103580 2024-06-13T06:50:52.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:52.622 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:50:36.667450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-06-13T06:50:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 msgr2=0x7f1d7c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 0x7f1d7c0a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d981008b0 tx=0x7f1d84009260 comp rx=0 tx=0).stop 2024-06-13T06:50:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 msgr2=0x7f1d9819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d8800bab0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 shutdown_connections 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9810c9b0 0x7f1d98198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d98103580 0x7f1d9819e850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1d7c0a2ca0 0x7f1d7c0a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 --2- 172.21.15.7:0/4029053740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d98103ea0 0x7f1d9819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.625+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 >> 172.21.15.7:0/4029053740 conn(0x7f1d980fcd90 msgr2=0x7f1d980fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.626+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 shutdown_connections 2024-06-13T06:50:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:52.626+0000 7f1d9dc00700 1 -- 172.21.15.7:0/4029053740 wait complete. 2024-06-13T06:50:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-06-13T06:50:52.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:52 smithi007 ceph-mon[32077]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:52.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:52 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4029053740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:52 smithi175 ceph-mon[36886]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:52 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4029053740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:52 smithi189 ceph-mon[36616]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:52 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4029053740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:50:53.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:50:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:55.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:54 smithi175 ceph-mon[36886]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:55.118 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:54 smithi007 ceph-mon[32077]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:54 smithi189 ceph-mon[36616]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:56.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.195+0000 7f7426400700 1 -- 172.21.15.7:0/3548572606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420113660 msgr2=0x7f7420115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:56.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.195+0000 7f7426400700 1 --2- 172.21.15.7:0/3548572606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420113660 0x7f7420115a50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f741800b0a0 tx=0x7f741800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 -- 172.21.15.7:0/3548572606 shutdown_connections 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 --2- 172.21.15.7:0/3548572606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420113660 0x7f7420115a50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 --2- 172.21.15.7:0/3548572606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74200746c0 0x7f7420074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 --2- 172.21.15.7:0/3548572606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7420074fe0 0x7f7420113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 -- 172.21.15.7:0/3548572606 >> 172.21.15.7:0/3548572606 conn(0x7f74200795f0 msgr2=0x7f7420079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 -- 172.21.15.7:0/3548572606 shutdown_connections 2024-06-13T06:50:56.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.196+0000 7f7426400700 1 -- 172.21.15.7:0/3548572606 wait complete. 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.197+0000 7f7426400700 1 Processor -- start 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.197+0000 7f7426400700 1 -- start start 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 0x7f742019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 0x7f742019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 0x7f74201a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:56.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7420118dc0 con 0x7f7420074fe0 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7420118c40 con 0x7f74200746c0 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7426400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7420118f40 con 0x7f7420113660 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7417e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 0x7f742019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7417e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 0x7f742019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42250/0 (socket says 172.21.15.7:42250) 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f7417e00700 1 -- 172.21.15.7:0/3817985540 learned_addr learned my addr 172.21.15.7:0/3817985540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f741f400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 0x7f742019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.198+0000 7f741fe00700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 0x7f74201a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 -- 172.21.15.7:0/3817985540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 msgr2=0x7f74201a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 0x7f74201a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 -- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 msgr2=0x7f742019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 0x7f742019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 -- 172.21.15.7:0/3817985540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74201a4250 con 0x7f74200746c0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741fe00700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 0x7f74201a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741f400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 0x7f742019f290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f741000e9f0 tx=0x7f741000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741000c9c0 con 0x7f74200746c0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741000cb60 con 0x7f74200746c0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74201a4540 con 0x7f74200746c0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410018680 con 0x7f74200746c0 2024-06-13T06:50:56.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.199+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74201abdc0 con 0x7f74200746c0 2024-06-13T06:50:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.200+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7420069180 con 0x7f74200746c0 2024-06-13T06:50:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.203+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7410010430 con 0x7f74200746c0 2024-06-13T06:50:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.204+0000 7f741cc00700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 0x7f74040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.204+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f74100cf740 con 0x7f74200746c0 2024-06-13T06:50:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.204+0000 7f7417e00700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 0x7f74040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:56.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.205+0000 7f7417e00700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 0x7f74040a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f742010feb0 tx=0x7f7408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.206+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f741009c400 con 0x7f74200746c0 2024-06-13T06:50:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.433+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74201abff0 con 0x7f74200746c0 2024-06-13T06:50:56.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.435+0000 7f741cc00700 1 -- 172.21.15.7:0/3817985540 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24071 (secure 0 0 0) 0x7f741009bd00 con 0x7f74200746c0 2024-06-13T06:50:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:50:56.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:50:36.667248+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:50:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 msgr2=0x7f74040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 0x7f74040a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f742010feb0 tx=0x7f7408009260 comp rx=0 tx=0).stop 2024-06-13T06:50:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 msgr2=0x7f742019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:56.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 0x7f742019f290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f741000e9f0 tx=0x7f741000ed00 comp rx=0 tx=0).stop 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 shutdown_connections 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7420113660 0x7f74201a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f74040a2c20 0x7f74040a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74200746c0 0x7f742019f290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 --2- 172.21.15.7:0/3817985540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7420074fe0 0x7f742019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 >> 172.21.15.7:0/3817985540 conn(0x7f74200795f0 msgr2=0x7f7420112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 shutdown_connections 2024-06-13T06:50:56.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:56.439+0000 7f7426400700 1 -- 172.21.15.7:0/3817985540 wait complete. 2024-06-13T06:50:57.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:56 smithi175 ceph-mon[36886]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:57.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:56 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3817985540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:56 smithi007 ceph-mon[32077]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:56 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3817985540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:57.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-13T06:50:19.413292+0000', 'modified': '2024-06-13T06:50:36.667450+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6839/4167274347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 4167274347}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 4167274347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24539, 'qdb_cluster': [24539]} 2024-06-13T06:50:57.286 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-13T06:50:57.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.b status 2024-06-13T06:50:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:56 smithi189 ceph-mon[36616]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:56 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3817985540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:50:57.763 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:50:59.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:58 smithi007 ceph-mon[32077]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:58 smithi175 ceph-mon[36886]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:59.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:58 smithi189 ceph-mon[36616]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:50:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 -- 172.21.15.7:0/282675492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d8108500 msgr2=0x7f31d81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 --2- 172.21.15.7:0/282675492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d8108500 0x7f31d81088e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009990 tx=0x7f31c8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 -- 172.21.15.7:0/282675492 shutdown_connections 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 --2- 172.21.15.7:0/282675492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806ab30 0x7f31d810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 --2- 172.21.15.7:0/282675492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d8108500 0x7f31d81088e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 --2- 172.21.15.7:0/282675492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d806a170 0x7f31d806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 -- 172.21.15.7:0/282675492 >> 172.21.15.7:0/282675492 conn(0x7f31d8078340 msgr2=0x7f31d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 -- 172.21.15.7:0/282675492 shutdown_connections 2024-06-13T06:50:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.914+0000 7f31dd600700 1 -- 172.21.15.7:0/282675492 wait complete. 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.915+0000 7f31dd600700 1 Processor -- start 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.915+0000 7f31dd600700 1 -- start start 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.915+0000 7f31dd600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.915+0000 7f31dd600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d806ab30 0x7f31d819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.915+0000 7f31dd600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d8108500 0x7f31d819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31dd600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81127d0 con 0x7f31d806ab30 2024-06-13T06:50:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31dd600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8112650 con 0x7f31d8108500 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31dd600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8112950 con 0x7f31d806a170 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d806ab30 0x7f31d819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54678/0 (socket says 172.21.15.7:54678) 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 -- 172.21.15.7:0/3550319439 learned_addr learned my addr 172.21.15.7:0/3550319439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d7400700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d8108500 0x7f31d819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 -- 172.21.15.7:0/3550319439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d8108500 msgr2=0x7f31d819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d8108500 0x7f31d819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 -- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d806ab30 msgr2=0x7f31d819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d806ab30 0x7f31d819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.916+0000 7f31d6a00700 1 -- 172.21.15.7:0/3550319439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d819fe70 con 0x7f31d806a170 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005910 tx=0x7f31c800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31cf400700 1 -- 172.21.15.7:0/3550319439 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8027070 con 0x7f31d806a170 2024-06-13T06:50:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31cf400700 1 -- 172.21.15.7:0/3550319439 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c8004650 con 0x7f31d806a170 2024-06-13T06:50:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d81a0100 con 0x7f31d806a170 2024-06-13T06:50:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31cf400700 1 -- 172.21.15.7:0/3550319439 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c800f460 con 0x7f31d806a170 2024-06-13T06:50:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.917+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d81a79e0 con 0x7f31d806a170 2024-06-13T06:50:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.921+0000 7f31cf400700 1 -- 172.21.15.7:0/3550319439 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f31c802a070 con 0x7f31d806a170 2024-06-13T06:50:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 Processor -- start 2024-06-13T06:50:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 -- start start 2024-06-13T06:50:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bf6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b059c0 con 0x55f4a7bfd460 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b48ab0 con 0x55f4a7bfcb40 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.922+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b038a0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31cf400700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 0x7f31bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bf6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bfcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bfcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60432/0 (socket says 172.21.15.7:60432) 2024-06-13T06:50:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d7400700 1 -- 172.21.15.7:0/933818989 learned_addr learned my addr 172.21.15.7:0/933818989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31cf400700 1 -- 172.21.15.7:0/3550319439 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9063+0+0 (secure 0 0 0) 0x7f31c80d02b0 con 0x7f31d806a170 2024-06-13T06:50:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d7400700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 0x7f31bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 -- 172.21.15.7:0/933818989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 msgr2=0x55f4a7bfcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bfcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 -- 172.21.15.7:0/933818989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 msgr2=0x55f4a7bf6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bf6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.923+0000 7f31d6000700 1 -- 172.21.15.7:0/933818989 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a7af0f40 con 0x55f4a7bf71f0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.924+0000 7f31d6000700 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf7630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009c80 tx=0x7f31c4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9e0aedcf484903 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.924+0000 7f31d7400700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 0x7f31bc0a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009fd0 tx=0x7f31d000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.924+0000 7f31c3e00700 1 -- 172.21.15.7:0/933818989 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4004310 con 0x55f4a7bf71f0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.924+0000 7f31c3e00700 1 -- 172.21.15.7:0/933818989 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4011e10 con 0x55f4a7bf71f0 2024-06-13T06:50:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.924+0000 7f31c3e00700 1 -- 172.21.15.7:0/933818989 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4005280 con 0x55f4a7bf71f0 2024-06-13T06:50:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/933818989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 msgr2=0x55f4a7bf7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf7630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009c80 tx=0x7f31c4009f90 comp rx=0 tx=0).stop 2024-06-13T06:50:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/933818989 shutdown_connections 2024-06-13T06:50:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf7630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bfcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/933818989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bf6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/933818989 >> 172.21.15.7:0/933818989 conn(0x55f4a7b23120 msgr2=0x55f4a7bf1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/933818989 shutdown_connections 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.925+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/933818989 wait complete. 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.926+0000 7f31e7f4fb80 1 -- start start 2024-06-13T06:50:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.926+0000 7f31e7f4fb80 1 Processor -- start 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7c031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7c009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7ae4f10 con 0x55f4a7bfd460 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b44a70 con 0x55f4a7bf71f0 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c03e40 con 0x55f4a7bfcb40 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31e7f4fb80 10 client.0 _ll_register_callbacks cb 0x55f4a7ade330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:50:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7c009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7c009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42292/0 (socket says 172.21.15.7:42292) 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d7400700 1 -- 172.21.15.7:0/3219361399 learned_addr learned my addr 172.21.15.7:0/3219361399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54694/0 (socket says 172.21.15.7:54694) 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.927+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7c031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 -- 172.21.15.7:0/3219361399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 msgr2=0x55f4a7c004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 -- 172.21.15.7:0/3219361399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 msgr2=0x55f4a7c009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7c009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 -- 172.21.15.7:0/3219361399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7a97880 con 0x55f4a7bf71f0 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6000700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31d6a00700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7c031a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8091ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63d113ecd7db5de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c809c340 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9063+0+0 (secure 0 0 0) 0x7f31c80870b0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4011e10 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f31c809c160 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31e7f4fb80 10 client.24635 fetch_fsmap learned FSMap version 16 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.928+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55f4a7c0e5a0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4011e10 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f31c809ca60 con 0x55f4a7bf71f0 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 10 client.24635 fetch_fsmap finished waiting for FSMap version 16 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 10 client.24635 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:50:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 20 client.24635 populate_metadata read hostname 'smithi007' 2024-06-13T06:50:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c10c00 0x55f4a7c12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 4 client.24635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:50:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a7b55fb0 con 0x55f4a7c10c00 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.929+0000 7f31d6000700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c10c00 0x55f4a7c12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.930+0000 7f31d6000700 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c10c00 0x55f4a7c12070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b67a5f2d5e3abc8 server_cookie=27014c58f0755be in_seq=0 out_seq=0 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.930+0000 7f31c0c00700 10 client.24635 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31c0c00700 1 -- 172.21.15.7:0/3219361399 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f4a7b55fb0 con 0x55f4a7c10c00 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31c0c00700 10 client.24635 handle_command_reply: tid=0 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 client.24635 shutdown 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 msgr2=0x55f4a7c031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7c031a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f31c8000c00 tx=0x7f31c8091ce0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 shutdown_connections 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c10c00 0x55f4a7c12070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7c009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7c031a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/3219361399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7c004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31c0c00700 0 client.24635 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 >> 172.21.15.7:0/3219361399 conn(0x55f4a7b23120 msgr2=0x55f4a7beb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.931+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 shutdown_connections 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.932+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/3219361399 wait complete. 2024-06-13T06:50:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.932+0000 7f31e7f4fb80 20 client.24635 trim_cache size 0 max 16384 2024-06-13T06:50:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 Processor -- start 2024-06-13T06:50:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 -- start start 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfd460 0x55f4a7bfed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c103b0 con 0x55f4a7bfcb40 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c10a80 con 0x55f4a7bf71f0 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.955+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c11150 con 0x55f4a7bfd460 2024-06-13T06:50:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60444/0 (socket says 172.21.15.7:60444) 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 -- 172.21.15.7:0/1760009073 learned_addr learned my addr 172.21.15.7:0/1760009073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6000700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d7400700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfd460 0x55f4a7bfed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 -- 172.21.15.7:0/1760009073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfd460 msgr2=0x55f4a7bfed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfd460 0x55f4a7bfed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 -- 172.21.15.7:0/1760009073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 msgr2=0x55f4a7bec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 -- 172.21.15.7:0/1760009073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a7bf2bc0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6000700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:50:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31d6a00700 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31c8099180 tx=0x7f31c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6b15448ed9f5e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.956+0000 7f31c3400700 1 -- 172.21.15.7:0/1760009073 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80043d0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31c3400700 1 -- 172.21.15.7:0/1760009073 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c80b5070 con 0x55f4a7bf71f0 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31c3400700 1 -- 172.21.15.7:0/1760009073 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8087070 con 0x55f4a7bf71f0 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/1760009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 msgr2=0x55f4a7bf6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31c8099180 tx=0x7f31c8009310 comp rx=0 tx=0).stop 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/1760009073 shutdown_connections 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfd460 0x55f4a7bfed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7bf6cb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/1760009073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfcb40 0x55f4a7bec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/1760009073 >> 172.21.15.7:0/1760009073 conn(0x55f4a7b23120 msgr2=0x55f4a7c00460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/1760009073 shutdown_connections 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.957+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/1760009073 wait complete. 2024-06-13T06:50:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 -- start start 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 Processor -- start 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7beba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c11ef0 con 0x55f4a7bfd460 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c11820 con 0x55f4a7bf71f0 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.958+0000 7f31e7f4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7c125c0 con 0x55f4a7bfcb40 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31e7f4fb80 10 client.0 _ll_register_callbacks cb 0x55f4a7ade330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60446/0 (socket says 172.21.15.7:60446) 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 -- 172.21.15.7:0/2441076506 learned_addr learned my addr 172.21.15.7:0/2441076506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6000700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7beba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d7400700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 -- 172.21.15.7:0/2441076506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 msgr2=0x55f4a7beba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7beba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 -- 172.21.15.7:0/2441076506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 msgr2=0x55f4a7bebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6a00700 1 -- 172.21.15.7:0/2441076506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7c19b50 con 0x55f4a7bf71f0 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d6000700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7beba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.959+0000 7f31d7400700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31d6a00700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31c800be00 tx=0x7f31c809c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2626c308df58ab server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80a5150 con 0x55f4a7bf71f0 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9063+0+0 (secure 0 0 0) 0x7f31c809db30 con 0x55f4a7bf71f0 2024-06-13T06:50:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c80b5070 con 0x55f4a7bf71f0 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f31c80043d0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31e7f4fb80 10 client.24641 fetch_fsmap learned FSMap version 16 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55f4a7c19b50 con 0x55f4a7bf71f0 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c80b5070 con 0x55f4a7bf71f0 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.960+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f31c80670d0 con 0x55f4a7bf71f0 2024-06-13T06:50:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 10 client.24641 fetch_fsmap finished waiting for FSMap version 16 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 10 client.24641 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 20 client.24641 populate_metadata read hostname 'smithi007' 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c80cf0 0x55f4a7c7fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 4 client.24641 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "status"}) -- 0x55f4a7b50530 con 0x55f4a7c80cf0 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31d6000700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c80cf0 0x55f4a7c7fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31c0c00700 10 client.24641 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.961+0000 7f31d6000700 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c80cf0 0x55f4a7c7fa00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49ab4fd3f5984c2 server_cookie=fa5832eb7f910d80 in_seq=0 out_seq=0 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31c0c00700 1 -- 172.21.15.7:0/2441076506 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f4a7b50530 con 0x55f4a7c80cf0 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31c0c00700 10 client.24641 handle_command_reply: tid=0 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 client.24641 shutdown 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 msgr2=0x55f4a7beb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31c800be00 tx=0x7f31c809c800 comp rx=0 tx=0).stop 2024-06-13T06:50:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 shutdown_connections 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f4a7c80cf0 0x55f4a7c7fa00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4a7bfd460 0x55f4a7bebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f4a7bf71f0 0x55f4a7beb4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 --2- 172.21.15.7:0/2441076506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4a7bfcb40 0x55f4a7beba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31c0c00700 0 client.24641 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 >> 172.21.15.7:0/2441076506 conn(0x55f4a7b23120 msgr2=0x55f4a7bfed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 shutdown_connections 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.962+0000 7f31e7f4fb80 1 -- 172.21.15.7:0/2441076506 wait complete. 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31e7f4fb80 20 client.24641 trim_cache size 0 max 16384 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:50:59.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24539, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 40.461061602000001, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 97, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 95, 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 62.338131347999997 2024-06-13T06:50:59.966 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 msgr2=0x7f31bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 0x7f31bc0a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009fd0 tx=0x7f31d000b040 comp rx=0 tx=0).stop 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 msgr2=0x7f31d819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005910 tx=0x7f31c800bab0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 shutdown_connections 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31bc0a2c20 0x7f31bc0a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31d8108500 0x7f31d819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31d806a170 0x7f31d819ae60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 --2- 172.21.15.7:0/3550319439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31d806ab30 0x7f31d819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:50:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.963+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 >> 172.21.15.7:0/3550319439 conn(0x7f31d8078340 msgr2=0x7f31d810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:50:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.964+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 shutdown_connections 2024-06-13T06:50:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:50:59.964+0000 7f31dd600700 1 -- 172.21.15.7:0/3550319439 wait complete. 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-13T06:51:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-13T06:51:00.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:50:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-13T06:51:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-13T06:51:00.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:50:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-06-13T06:51:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-06-13T06:51:00.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-13T06:51:00.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:50:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-13T06:51:00.768 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-13T06:51:00.768 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:00.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs ls 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-13T06:51:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-13T06:51:01.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:00 smithi007 ceph-mon[32077]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:51:01.262 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-13T06:51:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-06-13T06:51:01.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-06-13T06:51:01.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-13T06:51:01.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-13T06:51:01.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-13T06:51:01.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:00 smithi189 ceph-mon[36616]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-06-13T06:51:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-13T06:51:01.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-13T06:51:01.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:00 smithi175 ceph-mon[36886]: osdmap e97: 12 total, 12 up, 12 in 2024-06-13T06:51:01.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:01 smithi007 ceph-mon[32077]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:51:02.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:01 smithi189 ceph-mon[36616]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:51:02.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:01 smithi175 ceph-mon[36886]: osdmap e98: 12 total, 12 up, 12 in 2024-06-13T06:51:03.100 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:02 smithi007 ceph-mon[32077]: pgmap v359: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:02 smithi189 ceph-mon[36616]: pgmap v359: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:03.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:02 smithi175 ceph-mon[36886]: pgmap v359: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 -- 172.21.15.7:0/702312841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 msgr2=0x7f01f406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 --2- 172.21.15.7:0/702312841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f406a5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009990 tx=0x7f01e0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 -- 172.21.15.7:0/702312841 shutdown_connections 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 --2- 172.21.15.7:0/702312841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f406ab10 0x7f01f410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 --2- 172.21.15.7:0/702312841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f41084e0 0x7f01f41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 --2- 172.21.15.7:0/702312841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f406a5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 -- 172.21.15.7:0/702312841 >> 172.21.15.7:0/702312841 conn(0x7f01f40782c0 msgr2=0x7f01f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 -- 172.21.15.7:0/702312841 shutdown_connections 2024-06-13T06:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.664+0000 7f01f8c00700 1 -- 172.21.15.7:0/702312841 wait complete. 2024-06-13T06:51:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.665+0000 7f01f8c00700 1 Processor -- start 2024-06-13T06:51:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 -- start start 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f406ab10 0x7f01f419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 0x7f01f419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41127b0 con 0x7f01f406ab10 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4112630 con 0x7f01f406a170 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.666+0000 7f01f8c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4112930 con 0x7f01f41084e0 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f406ab10 0x7f01f419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43728/0 (socket says 172.21.15.7:43728) 2024-06-13T06:51:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 -- 172.21.15.7:0/4234584808 learned_addr learned my addr 172.21.15.7:0/4234584808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f2800700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 0x7f01f419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 -- 172.21.15.7:0/4234584808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 msgr2=0x7f01f419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 0x7f01f419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 -- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f406ab10 msgr2=0x7f01f419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f406ab10 0x7f01f419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f1e00700 1 -- 172.21.15.7:0/4234584808 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f419fe40 con 0x7f01f406a170 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.667+0000 7f01f2800700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 0x7f01f419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01f1e00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d980 tx=0x7f01e400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4016070 con 0x7f01f406a170 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f41a0130 con 0x7f01f406a170 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e4004d10 con 0x7f01f406a170 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41a79b0 con 0x7f01f406a170 2024-06-13T06:51:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.668+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4005600 con 0x7f01f406a170 2024-06-13T06:51:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.669+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f4069180 con 0x7f01f406a170 2024-06-13T06:51:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.673+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f01e4004750 con 0x7f01f406a170 2024-06-13T06:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.674+0000 7f01eaa00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 0x7f01d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.674+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f01e40cf0d0 con 0x7f01f406a170 2024-06-13T06:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.675+0000 7f01f1400700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 0x7f01d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.675+0000 7f01f1400700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 0x7f01d80a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01f4078850 tx=0x7f01e000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.679+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f01e401f020 con 0x7f01f406a170 2024-06-13T06:51:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.965+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f01f402dca0 con 0x7f01f406a170 2024-06-13T06:51:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.966+0000 7f01eaa00700 1 -- 172.21.15.7:0/4234584808 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f01e409bc90 con 0x7f01f406a170 2024-06-13T06:51:03.965 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-13T06:51:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.970+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 msgr2=0x7f01d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.970+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 0x7f01d80a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01f4078850 tx=0x7f01e000b260 comp rx=0 tx=0).stop 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.970+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 msgr2=0x7f01f419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.970+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d980 tx=0x7f01e400dc90 comp rx=0 tx=0).stop 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 shutdown_connections 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f41084e0 0x7f01f419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f01d80a2ca0 0x7f01d80a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01f406a170 0x7f01f419ae30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 --2- 172.21.15.7:0/4234584808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01f406ab10 0x7f01f419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 >> 172.21.15.7:0/4234584808 conn(0x7f01f40782c0 msgr2=0x7f01f410d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 shutdown_connections 2024-06-13T06:51:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:03.971+0000 7f01f8c00700 1 -- 172.21.15.7:0/4234584808 wait complete. 2024-06-13T06:51:04.818 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-13T06:51:04.819 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-13T06:51:04.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:04.819 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-06-13T06:51:04.869 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-13T06:51:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-13T06:51:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-06-13T06:51:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:51:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-06-13T06:51:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41977sec preferred_lft 41977sec 2024-06-13T06:51:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-13T06:51:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:51:04.871 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:51:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-06-13T06:51:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-06-13T06:51:04.872 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-13T06:51:04.873 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:51:04.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:51:05.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:04 smithi007 ceph-mon[32077]: pgmap v360: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:05.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:04 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4234584808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:51:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:04 smithi189 ceph-mon[36616]: pgmap v360: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:04 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4234584808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:51:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:05.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:05.366 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-13T06:51:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:04 smithi175 ceph-mon[36886]: pgmap v360: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:04 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4234584808' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:51:05.440 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-06-13T06:51:05.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:05.444 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-06-13T06:51:05.511 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-06-13T06:51:05.511 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-13T06:51:05.511 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-13T06:51:05.511 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-13T06:51:05.511 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-13T06:51:05.512 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:51:05.512 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:51:05.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-13T06:51:05.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-13T06:51:05.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-13T06:51:05.513 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:51:05.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:51:06.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:06.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:06.330 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-13T06:51:06.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:06.387 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-06-13T06:51:06.445 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:51:06.445 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:51:06.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:06.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-13T06:51:06.445 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:51:06.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:51:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:06.647 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-13T06:51:06.647 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:51:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:51:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:06.974 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:51:06.974 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:51:06.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-06-13T06:51:06.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-06-13T06:51:06.974 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:51:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:51:07.106 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:06 smithi007 ceph-mon[32077]: pgmap v361: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:07.186 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-13T06:51:07.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:51:07.186 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:07.238 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-13T06:51:07.238 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:51:07.238 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:07.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-13T06:51:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:06 smithi189 ceph-mon[36616]: pgmap v361: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:06 smithi175 ceph-mon[36886]: pgmap v361: 129 pgs: 4 activating, 125 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:07.377 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-13T06:51:07.431 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-13T06:51:07.432 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cfc8cc77fd11cdf9d5c5f708108dd6ca4dc733d3c50c971d0f57a66f3cb6d1c/merged 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8bd863536e88dc4bebe0de6948b8d26d8c96c201986e9513fec9da0efbd73f0c/merged 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:07.433 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/aedf5b65fbc61b08507e71e8ca41e21d589f901101695070b367907c87d2fc91/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0982e9b9ea880744a67380ff2c024361aae0c3b23b1015cf9831ac632fac401b/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/15f2ace05703653b076dda4f68647cc63cbef3c216096409ad6946ba64b9cb8a/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cc46eac5d39f6a0db554e1936368938b6557c12de5ccdbb357e0dec29f28a7e/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/49330133d8cab51bd995ba0f908843a37a22a54ac764af4f4e2a3d8e01503ac4/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7dc2fc2a60c201071c9fcd02885fbe6611c278bfedc4f7821af04a9c8b211d32/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9365433b070ba8373e2b98151e422de051d8dcb7acbda9b32cd4a221ae2bc461/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f6a75b6f7d44699e75a45bccac799775b397c569d5d652dc332baedf901fd3d4/merged 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:07.434 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:07.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:07.435 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-13T06:51:07.490 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-13T06:51:07.490 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-13T06:51:07.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-13T06:51:07.569 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:51:07.618 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-13T06:51:07.619 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-13T06:51:07.620 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cfc8cc77fd11cdf9d5c5f708108dd6ca4dc733d3c50c971d0f57a66f3cb6d1c/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8bd863536e88dc4bebe0de6948b8d26d8c96c201986e9513fec9da0efbd73f0c/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/aedf5b65fbc61b08507e71e8ca41e21d589f901101695070b367907c87d2fc91/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0982e9b9ea880744a67380ff2c024361aae0c3b23b1015cf9831ac632fac401b/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/15f2ace05703653b076dda4f68647cc63cbef3c216096409ad6946ba64b9cb8a/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cc46eac5d39f6a0db554e1936368938b6557c12de5ccdbb357e0dec29f28a7e/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/49330133d8cab51bd995ba0f908843a37a22a54ac764af4f4e2a3d8e01503ac4/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7dc2fc2a60c201071c9fcd02885fbe6611c278bfedc4f7821af04a9c8b211d32/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9365433b070ba8373e2b98151e422de051d8dcb7acbda9b32cd4a221ae2bc461/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f6a75b6f7d44699e75a45bccac799775b397c569d5d652dc332baedf901fd3d4/merged 2024-06-13T06:51:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-13T06:51:07.622 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:07.622 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:07.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:07.623 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-13T06:51:07.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-06-13T06:51:07.732+0000 7f8d7a00b480 -1 init, newargv = 0x55ff182c18d0 newargc=15 2024-06-13T06:51:07.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[75113]: starting ceph client 2024-06-13T06:51:07.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[75113]: starting fuse 2024-06-13T06:51:08.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-06-13T06:51:08.871 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-06-13T06:51:08.930 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-06-13T06:51:08.930 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-06-13T06:51:08.930 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-06-13T06:51:08.931 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cfc8cc77fd11cdf9d5c5f708108dd6ca4dc733d3c50c971d0f57a66f3cb6d1c/merged 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8bd863536e88dc4bebe0de6948b8d26d8c96c201986e9513fec9da0efbd73f0c/merged 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/aedf5b65fbc61b08507e71e8ca41e21d589f901101695070b367907c87d2fc91/merged 2024-06-13T06:51:08.932 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0982e9b9ea880744a67380ff2c024361aae0c3b23b1015cf9831ac632fac401b/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/15f2ace05703653b076dda4f68647cc63cbef3c216096409ad6946ba64b9cb8a/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/8cc46eac5d39f6a0db554e1936368938b6557c12de5ccdbb357e0dec29f28a7e/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/49330133d8cab51bd995ba0f908843a37a22a54ac764af4f4e2a3d8e01503ac4/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/7dc2fc2a60c201071c9fcd02885fbe6611c278bfedc4f7821af04a9c8b211d32/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9365433b070ba8373e2b98151e422de051d8dcb7acbda9b32cd4a221ae2bc461/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f6a75b6f7d44699e75a45bccac799775b397c569d5d652dc332baedf901fd3d4/merged 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:51:08.933 INFO:teuthology.orchestra.run.smithi007.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:08.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:08.934 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-06-13T06:51:08.986 INFO:teuthology.orchestra.run.smithi007.stdout:119 2024-06-13T06:51:08.986 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-06-13T06:51:08.986 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-13T06:51:08.987 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-06-13T06:51:08.988 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-06-13T06:51:09.161 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.75113.asok 2024-06-13T06:51:09.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:08 smithi007 ceph-mon[32077]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:09.181 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75113.asok 2024-06-13T06:51:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:09.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75113.asok status 2024-06-13T06:51:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:08 smithi189 ceph-mon[36616]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:08 smithi175 ceph-mon[36886]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "75113", 2024-06-13T06:51:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 14529, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 14529 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2953061924 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T06:51:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2953061924 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.14529 192.168.0.1:0/2953061924", 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/2953061924", 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 16, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 98, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-06-13T06:51:09.397 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:51:09.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:09.410 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-06-13T06:51:09.410 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:09.459 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-06-13T06:51:09.459 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:09.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:09.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:09.538 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-13T06:51:09.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T06:51:09.670 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-06-13T06:51:09.670 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-13T06:51:09.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:51:09.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:51:09.680 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:51:09.680 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T06:51:09.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T06:51:10.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:10.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:10.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:51:10.505 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:10 smithi189 ceph-mon[36616]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:10 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:10 smithi007 ceph-mon[32077]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:11.339 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.339 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.339 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.339 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:10 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:10 smithi175 ceph-mon[36886]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:10 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:13.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:12 smithi007 ceph-mon[32077]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 0 op/s 2024-06-13T06:51:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.326+0000 7f7112600700 1 -- 172.21.15.7:0/1395310715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c104ed0 msgr2=0x7f710c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.326+0000 7f7112600700 1 --2- 172.21.15.7:0/1395310715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c104ed0 0x7f710c105360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f710400b0a0 tx=0x7f710400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 -- 172.21.15.7:0/1395310715 shutdown_connections 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 --2- 172.21.15.7:0/1395310715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c104ed0 0x7f710c105360 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 --2- 172.21.15.7:0/1395310715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 0x7f710c06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 --2- 172.21.15.7:0/1395310715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c06a6e0 0x7f710c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 -- 172.21.15.7:0/1395310715 >> 172.21.15.7:0/1395310715 conn(0x7f710c078340 msgr2=0x7f710c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 -- 172.21.15.7:0/1395310715 shutdown_connections 2024-06-13T06:51:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.327+0000 7f7112600700 1 -- 172.21.15.7:0/1395310715 wait complete. 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.328+0000 7f7112600700 1 Processor -- start 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.328+0000 7f7112600700 1 -- start start 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.328+0000 7f7112600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c06a6e0 0x7f710c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.328+0000 7f7112600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f7112600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 0x7f710c1a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f7112600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f710c118da0 con 0x7f710c1073c0 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f7112600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f710c118c20 con 0x7f710c06a6e0 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f7112600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f710c118f20 con 0x7f710c104ed0 2024-06-13T06:51:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55708/0 (socket says 172.21.15.7:55708) 2024-06-13T06:51:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710aa00700 1 -- 172.21.15.7:0/4231917232 learned_addr learned my addr 172.21.15.7:0/4231917232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710b400700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c06a6e0 0x7f710c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710be00700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 0x7f710c1a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710aa00700 1 -- 172.21.15.7:0/4231917232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c06a6e0 msgr2=0x7f710c19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.329+0000 7f710aa00700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c06a6e0 0x7f710c19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f710aa00700 1 -- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 msgr2=0x7f710c1a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f710aa00700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 0x7f710c1a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f710aa00700 1 -- 172.21.15.7:0/4231917232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c1a41d0 con 0x7f710c104ed0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f710aa00700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70f800eb30 tx=0x7f70f800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f800cbd0 con 0x7f710c104ed0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8004500 con 0x7f710c104ed0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8010430 con 0x7f710c104ed0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c1a44c0 con 0x7f710c104ed0 2024-06-13T06:51:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.330+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c1abd40 con 0x7f710c104ed0 2024-06-13T06:51:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.332+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c069180 con 0x7f710c104ed0 2024-06-13T06:51:13.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.334+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f70f8003f00 con 0x7f710c104ed0 2024-06-13T06:51:13.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.335+0000 7f7103e00700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 0x7f70f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:13.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.336+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f70f80cf6c0 con 0x7f710c104ed0 2024-06-13T06:51:13.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.336+0000 7f710b400700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 0x7f70f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:13.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.337+0000 7f710b400700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 0x7f70f00a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:12 smithi189 ceph-mon[36616]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 0 op/s 2024-06-13T06:51:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.341+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70f809c280 con 0x7f710c104ed0 2024-06-13T06:51:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:12 smithi175 ceph-mon[36886]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 381 B/s wr, 0 op/s 2024-06-13T06:51:13.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.621+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f710c1a0520 con 0x7f710c104ed0 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.621+0000 7f7103e00700 1 -- 172.21.15.7:0/4231917232 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+581 (secure 0 0 0) 0x7f710c1a0520 con 0x7f710c104ed0 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3381938819 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1735370722 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1576245213 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/239861889 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:13.621 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1109324950 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:13.622 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:13.622 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1476683041 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 msgr2=0x7f70f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 0x7f70f00a5160 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc00b410 comp rx=0 tx=0).stop 2024-06-13T06:51:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 msgr2=0x7f710c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70f800eb30 tx=0x7f70f800ee40 comp rx=0 tx=0).stop 2024-06-13T06:51:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 shutdown_connections 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f710c1073c0 0x7f710c1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f70f00a2ca0 0x7f70f00a5160 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f710c06a6e0 0x7f710c19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 --2- 172.21.15.7:0/4231917232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710c104ed0 0x7f710c19f750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 >> 172.21.15.7:0/4231917232 conn(0x7f710c078340 msgr2=0x7f710c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 shutdown_connections 2024-06-13T06:51:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:13.625+0000 7f7112600700 1 -- 172.21.15.7:0/4231917232 wait complete. 2024-06-13T06:51:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-13T06:51:14.478 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:51:14.478 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T06:51:14.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T06:51:14.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:14 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:14 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4231917232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:14.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:14 smithi007 ceph-mon[32077]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:14 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:14 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4231917232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:14 smithi189 ceph-mon[36616]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:14 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:14 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4231917232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:14 smithi175 ceph-mon[36886]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:15.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:15.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:15.677 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:51:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:51:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:51:16.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:16 smithi007 ceph-mon[32077]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:16.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:16 smithi175 ceph-mon[36886]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:17.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:16 smithi189 ceph-mon[36616]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 -- 172.21.15.7:0/4157678854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc113190 msgr2=0x7fc8bc115580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4157678854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc113190 0x7fc8bc115580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b0a0 tx=0x7fc8b400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 -- 172.21.15.7:0/4157678854 shutdown_connections 2024-06-13T06:51:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4157678854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc113190 0x7fc8bc115580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4157678854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc076470 0x7fc8bc0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4157678854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc074c00 0x7fc8bc075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 -- 172.21.15.7:0/4157678854 >> 172.21.15.7:0/4157678854 conn(0x7fc8bc079650 msgr2=0x7fc8bc079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 -- 172.21.15.7:0/4157678854 shutdown_connections 2024-06-13T06:51:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.350+0000 7fc8c4600700 1 -- 172.21.15.7:0/4157678854 wait complete. 2024-06-13T06:51:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.351+0000 7fc8c4600700 1 Processor -- start 2024-06-13T06:51:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 -- start start 2024-06-13T06:51:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 0x7fc8bc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 0x7fc8bc1a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc118da0 con 0x7fc8bc074c00 2024-06-13T06:51:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc118c20 con 0x7fc8bc113190 2024-06-13T06:51:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.352+0000 7fc8c4600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc118f20 con 0x7fc8bc076470 2024-06-13T06:51:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 0x7fc8bc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52504/0 (socket says 172.21.15.7:52504) 2024-06-13T06:51:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 -- 172.21.15.7:0/4279682900 learned_addr learned my addr 172.21.15.7:0/4279682900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c2600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 0x7fc8bc1a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 -- 172.21.15.7:0/4279682900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 msgr2=0x7fc8bc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 0x7fc8bc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 -- 172.21.15.7:0/4279682900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 msgr2=0x7fc8bc1a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 0x7fc8bc1a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1c00700 1 -- 172.21.15.7:0/4279682900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc1a4200 con 0x7fc8bc074c00 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c1200700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 0x7fc8bc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.353+0000 7fc8c2600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 0x7fc8bc1a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8c1c00700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9a0 tx=0x7fc8b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ca10 con 0x7fc8bc074c00 2024-06-13T06:51:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b000cbb0 con 0x7fc8bc074c00 2024-06-13T06:51:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018680 con 0x7fc8bc074c00 2024-06-13T06:51:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc1a44f0 con 0x7fc8bc074c00 2024-06-13T06:51:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.354+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1abd70 con 0x7fc8bc074c00 2024-06-13T06:51:17.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.356+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc069180 con 0x7fc8bc074c00 2024-06-13T06:51:17.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.358+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc8b0010430 con 0x7fc8bc074c00 2024-06-13T06:51:17.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.359+0000 7fc8ba000700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 0x7fc8a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.359+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc8b00cf970 con 0x7fc8bc074c00 2024-06-13T06:51:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.360+0000 7fc8c1200700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 0x7fc8a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.360+0000 7fc8c1200700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 0x7fc8a40a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009680 tx=0x7fc8a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:17.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.362+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8b009c530 con 0x7fc8bc074c00 2024-06-13T06:51:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.613+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8bc1abf70 con 0x7fc8bc074c00 2024-06-13T06:51:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.614+0000 7fc8ba000700 1 -- 172.21.15.7:0/4279682900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+581 (secure 0 0 0) 0x7fc8b009be30 con 0x7fc8bc074c00 2024-06-13T06:51:17.614 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3381938819 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1735370722 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1576245213 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/239861889 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1109324950 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T06:51:17.615 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1476683041 2024-06-14T06:41:59.455768+0000 2024-06-13T06:51:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 msgr2=0x7fc8a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 0x7fc8a40a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009680 tx=0x7fc8a8009260 comp rx=0 tx=0).stop 2024-06-13T06:51:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 msgr2=0x7fc8bc19f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9a0 tx=0x7fc8b000ecb0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 shutdown_connections 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8a40a2ca0 0x7fc8a40a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8bc113190 0x7fc8bc1a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc074c00 0x7fc8bc19f1f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 --2- 172.21.15.7:0/4279682900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8bc076470 0x7fc8bc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 >> 172.21.15.7:0/4279682900 conn(0x7fc8bc079650 msgr2=0x7fc8bc111eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 shutdown_connections 2024-06-13T06:51:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:17.618+0000 7fc8c4600700 1 -- 172.21.15.7:0/4279682900 wait complete. 2024-06-13T06:51:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-13T06:51:17.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:17 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4279682900' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:18.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:17 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4279682900' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:18.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:17 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4279682900' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T06:51:18.449 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-06-13T06:51:18.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:18.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-13T06:51:18.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[75113]: fuse finished with error 0 and tester_r 0 2024-06-13T06:51:18.554 INFO:teuthology.orchestra.run:waiting for 300 2024-06-13T06:51:19.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:18 smithi189 ceph-mon[36616]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:19.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:18 smithi175 ceph-mon[36886]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:18 smithi007 ceph-mon[32077]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:21.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:20 smithi189 ceph-mon[36616]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:20 smithi175 ceph-mon[36886]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:21.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:20 smithi007 ceph-mon[32077]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-13T06:51:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:22 smithi189 ceph-mon[36616]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-13T06:51:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:22 smithi175 ceph-mon[36886]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-13T06:51:23.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:22 smithi007 ceph-mon[32077]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-13T06:51:24.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:51:24.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:51:24.656 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-13T06:51:24.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:51:25.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:24 smithi189 ceph-mon[36616]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:25.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:24 smithi175 ceph-mon[36886]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:25.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:24 smithi007 ceph-mon[32077]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:26.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:26 smithi007 ceph-mon[32077]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:26 smithi189 ceph-mon[36616]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:26 smithi175 ceph-mon[36886]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 -- 172.21.15.7:0/2103082618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c010a6b0 msgr2=0x7f91c010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 --2- 172.21.15.7:0/2103082618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c010a6b0 0x7f91c010aa90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009990 tx=0x7f91b0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 -- 172.21.15.7:0/2103082618 shutdown_connections 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 --2- 172.21.15.7:0/2103082618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c0105070 0x7f91c0111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 --2- 172.21.15.7:0/2103082618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c010a6b0 0x7f91c010aa90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 --2- 172.21.15.7:0/2103082618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c01046b0 0x7f91c0104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 -- 172.21.15.7:0/2103082618 >> 172.21.15.7:0/2103082618 conn(0x7f91c01001a0 msgr2=0x7f91c01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 -- 172.21.15.7:0/2103082618 shutdown_connections 2024-06-13T06:51:27.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.575+0000 7f91c7200700 1 -- 172.21.15.7:0/2103082618 wait complete. 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 Processor -- start 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 -- start start 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 0x7f91c019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c010a6b0 0x7f91c019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91c0114a50 con 0x7f91c0105070 2024-06-13T06:51:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01148d0 con 0x7f91c01046b0 2024-06-13T06:51:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91c7200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91c0114bd0 con 0x7f91c010a6b0 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.577+0000 7f91bf400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 0x7f91c019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54438/0 (socket says 172.21.15.7:54438) 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bfe00700 1 -- 172.21.15.7:0/1953883758 learned_addr learned my addr 172.21.15.7:0/1953883758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91c5200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c010a6b0 0x7f91c019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 -- 172.21.15.7:0/1953883758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c010a6b0 msgr2=0x7f91c019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c010a6b0 0x7f91c019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 -- 172.21.15.7:0/1953883758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 msgr2=0x7f91c019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 -- 172.21.15.7:0/1953883758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c019ff30 con 0x7f91c0105070 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bfe00700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.578+0000 7f91bf400700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 0x7f91c019b460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00e9f0 tx=0x7f91ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.579+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac004220 con 0x7f91c0105070 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.579+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ac004d10 con 0x7f91c0105070 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.579+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c01a0220 con 0x7f91c0105070 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.579+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac005020 con 0x7f91c0105070 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.579+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c01a7aa0 con 0x7f91c0105070 2024-06-13T06:51:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.580+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c0069180 con 0x7f91c0105070 2024-06-13T06:51:27.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.583+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f91ac004750 con 0x7f91c0105070 2024-06-13T06:51:27.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.584+0000 7f91bcc00700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 0x7f91a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:27.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.584+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f91ac0cf290 con 0x7f91c0105070 2024-06-13T06:51:27.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.585+0000 7f91bfe00700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 0x7f91a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:27.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.585+0000 7f91bfe00700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 0x7f91a40a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009750 tx=0x7f91b000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.589+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91ac09be50 con 0x7f91c0105070 2024-06-13T06:51:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.819+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91c019c2b0 con 0x7f91c0105070 2024-06-13T06:51:27.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.822+0000 7f91bcc00700 1 -- 172.21.15.7:0/1953883758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f91c019c2b0 con 0x7f91c0105070 2024-06-13T06:51:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:51:27.822 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:51:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 msgr2=0x7f91a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 0x7f91a40a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009750 tx=0x7f91b000b280 comp rx=0 tx=0).stop 2024-06-13T06:51:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 msgr2=0x7f91c019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 0x7f91c019b460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00e9f0 tx=0x7f91ac00ed00 comp rx=0 tx=0).stop 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 shutdown_connections 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91c010a6b0 0x7f91c019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f91a40a2c20 0x7f91a40a50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91c01046b0 0x7f91c019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 --2- 172.21.15.7:0/1953883758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c0105070 0x7f91c019b460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.825+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 >> 172.21.15.7:0/1953883758 conn(0x7f91c01001a0 msgr2=0x7f91c0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.826+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 shutdown_connections 2024-06-13T06:51:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:27.826+0000 7f91c7200700 1 -- 172.21.15.7:0/1953883758 wait complete. 2024-06-13T06:51:28.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:51:28.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:28 smithi007 ceph-mon[32077]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:28.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:28 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1953883758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:28 smithi189 ceph-mon[36616]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:28 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1953883758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:28 smithi175 ceph-mon[36886]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:28 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1953883758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:30.864 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:30 smithi007 ceph-mon[32077]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:30 smithi189 ceph-mon[36616]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:30 smithi175 ceph-mon[36886]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.469+0000 7fca54600700 1 -- 172.21.15.7:0/1778717304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 msgr2=0x7fca4c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.469+0000 7fca54600700 1 --2- 172.21.15.7:0/1778717304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c1089d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca40009990 tx=0x7fca40009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 -- 172.21.15.7:0/1778717304 shutdown_connections 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 --2- 172.21.15.7:0/1778717304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c102ea0 0x7fca4c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 --2- 172.21.15.7:0/1778717304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c1089d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 --2- 172.21.15.7:0/1778717304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c1024e0 0x7fca4c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 -- 172.21.15.7:0/1778717304 >> 172.21.15.7:0/1778717304 conn(0x7fca4c079640 msgr2=0x7fca4c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 -- 172.21.15.7:0/1778717304 shutdown_connections 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.470+0000 7fca54600700 1 -- 172.21.15.7:0/1778717304 wait complete. 2024-06-13T06:51:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.471+0000 7fca54600700 1 Processor -- start 2024-06-13T06:51:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.471+0000 7fca54600700 1 -- start start 2024-06-13T06:51:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c102ea0 0x7fca4c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1127b0 con 0x7fca4c1024e0 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c112630 con 0x7fca4c102ea0 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca54600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c112930 con 0x7fca4c1085f0 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca51c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca51c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33062/0 (socket says 172.21.15.7:33062) 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca51c00700 1 -- 172.21.15.7:0/2826056055 learned_addr learned my addr 172.21.15.7:0/2826056055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca52600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.472+0000 7fca51200700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c102ea0 0x7fca4c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 -- 172.21.15.7:0/2826056055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 msgr2=0x7fca4c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 -- 172.21.15.7:0/2826056055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c102ea0 msgr2=0x7fca4c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c102ea0 0x7fca4c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 -- 172.21.15.7:0/2826056055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c1a71a0 con 0x7fca4c1024e0 2024-06-13T06:51:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca52600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.473+0000 7fca51c00700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca4000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.474+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40027070 con 0x7fca4c1024e0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.474+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca40003d70 con 0x7fca4c1024e0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.474+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4000f460 con 0x7fca4c1024e0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.474+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c1a7430 con 0x7fca4c1024e0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.474+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1a78e0 con 0x7fca4c1024e0 2024-06-13T06:51:31.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.475+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c10aa40 con 0x7fca4c1024e0 2024-06-13T06:51:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.477+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fca4002a070 con 0x7fca4c1024e0 2024-06-13T06:51:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.478+0000 7fca4a000700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 0x7fca340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.479+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca400d0140 con 0x7fca4c1024e0 2024-06-13T06:51:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.479+0000 7fca51200700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 0x7fca340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.480+0000 7fca51200700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 0x7fca340a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca4c079c80 tx=0x7fca38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:31.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.483+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca40013070 con 0x7fca4c1024e0 2024-06-13T06:51:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.713+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca4c075c70 con 0x7fca4c1024e0 2024-06-13T06:51:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.719+0000 7fca4a000700 1 -- 172.21.15.7:0/2826056055 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7fca4009cd90 con 0x7fca4c1024e0 2024-06-13T06:51:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:51:31.720 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:51:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.722+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 msgr2=0x7fca340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.722+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 0x7fca340a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca4c079c80 tx=0x7fca38009260 comp rx=0 tx=0).stop 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.722+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 msgr2=0x7fca4c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.722+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca4000bab0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 shutdown_connections 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca4c1085f0 0x7fca4c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4c1024e0 0x7fca4c074790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca340a2ca0 0x7fca340a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 --2- 172.21.15.7:0/2826056055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca4c102ea0 0x7fca4c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:31.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 >> 172.21.15.7:0/2826056055 conn(0x7fca4c079640 msgr2=0x7fca4c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:31.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 shutdown_connections 2024-06-13T06:51:31.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:31.723+0000 7fca54600700 1 -- 172.21.15.7:0/2826056055 wait complete. 2024-06-13T06:51:32.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-13T06:51:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:32 smithi189 ceph-mon[36616]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2826056055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:32 smithi175 ceph-mon[36886]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2826056055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:33.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:32 smithi007 ceph-mon[32077]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-13T06:51:33.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2826056055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:51:34.899 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:34 smithi007 ceph-mon[32077]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:34 smithi189 ceph-mon[36616]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:34 smithi175 ceph-mon[36886]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 -- 172.21.15.7:0/1871441707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341068a0 msgr2=0x7f7f34111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 --2- 172.21.15.7:0/1871441707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341068a0 0x7f7f34111530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009990 tx=0x7f7f2c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 -- 172.21.15.7:0/1871441707 shutdown_connections 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 --2- 172.21.15.7:0/1871441707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341068a0 0x7f7f34111530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 --2- 172.21.15.7:0/1871441707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341048d0 0x7f7f34104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 --2- 172.21.15.7:0/1871441707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f34106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.429+0000 7f7f39800700 1 -- 172.21.15.7:0/1871441707 >> 172.21.15.7:0/1871441707 conn(0x7f7f34100040 msgr2=0x7f7f34102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.430+0000 7f7f39800700 1 -- 172.21.15.7:0/1871441707 shutdown_connections 2024-06-13T06:51:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.430+0000 7f7f39800700 1 -- 172.21.15.7:0/1871441707 wait complete. 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.431+0000 7f7f39800700 1 Processor -- start 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.431+0000 7f7f39800700 1 -- start start 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f3419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341068a0 0x7f7f3419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341149f0 con 0x7f7f341068a0 2024-06-13T06:51:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34114870 con 0x7f7f341051f0 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f39800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34114b70 con 0x7f7f341048d0 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f33e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f33e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40960/0 (socket says 172.21.15.7:40960) 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f33e00700 1 -- 172.21.15.7:0/3693976833 learned_addr learned my addr 172.21.15.7:0/3693976833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f38e00700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341068a0 0x7f7f3419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.432+0000 7f7f33400700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f3419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 -- 172.21.15.7:0/3693976833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 msgr2=0x7f7f3419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 -- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341068a0 msgr2=0x7f7f3419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341068a0 0x7f7f3419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 -- 172.21.15.7:0/3693976833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3419fd90 con 0x7f7f341051f0 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33e00700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f33400700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f3419b2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e9c0 tx=0x7f7f2000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.433+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000c9c0 con 0x7f7f341051f0 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.434+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2000cb60 con 0x7f7f341051f0 2024-06-13T06:51:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.434+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20018680 con 0x7f7f341051f0 2024-06-13T06:51:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.434+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f341a0080 con 0x7f7f341051f0 2024-06-13T06:51:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.434+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f341a7900 con 0x7f7f341051f0 2024-06-13T06:51:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.435+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f34069180 con 0x7f7f341051f0 2024-06-13T06:51:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.437+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7f20010430 con 0x7f7f341051f0 2024-06-13T06:51:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.438+0000 7f7f30c00700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 0x7f7f180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.439+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7f200cf780 con 0x7f7f341051f0 2024-06-13T06:51:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.439+0000 7f7f33e00700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 0x7f7f180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.439+0000 7f7f33e00700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 0x7f7f180a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009c10 tx=0x7f7f2400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:35.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.441+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f2009c3d0 con 0x7f7f341051f0 2024-06-13T06:51:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:35.724+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7f341a7ad0 con 0x7f7f341051f0 2024-06-13T06:51:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.735+0000 7f7f30c00700 1 -- 172.21.15.7:0/3693976833 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f7f2009bcd0 con 0x7f7f341051f0 2024-06-13T06:51:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.739+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 msgr2=0x7f7f180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.739+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 0x7f7f180a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009c10 tx=0x7f7f2400b040 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.739+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 msgr2=0x7f7f3419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.739+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f3419b2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000e9c0 tx=0x7f7f2000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 shutdown_connections 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f341068a0 0x7f7f3419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f341048d0 0x7f7f3419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f180a2ca0 0x7f7f180a5160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 --2- 172.21.15.7:0/3693976833 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f341051f0 0x7f7f3419b2c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 >> 172.21.15.7:0/3693976833 conn(0x7f7f34100040 msgr2=0x7f7f34101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 shutdown_connections 2024-06-13T06:51:36.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:36.740+0000 7f7f39800700 1 -- 172.21.15.7:0/3693976833 wait complete. 2024-06-13T06:51:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:36 smithi189 ceph-mon[36616]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:36 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3693976833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:36 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:36 smithi175 ceph-mon[36886]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:36 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3693976833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:36 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:36 smithi007 ceph-mon[32077]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:37.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:36 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3693976833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.132 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:36 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-13T06:51:37.585 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-06-13T06:51:37.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-13T06:51:37.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:37 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:51:37.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:37 smithi007 ceph-mon[32077]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:38.060 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:37 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:51:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:37 smithi189 ceph-mon[36616]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:37 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-13T06:51:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:37 smithi175 ceph-mon[36886]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:38.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:38 smithi007 ceph-mon[32077]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:38 smithi189 ceph-mon[36616]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:38 smithi175 ceph-mon[36886]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 -- 172.21.15.7:0/1032032966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 msgr2=0x7f193c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 --2- 172.21.15.7:0/1032032966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c108e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1928009990 tx=0x7f1928009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 -- 172.21.15.7:0/1032032966 shutdown_connections 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 --2- 172.21.15.7:0/1032032966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 0x7f193c115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 --2- 172.21.15.7:0/1032032966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 --2- 172.21.15.7:0/1032032966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c108e50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 -- 172.21.15.7:0/1032032966 >> 172.21.15.7:0/1032032966 conn(0x7f193c078220 msgr2=0x7f193c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 -- 172.21.15.7:0/1032032966 shutdown_connections 2024-06-13T06:51:40.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.397+0000 7f1942c00700 1 -- 172.21.15.7:0/1032032966 wait complete. 2024-06-13T06:51:40.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.398+0000 7f1942c00700 1 Processor -- start 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.398+0000 7f1942c00700 1 -- start start 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 0x7f193c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118d20 con 0x7f193c1089d0 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118ba0 con 0x7f193c109390 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118ea0 con 0x7f193c10d8a0 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1941800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40980/0 (socket says 172.21.15.7:40980) 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1942200700 1 -- 172.21.15.7:0/599395182 learned_addr learned my addr 172.21.15.7:0/599395182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.399+0000 7f1940e00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 0x7f193c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 -- 172.21.15.7:0/599395182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 msgr2=0x7f193c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 0x7f193c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 -- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 msgr2=0x7f193c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 -- 172.21.15.7:0/599395182 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a4240 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942200700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f193400ef30 tx=0x7f193400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19340092e0 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1934004bb0 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a44d0 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.400+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934004280 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.401+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1abdb0 con 0x7f193c10d8a0 2024-06-13T06:51:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.402+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c069180 con 0x7f193c10d8a0 2024-06-13T06:51:40.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.404+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1934044d10 con 0x7f193c10d8a0 2024-06-13T06:51:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.406+0000 7f193a000700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 0x7f19240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.406+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f19340cf2e0 con 0x7f193c10d8a0 2024-06-13T06:51:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.406+0000 7f1941800700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 0x7f19240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.406+0000 7f1941800700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 0x7f19240a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1930009be0 tx=0x7f1930009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.408+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1934022020 con 0x7f193c10d8a0 2024-06-13T06:51:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.688+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f193c02dca0 con 0x7f193c10d8a0 2024-06-13T06:51:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.759+0000 7f193a000700 1 -- 172.21.15.7:0/599395182 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f1934010040 con 0x7f193c10d8a0 2024-06-13T06:51:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 msgr2=0x7f19240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 0x7f19240a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1930009be0 tx=0x7f1930009260 comp rx=0 tx=0).stop 2024-06-13T06:51:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 msgr2=0x7f193c1a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f193400ef30 tx=0x7f193400c3a0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 shutdown_connections 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f193c10d8a0 0x7f193c1a3b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c1089d0 0x7f193c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19240a2c20 0x7f19240a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 --2- 172.21.15.7:0/599395182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f193c109390 0x7f193c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.764+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 >> 172.21.15.7:0/599395182 conn(0x7f193c078220 msgr2=0x7f193c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.765+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 shutdown_connections 2024-06-13T06:51:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:40.765+0000 7f1942c00700 1 -- 172.21.15.7:0/599395182 wait complete. 2024-06-13T06:51:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:40 smithi189 ceph-mon[36616]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:40 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/599395182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:40 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:40 smithi175 ceph-mon[36886]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:40 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/599395182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:40 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:40 smithi007 ceph-mon[32077]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T06:51:41.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:40 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/599395182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:40 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-13T06:51:41.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-13T06:51:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:51:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:51:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:51:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:41.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-13T06:51:41.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:41 smithi007 ceph-mon[32077]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-13T06:51:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:41 smithi189 ceph-mon[36616]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-13T06:51:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:41 smithi175 ceph-mon[36886]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-13T06:51:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:active 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: Cluster is now healthy 2024-06-13T06:51:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-13T06:51:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-13T06:51:43.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:42 smithi189 ceph-mon[36616]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:active 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: Cluster is now healthy 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-13T06:51:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:42 smithi175 ceph-mon[36886]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] up:active 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: Cluster is now healthy 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-13T06:51:43.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:42 smithi007 ceph-mon[32077]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-13T06:51:43.617 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:51:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T06:51:43.227+0000 7f97abd43700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-13T06:51:43.617 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:51:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T06:51:43.227+0000 7f97abd43700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-13T06:51:44.054 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:43 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:active 2024-06-13T06:51:44.054 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:43 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:43 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:active 2024-06-13T06:51:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:43 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:43 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] up:active 2024-06-13T06:51:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:43 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 -- 172.21.15.7:0/3373407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006ab30 msgr2=0x7ffb4010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 --2- 172.21.15.7:0/3373407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006ab30 0x7ffb4010f320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009f90 tx=0x7ffb3800b330 comp rx=0 tx=0).stop 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 -- 172.21.15.7:0/3373407972 shutdown_connections 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 --2- 172.21.15.7:0/3373407972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006ab30 0x7ffb4010f320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 --2- 172.21.15.7:0/3373407972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 0x7ffb401088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 --2- 172.21.15.7:0/3373407972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006a170 0x7ffb4006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 -- 172.21.15.7:0/3373407972 >> 172.21.15.7:0/3373407972 conn(0x7ffb40078340 msgr2=0x7ffb40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.591+0000 7ffb46a00700 1 -- 172.21.15.7:0/3373407972 shutdown_connections 2024-06-13T06:51:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.592+0000 7ffb46a00700 1 -- 172.21.15.7:0/3373407972 wait complete. 2024-06-13T06:51:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.593+0000 7ffb46a00700 1 Processor -- start 2024-06-13T06:51:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.593+0000 7ffb46a00700 1 -- start start 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 0x7ffb4019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 0x7ffb4019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40112760 con 0x7ffb4006a170 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401125e0 con 0x7ffb4006ab30 2024-06-13T06:51:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb46a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401128e0 con 0x7ffb40108500 2024-06-13T06:51:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb3fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb3fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34172/0 (socket says 172.21.15.7:34172) 2024-06-13T06:51:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb3fe00700 1 -- 172.21.15.7:0/1097556907 learned_addr learned my addr 172.21.15.7:0/1097556907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb37e00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 0x7ffb4019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.594+0000 7ffb44a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 0x7ffb4019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 -- 172.21.15.7:0/1097556907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 msgr2=0x7ffb4019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 0x7ffb4019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 -- 172.21.15.7:0/1097556907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 msgr2=0x7ffb4019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 0x7ffb4019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 -- 172.21.15.7:0/1097556907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4019fda0 con 0x7ffb4006a170 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb37e00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 0x7ffb4019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3fe00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000ec10 tx=0x7ffb3000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000cc90 con 0x7ffb4006a170 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb3000ce30 con 0x7ffb4006a170 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb401a0090 con 0x7ffb4006a170 2024-06-13T06:51:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.595+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30010690 con 0x7ffb4006a170 2024-06-13T06:51:44.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.596+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a7910 con 0x7ffb4006a170 2024-06-13T06:51:44.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.597+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb40069180 con 0x7ffb4006a170 2024-06-13T06:51:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.599+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ffb30003680 con 0x7ffb4006a170 2024-06-13T06:51:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.600+0000 7ffb3d600700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 0x7ffb240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.600+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ffb300cf6c0 con 0x7ffb4006a170 2024-06-13T06:51:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.601+0000 7ffb37e00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 0x7ffb240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.601+0000 7ffb37e00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 0x7ffb240a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb401033e0 tx=0x7ffb28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.605+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb3009c280 con 0x7ffb4006a170 2024-06-13T06:51:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:44.891+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ffb401a7b40 con 0x7ffb4006a170 2024-06-13T06:51:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:44 smithi189 ceph-mon[36616]: pgmap v380: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-06-13T06:51:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:44 smithi175 ceph-mon[36886]: pgmap v380: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-06-13T06:51:45.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:44 smithi007 ceph-mon[32077]: pgmap v380: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-06-13T06:51:45.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.816+0000 7ffb3d600700 1 -- 172.21.15.7:0/1097556907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ffb3009bb80 con 0x7ffb4006a170 2024-06-13T06:51:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 msgr2=0x7ffb240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 0x7ffb240a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb401033e0 tx=0x7ffb28009260 comp rx=0 tx=0).stop 2024-06-13T06:51:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 msgr2=0x7ffb4019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000ec10 tx=0x7ffb3000ef20 comp rx=0 tx=0).stop 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 shutdown_connections 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb40108500 0x7ffb4019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb4006a170 0x7ffb4019ad90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffb240a2c20 0x7ffb240a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 --2- 172.21.15.7:0/1097556907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb4006ab30 0x7ffb4019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 >> 172.21.15.7:0/1097556907 conn(0x7ffb40078340 msgr2=0x7ffb4010d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.821+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 shutdown_connections 2024-06-13T06:51:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:45.822+0000 7ffb46a00700 1 -- 172.21.15.7:0/1097556907 wait complete. 2024-06-13T06:51:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:45 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:51:46.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:45 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:51:46.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:45 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-13T06:51:46.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-13T06:51:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:46 smithi189 ceph-mon[36616]: pgmap v381: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-13T06:51:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:46 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:51:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:46 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:47.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:46 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:46 smithi175 ceph-mon[36886]: pgmap v381: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-13T06:51:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:46 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:51:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:46 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:46 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:47.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:46 smithi007 ceph-mon[32077]: pgmap v381: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-13T06:51:47.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:46 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1097556907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-13T06:51:47.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:46 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-13T06:51:47.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:46 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:49.011 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:48 smithi007 ceph-mon[32077]: pgmap v382: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-13T06:51:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:48 smithi189 ceph-mon[36616]: pgmap v382: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-13T06:51:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:48 smithi175 ceph-mon[36886]: pgmap v382: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-13T06:51:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 -- 172.21.15.7:0/2968957232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0102ee0 msgr2=0x7f60d010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 --2- 172.21.15.7:0/2968957232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0102ee0 0x7f60d010f3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009f90 tx=0x7f60c800b330 comp rx=0 tx=0).stop 2024-06-13T06:51:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 -- 172.21.15.7:0/2968957232 shutdown_connections 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 --2- 172.21.15.7:0/2968957232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0102ee0 0x7f60d010f3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 --2- 172.21.15.7:0/2968957232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0107450 0x7f60d0107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 --2- 172.21.15.7:0/2968957232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 0x7f60d01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 -- 172.21.15.7:0/2968957232 >> 172.21.15.7:0/2968957232 conn(0x7f60d00795f0 msgr2=0x7f60d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 -- 172.21.15.7:0/2968957232 shutdown_connections 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.551+0000 7f60d7400700 1 -- 172.21.15.7:0/2968957232 wait complete. 2024-06-13T06:51:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 Processor -- start 2024-06-13T06:51:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 -- start start 2024-06-13T06:51:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 0x7f60d0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0107450 0x7f60d0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0112870 con 0x7f60d0102540 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01126f0 con 0x7f60d0102ee0 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.553+0000 7f60d7400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01129f0 con 0x7f60d0107450 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60d4a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 0x7f60d0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34952/0 (socket says 172.21.15.7:34952) 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 -- 172.21.15.7:0/767560843 learned_addr learned my addr 172.21.15.7:0/767560843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60d5400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0107450 0x7f60d0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 -- 172.21.15.7:0/767560843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0107450 msgr2=0x7f60d0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0107450 0x7f60d0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 -- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 msgr2=0x7f60d0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 0x7f60d0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 -- 172.21.15.7:0/767560843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01a7110 con 0x7f60d0102ee0 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.554+0000 7f60cfe00700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00ec10 tx=0x7f60bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.555+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cc90 con 0x7f60d0102ee0 2024-06-13T06:51:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.555+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01a7400 con 0x7f60d0102ee0 2024-06-13T06:51:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.555+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60bc00ce30 con 0x7f60d0102ee0 2024-06-13T06:51:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.555+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010690 con 0x7f60d0102ee0 2024-06-13T06:51:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.555+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a7990 con 0x7f60d0102ee0 2024-06-13T06:51:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.556+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d0069180 con 0x7f60d0102ee0 2024-06-13T06:51:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.558+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f60bc044b30 con 0x7f60d0102ee0 2024-06-13T06:51:49.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.560+0000 7f60cd600700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 0x7f60b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:49.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.560+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f60bc0cf630 con 0x7f60d0102ee0 2024-06-13T06:51:49.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.560+0000 7f60d4a00700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 0x7f60b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:49.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.561+0000 7f60d4a00700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 0x7f60b80a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60d0103720 tx=0x7f60c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:49.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.562+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60bc044e20 con 0x7f60d0102ee0 2024-06-13T06:51:49.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:49.854+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f60d0075cc0 con 0x7f60d0102ee0 2024-06-13T06:51:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.841+0000 7f60cd600700 1 -- 172.21.15.7:0/767560843 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f60d0075cc0 con 0x7f60d0102ee0 2024-06-13T06:51:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 msgr2=0x7f60b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 0x7f60b80a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60d0103720 tx=0x7f60c4009260 comp rx=0 tx=0).stop 2024-06-13T06:51:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 msgr2=0x7f60d0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00ec10 tx=0x7f60bc00ef20 comp rx=0 tx=0).stop 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 shutdown_connections 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60d0107450 0x7f60d0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d0102540 0x7f60d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60b80a2ca0 0x7f60b80a5160 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 --2- 172.21.15.7:0/767560843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60d0102ee0 0x7f60d0074cd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.845+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 >> 172.21.15.7:0/767560843 conn(0x7f60d00795f0 msgr2=0x7f60d01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.846+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 shutdown_connections 2024-06-13T06:51:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:50.846+0000 7f60d7400700 1 -- 172.21.15.7:0/767560843 wait complete. 2024-06-13T06:51:50.929 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:50 smithi007 ceph-mon[32077]: pgmap v383: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-13T06:51:50.929 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:50 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/767560843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:50.929 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:50 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:50 smithi189 ceph-mon[36616]: pgmap v383: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-13T06:51:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:50 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/767560843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:50 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:50 smithi175 ceph-mon[36886]: pgmap v383: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-13T06:51:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:50 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/767560843' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:50 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-13T06:51:51.663 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-06-13T06:51:51.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-13T06:51:51.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:51 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:51:51.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:51 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:51 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:51:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:51 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:52.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:51 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-13T06:51:52.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:51 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:51:53.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:52 smithi175 ceph-mon[36886]: pgmap v384: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:52 smithi007 ceph-mon[32077]: pgmap v384: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:52 smithi189 ceph-mon[36616]: pgmap v384: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 -- 172.21.15.7:0/1704057586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541810a6b0 msgr2=0x7f541810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 --2- 172.21.15.7:0/1704057586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541810a6b0 0x7f541810aa90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5408009990 tx=0x7f5408009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 -- 172.21.15.7:0/1704057586 shutdown_connections 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 --2- 172.21.15.7:0/1704057586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f5418111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 --2- 172.21.15.7:0/1704057586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f541810a6b0 0x7f541810aa90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 --2- 172.21.15.7:0/1704057586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54181046b0 0x7f5418104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 -- 172.21.15.7:0/1704057586 >> 172.21.15.7:0/1704057586 conn(0x7f54181001a0 msgr2=0x7f54181025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 -- 172.21.15.7:0/1704057586 shutdown_connections 2024-06-13T06:51:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.502+0000 7f541d200700 1 -- 172.21.15.7:0/1704057586 wait complete. 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 Processor -- start 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 -- start start 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 0x7f541819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541810a6b0 0x7f541819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5418114a00 con 0x7f5418105070 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5418114880 con 0x7f54181046b0 2024-06-13T06:51:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.503+0000 7f541d200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5418114b80 con 0x7f541810a6b0 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57734/0 (socket says 172.21.15.7:57734) 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 -- 172.21.15.7:0/1711995226 learned_addr learned my addr 172.21.15.7:0/1711995226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415e00700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 0x7f541819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5416800700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541810a6b0 0x7f541819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 -- 172.21.15.7:0/1711995226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541810a6b0 msgr2=0x7f541819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541810a6b0 0x7f541819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 -- 172.21.15.7:0/1711995226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 msgr2=0x7f541819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 0x7f541819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415400700 1 -- 172.21.15.7:0/1711995226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541819fe50 con 0x7f5418105070 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.504+0000 7f5415e00700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 0x7f541819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f5415400700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9c0 tx=0x7f540400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400cbd0 con 0x7f5418105070 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54181a0140 con 0x7f5418105070 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f540400ee60 con 0x7f5418105070 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404005630 con 0x7f5418105070 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.505+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54181a79c0 con 0x7f5418105070 2024-06-13T06:51:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.507+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541810cc40 con 0x7f5418105070 2024-06-13T06:51:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.509+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5404004750 con 0x7f5418105070 2024-06-13T06:51:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.511+0000 7f540ea00700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 0x7f53fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.511+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54040cf5b0 con 0x7f5418105070 2024-06-13T06:51:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.511+0000 7f5415e00700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 0x7f53fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.512+0000 7f5415e00700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 0x7f53fc0a5160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54080098e0 tx=0x7f5408005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.516+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f540409c170 con 0x7f5418105070 2024-06-13T06:51:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.778+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f54180628d0 con 0x7f53fc0a2ca0 2024-06-13T06:51:54.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.814+0000 7f540ea00700 1 -- 172.21.15.7:0/1711995226 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54180628d0 con 0x7f53fc0a2ca0 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.816+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 msgr2=0x7f53fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.816+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 0x7f53fc0a5160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54080098e0 tx=0x7f5408005960 comp rx=0 tx=0).stop 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 msgr2=0x7f541819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f540400e9c0 tx=0x7f540400ecd0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 shutdown_connections 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541810a6b0 0x7f541819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2ca0 0x7f53fc0a5160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54181046b0 0x7f541819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 --2- 172.21.15.7:0/1711995226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5418105070 0x7f541819b3d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 >> 172.21.15.7:0/1711995226 conn(0x7f54181001a0 msgr2=0x7f54181007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 shutdown_connections 2024-06-13T06:51:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:54.817+0000 7f541d200700 1 -- 172.21.15.7:0/1711995226 wait complete. 2024-06-13T06:51:55.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:54 smithi175 ceph-mon[36886]: pgmap v385: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-mon[32077]: pgmap v385: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:55.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:51:54.788+0000 7f7f4b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:51:55.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:51:54.788+0000 7f7f4b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:51:55.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:51:54.788+0000 7f7f4b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:51:55.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:51:54.788+0000 7f7f4b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:51:55.164 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 06:51:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T06:51:54.788+0000 7f7f4b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-13T06:51:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:54 smithi189 ceph-mon[36616]: pgmap v385: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-13T06:51:55.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-13T06:51:55.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:55 smithi007 ceph-mon[32077]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:51:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:55 smithi189 ceph-mon[36616]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:56.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:55 smithi175 ceph-mon[36886]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:51:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:56 smithi007 ceph-mon[32077]: pgmap v386: 129 pgs: 129 active+clean; 590 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-13T06:51:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:56 smithi007 ceph-mon[32077]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-13T06:51:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:56 smithi189 ceph-mon[36616]: pgmap v386: 129 pgs: 129 active+clean; 590 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-13T06:51:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:56 smithi189 ceph-mon[36616]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-13T06:51:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:56 smithi175 ceph-mon[36886]: pgmap v386: 129 pgs: 129 active+clean; 590 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-13T06:51:57.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:56 smithi175 ceph-mon[36886]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 -- 172.21.15.7:0/590981095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03300746c0 msgr2=0x7f0330074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 --2- 172.21.15.7:0/590981095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03300746c0 0x7f0330074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0320009990 tx=0x7f0320009ca0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 -- 172.21.15.7:0/590981095 shutdown_connections 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 --2- 172.21.15.7:0/590981095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0330113660 0x7f0330115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 --2- 172.21.15.7:0/590981095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03300746c0 0x7f0330074aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 --2- 172.21.15.7:0/590981095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f0330113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 -- 172.21.15.7:0/590981095 >> 172.21.15.7:0/590981095 conn(0x7f03300795f0 msgr2=0x7f0330079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.460+0000 7f0334e00700 1 -- 172.21.15.7:0/590981095 shutdown_connections 2024-06-13T06:51:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.461+0000 7f0334e00700 1 -- 172.21.15.7:0/590981095 wait complete. 2024-06-13T06:51:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.462+0000 7f0334e00700 1 Processor -- start 2024-06-13T06:51:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.462+0000 7f0334e00700 1 -- start start 2024-06-13T06:51:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 0x7f033019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330113660 0x7f03301a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0330118e10 con 0x7f03300746c0 2024-06-13T06:51:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0330118c90 con 0x7f0330074fe0 2024-06-13T06:51:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0334e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0330118f90 con 0x7f0330113660 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0325400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f032de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 0x7f033019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0325400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37792/0 (socket says 172.21.15.7:37792) 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f0325400700 1 -- 172.21.15.7:0/2405555420 learned_addr learned my addr 172.21.15.7:0/2405555420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.463+0000 7f032e800700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330113660 0x7f03301a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 -- 172.21.15.7:0/2405555420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330113660 msgr2=0x7f03301a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330113660 0x7f03301a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 -- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 msgr2=0x7f033019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 0x7f033019f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 -- 172.21.15.7:0/2405555420 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03301a4400 con 0x7f0330074fe0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f032de00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 0x7f033019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0325400700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f031800e9c0 tx=0x7f031800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.464+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800cbd0 con 0x7f0330074fe0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.465+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031800ee50 con 0x7f0330074fe0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.465+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301a46f0 con 0x7f0330074fe0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.465+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318005680 con 0x7f0330074fe0 2024-06-13T06:51:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.465+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301abf70 con 0x7f0330074fe0 2024-06-13T06:51:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.466+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330069180 con 0x7f0330074fe0 2024-06-13T06:51:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.468+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0318044b60 con 0x7f0330074fe0 2024-06-13T06:51:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.469+0000 7f0327400700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 0x7f03140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:51:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.469+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f03180cf7a0 con 0x7f0330074fe0 2024-06-13T06:51:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.469+0000 7f032de00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 0x7f03140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:51:58.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.470+0000 7f032de00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 0x7f03140a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f032000b2e0 tx=0x7f0320005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:51:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.471+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f031809c200 con 0x7f0330074fe0 2024-06-13T06:51:58.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.736+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0330112b30 con 0x7f03140a2d30 2024-06-13T06:51:58.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.780+0000 7f0327400700 1 -- 172.21.15.7:0/2405555420 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0330112b30 con 0x7f03140a2d30 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 msgr2=0x7f03140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 0x7f03140a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f032000b2e0 tx=0x7f0320005fd0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 msgr2=0x7f033019f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f031800e9c0 tx=0x7f031800ecd0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 shutdown_connections 2024-06-13T06:51:58.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330113660 0x7f03301a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03300746c0 0x7f033019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03140a2d30 0x7f03140a51f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 --2- 172.21.15.7:0/2405555420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0330074fe0 0x7f033019f930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 >> 172.21.15.7:0/2405555420 conn(0x7f03300795f0 msgr2=0x7f0330112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 shutdown_connections 2024-06-13T06:51:58.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:51:58.784+0000 7f0334e00700 1 -- 172.21.15.7:0/2405555420 wait complete. 2024-06-13T06:51:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:58 smithi007 ceph-mon[32077]: pgmap v387: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:51:59.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:58 smithi189 ceph-mon[36616]: pgmap v387: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:51:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:58 smithi175 ceph-mon[36886]: pgmap v387: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:51:59.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-13T06:52:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:51:59 smithi007 ceph-mon[32077]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:51:59 smithi189 ceph-mon[36616]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:51:59 smithi175 ceph-mon[36886]: from='client.24719 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:01.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:00 smithi007 ceph-mon[32077]: pgmap v388: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:52:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:00 smithi189 ceph-mon[36616]: pgmap v388: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:52:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:00 smithi175 ceph-mon[36886]: pgmap v388: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 -- 172.21.15.7:0/2635526460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 msgr2=0x7f518810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 --2- 172.21.15.7:0/2635526460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518810aa90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5178009990 tx=0x7f5178009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 -- 172.21.15.7:0/2635526460 shutdown_connections 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 --2- 172.21.15.7:0/2635526460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5188105070 0x7f5188111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 --2- 172.21.15.7:0/2635526460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518810aa90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 --2- 172.21.15.7:0/2635526460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51881046b0 0x7f5188104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 -- 172.21.15.7:0/2635526460 >> 172.21.15.7:0/2635526460 conn(0x7f51881001a0 msgr2=0x7f51881025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 -- 172.21.15.7:0/2635526460 shutdown_connections 2024-06-13T06:52:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.414+0000 7f518ea00700 1 -- 172.21.15.7:0/2635526460 wait complete. 2024-06-13T06:52:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.415+0000 7f518ea00700 1 Processor -- start 2024-06-13T06:52:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 -- start start 2024-06-13T06:52:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 0x7f518819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 0x7f518819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5188114a50 con 0x7f518810a6b0 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51881148d0 con 0x7f5188105070 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ea00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5188114bd0 con 0x7f51881046b0 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.416+0000 7f518ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 0x7f518819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 0x7f518819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f518ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34028/0 (socket says 172.21.15.7:34028) 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f518ca00700 1 -- 172.21.15.7:0/823261145 learned_addr learned my addr 172.21.15.7:0/823261145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 -- 172.21.15.7:0/823261145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 msgr2=0x7f518819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 0x7f518819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 -- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 msgr2=0x7f518819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187400700 1 -- 172.21.15.7:0/823261145 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518819fe90 con 0x7f5188105070 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f518ca00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.417+0000 7f5187e00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 0x7f518819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f5187400700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 0x7f518819b3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f517400e9f0 tx=0x7f517400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517400c9c0 con 0x7f5188105070 2024-06-13T06:52:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51881a0180 con 0x7f5188105070 2024-06-13T06:52:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f517400cb60 con 0x7f5188105070 2024-06-13T06:52:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174018680 con 0x7f5188105070 2024-06-13T06:52:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.418+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a7a00 con 0x7f5188105070 2024-06-13T06:52:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.420+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5188069180 con 0x7f5188105070 2024-06-13T06:52:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.422+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5174010430 con 0x7f5188105070 2024-06-13T06:52:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.423+0000 7f5184c00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 0x7f516c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.424+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f51740cf680 con 0x7f5188105070 2024-06-13T06:52:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.424+0000 7f5187e00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 0x7f516c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.424+0000 7f5187e00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 0x7f516c0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f517800b2e0 tx=0x7f5178005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.428+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f517409c0e0 con 0x7f5188105070 2024-06-13T06:52:02.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.699+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f51880628d0 con 0x7f516c0a2cb0 2024-06-13T06:52:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.712+0000 7f5184c00700 1 -- 172.21.15.7:0/823261145 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f51880628d0 con 0x7f516c0a2cb0 2024-06-13T06:52:02.712 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/ded40084-ee99-4b2a-9dac-ad04dbf7cf2c 2024-06-13T06:52:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.715+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 msgr2=0x7f516c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.715+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 0x7f516c0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f517800b2e0 tx=0x7f5178005fd0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.715+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 msgr2=0x7f518819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.715+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 0x7f518819b3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f517400e9f0 tx=0x7f517400ed00 comp rx=0 tx=0).stop 2024-06-13T06:52:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 shutdown_connections 2024-06-13T06:52:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f518810a6b0 0x7f518819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51881046b0 0x7f518819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f516c0a2cb0 0x7f516c0a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 --2- 172.21.15.7:0/823261145 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5188105070 0x7f518819b3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 >> 172.21.15.7:0/823261145 conn(0x7f51881001a0 msgr2=0x7f51881006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 shutdown_connections 2024-06-13T06:52:02.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:02.716+0000 7f518ea00700 1 -- 172.21.15.7:0/823261145 wait complete. 2024-06-13T06:52:03.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:02 smithi007 ceph-mon[32077]: pgmap v389: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-13T06:52:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:02 smithi189 ceph-mon[36616]: pgmap v389: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-13T06:52:03.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:02 smithi175 ceph-mon[36886]: pgmap v389: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-13T06:52:03.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-13T06:52:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:04.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:03 smithi007 ceph-mon[32077]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:03 smithi189 ceph-mon[36616]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:04.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:03 smithi175 ceph-mon[36886]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:05.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:04 smithi007 ceph-mon[32077]: pgmap v390: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:04 smithi189 ceph-mon[36616]: pgmap v390: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:04 smithi175 ceph-mon[36886]: pgmap v390: 129 pgs: 129 active+clean; 605 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 -- 172.21.15.7:0/2817526264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484102620 msgr2=0x7f5484102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 --2- 172.21.15.7:0/2817526264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484102620 0x7f5484102a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54740099c0 tx=0x7f5474009cd0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 -- 172.21.15.7:0/2817526264 shutdown_connections 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 --2- 172.21.15.7:0/2817526264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484107700 0x7f548410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 --2- 172.21.15.7:0/2817526264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484102620 0x7f5484102a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 --2- 172.21.15.7:0/2817526264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102fd0 0x7f5484107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 -- 172.21.15.7:0/2817526264 >> 172.21.15.7:0/2817526264 conn(0x7f54840fdf10 msgr2=0x7f5484100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 -- 172.21.15.7:0/2817526264 shutdown_connections 2024-06-13T06:52:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.364+0000 7f5489200700 1 -- 172.21.15.7:0/2817526264 wait complete. 2024-06-13T06:52:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.365+0000 7f5489200700 1 Processor -- start 2024-06-13T06:52:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 -- start start 2024-06-13T06:52:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484102fd0 0x7f548419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484107700 0x7f54841989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54841126e0 con 0x7f5484102fd0 2024-06-13T06:52:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5484112560 con 0x7f5484102620 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5489200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5484112860 con 0x7f5484107700 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f5483400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.366+0000 7f547aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484102fd0 0x7f548419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45624/0 (socket says 172.21.15.7:45624) 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 -- 172.21.15.7:0/2509567463 learned_addr learned my addr 172.21.15.7:0/2509567463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483e00700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484107700 0x7f54841989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 -- 172.21.15.7:0/2509567463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484107700 msgr2=0x7f54841989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484107700 0x7f54841989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 -- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484102fd0 msgr2=0x7f548419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484102fd0 0x7f548419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 -- 172.21.15.7:0/2509567463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5484199140 con 0x7f5484102620 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.367+0000 7f5483400700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5474006010 tx=0x7f5474003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.368+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474003680 con 0x7f5484102620 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.368+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841993d0 con 0x7f5484102620 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.368+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841a56f0 con 0x7f5484102620 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.368+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5474003e50 con 0x7f5484102620 2024-06-13T06:52:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.368+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400fe10 con 0x7f5484102620 2024-06-13T06:52:06.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.369+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5484069180 con 0x7f5484102620 2024-06-13T06:52:06.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.371+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f547400f460 con 0x7f5484102620 2024-06-13T06:52:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.372+0000 7f5480c00700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 0x7f54680a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.372+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54740d00b0 con 0x7f5484102620 2024-06-13T06:52:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.372+0000 7f547aa00700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 0x7f54680a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:06.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.373+0000 7f547aa00700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 0x7f54680a5390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5484100610 tx=0x7f546c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:06.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.374+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5474013070 con 0x7f5484102620 2024-06-13T06:52:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.638+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5484199f10 con 0x7f54680a2ed0 2024-06-13T06:52:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.680+0000 7f5480c00700 1 -- 172.21.15.7:0/2509567463 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5484199f10 con 0x7f54680a2ed0 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.683+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 msgr2=0x7f54680a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.683+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 0x7f54680a5390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5484100610 tx=0x7f546c009260 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.683+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 msgr2=0x7f548419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.683+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5474006010 tx=0x7f5474003a90 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 shutdown_connections 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5484107700 0x7f54841989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54680a2ed0 0x7f54680a5390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5484102620 0x7f548419e820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 --2- 172.21.15.7:0/2509567463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5484102fd0 0x7f548419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 >> 172.21.15.7:0/2509567463 conn(0x7f54840fdf10 msgr2=0x7f5484106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 shutdown_connections 2024-06-13T06:52:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:06.684+0000 7f5489200700 1 -- 172.21.15.7:0/2509567463 wait complete. 2024-06-13T06:52:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:06 smithi007 ceph-mon[32077]: pgmap v391: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-13T06:52:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:06 smithi189 ceph-mon[36616]: pgmap v391: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-13T06:52:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:06 smithi175 ceph-mon[36886]: pgmap v391: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-06-13T06:52:07.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-13T06:52:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:08.198 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:07 smithi007 ceph-mon[32077]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:07 smithi189 ceph-mon[36616]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:07 smithi175 ceph-mon[36886]: from='client.24731 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:09.212 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:08 smithi007 ceph-mon[32077]: pgmap v392: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:52:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:08 smithi189 ceph-mon[36616]: pgmap v392: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:52:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:08 smithi175 ceph-mon[36886]: pgmap v392: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-13T06:52:10.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 -- 172.21.15.7:0/3976234924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 msgr2=0x7f9950111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 --2- 172.21.15.7:0/3976234924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f9950111280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9948009f90 tx=0x7f994800b330 comp rx=0 tx=0).stop 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 -- 172.21.15.7:0/3976234924 shutdown_connections 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 --2- 172.21.15.7:0/3976234924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f9950111280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 --2- 172.21.15.7:0/3976234924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 0x7f9950074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 --2- 172.21.15.7:0/3976234924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 -- 172.21.15.7:0/3976234924 >> 172.21.15.7:0/3976234924 conn(0x7f99500795f0 msgr2=0x7f9950079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 -- 172.21.15.7:0/3976234924 shutdown_connections 2024-06-13T06:52:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.322+0000 7f9957c00700 1 -- 172.21.15.7:0/3976234924 wait complete. 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.323+0000 7f9957c00700 1 Processor -- start 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.323+0000 7f9957c00700 1 -- start start 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 0x7f995019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f995019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99501149d0 con 0x7f9950074fe0 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9950114850 con 0x7f99500746c0 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9957c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9950114b50 con 0x7f995010ee90 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9947e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9947e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34068/0 (socket says 172.21.15.7:34068) 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9947e00700 1 -- 172.21.15.7:0/1299973840 learned_addr learned my addr 172.21.15.7:0/1299973840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9955c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f995019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.324+0000 7f9955200700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 0x7f995019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 -- 172.21.15.7:0/1299973840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 msgr2=0x7f995019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f995019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 -- 172.21.15.7:0/1299973840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 msgr2=0x7f995019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 0x7f995019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 -- 172.21.15.7:0/1299973840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995019fdd0 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9955c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f995019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f9947e00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f993800ea50 tx=0x7f993800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.325+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993800cbd0 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.326+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993800cd70 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.326+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99501a00c0 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.326+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938018750 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.326+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501a7940 con 0x7f9950074fe0 2024-06-13T06:52:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.327+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9950069180 con 0x7f9950074fe0 2024-06-13T06:52:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.329+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f993801f020 con 0x7f9950074fe0 2024-06-13T06:52:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.330+0000 7f994e000700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 0x7f99340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.331+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f99380cf6d0 con 0x7f9950074fe0 2024-06-13T06:52:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.331+0000 7f9955200700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 0x7f99340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.331+0000 7f9955200700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 0x7f99340a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9940009750 tx=0x7f9940009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.335+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9938053850 con 0x7f9950074fe0 2024-06-13T06:52:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.609+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f99500ffc70 con 0x7f99340a2cb0 2024-06-13T06:52:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.621+0000 7f994e000700 1 -- 172.21.15.7:0/1299973840 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f99500ffc70 con 0x7f99340a2cb0 2024-06-13T06:52:10.620 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.624+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 msgr2=0x7f99340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.624+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 0x7f99340a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9940009750 tx=0x7f9940009310 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.624+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 msgr2=0x7f995019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.624+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f993800ea50 tx=0x7f993800ed60 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.624+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 shutdown_connections 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f995010ee90 0x7f995019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99340a2cb0 0x7f99340a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500746c0 0x7f995019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 --2- 172.21.15.7:0/1299973840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950074fe0 0x7f995019b300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 >> 172.21.15.7:0/1299973840 conn(0x7f99500795f0 msgr2=0x7f99500ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 shutdown_connections 2024-06-13T06:52:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:10.625+0000 7f9957c00700 1 -- 172.21.15.7:0/1299973840 wait complete. 2024-06-13T06:52:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:10 smithi189 ceph-mon[36616]: pgmap v393: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:10 smithi175 ceph-mon[36886]: pgmap v393: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:11.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:10 smithi007 ceph-mon[32077]: pgmap v393: 129 pgs: 129 active+clean; 610 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-06-13T06:52:11.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:52:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:12.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:11 smithi007 ceph-mon[32077]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:12.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:11 smithi189 ceph-mon[36616]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:12.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:11 smithi175 ceph-mon[36886]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:52:13.237 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:12 smithi189 ceph-mon[36616]: pgmap v394: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-13T06:52:13.336 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:12 smithi007 ceph-mon[32077]: pgmap v394: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-13T06:52:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:12 smithi175 ceph-mon[36886]: pgmap v394: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 -- 172.21.15.7:0/1590292417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810ee90 msgr2=0x7fae48111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 --2- 172.21.15.7:0/1590292417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810ee90 0x7fae48111280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae4000b0a0 tx=0x7fae4000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 -- 172.21.15.7:0/1590292417 shutdown_connections 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 --2- 172.21.15.7:0/1590292417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810ee90 0x7fae48111280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 --2- 172.21.15.7:0/1590292417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae480746c0 0x7fae48074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 --2- 172.21.15.7:0/1590292417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 -- 172.21.15.7:0/1590292417 >> 172.21.15.7:0/1590292417 conn(0x7fae480795f0 msgr2=0x7fae48079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 -- 172.21.15.7:0/1590292417 shutdown_connections 2024-06-13T06:52:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.219+0000 7fae4de00700 1 -- 172.21.15.7:0/1590292417 wait complete. 2024-06-13T06:52:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 Processor -- start 2024-06-13T06:52:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 -- start start 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae481149e0 con 0x7fae480746c0 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae48114860 con 0x7fae48074fe0 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.220+0000 7fae4de00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae48114b60 con 0x7fae4810ee90 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae46a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae46000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41718/0 (socket says 172.21.15.7:41718) 2024-06-13T06:52:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae46a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59652/0 (socket says 172.21.15.7:59652) 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 -- 172.21.15.7:0/4004599956 learned_addr learned my addr 172.21.15.7:0/4004599956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 -- 172.21.15.7:0/4004599956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 msgr2=0x7fae4819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 -- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 msgr2=0x7fae4819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae47400700 1 -- 172.21.15.7:0/4004599956 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4819fde0 con 0x7fae4810ee90 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae46000700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.221+0000 7fae46a00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae47400700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae40006070 tx=0x7fae40004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40021070 con 0x7fae4810ee90 2024-06-13T06:52:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae481a0070 con 0x7fae4810ee90 2024-06-13T06:52:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae40003c00 con 0x7fae4810ee90 2024-06-13T06:52:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400046c0 con 0x7fae4810ee90 2024-06-13T06:52:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.222+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae481a7950 con 0x7fae4810ee90 2024-06-13T06:52:14.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.224+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae48075db0 con 0x7fae4810ee90 2024-06-13T06:52:14.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.225+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fae40007980 con 0x7fae4810ee90 2024-06-13T06:52:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.227+0000 7fae4ca00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 0x7fae2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.227+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fae400d04d0 con 0x7fae4810ee90 2024-06-13T06:52:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.227+0000 7fae46a00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 0x7fae2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.228+0000 7fae46a00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 0x7fae2c0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae38011490 tx=0x7fae38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:14.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.229+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae4009cfc0 con 0x7fae4810ee90 2024-06-13T06:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.504+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae481a7b20 con 0x7fae4810ee90 2024-06-13T06:52:14.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.505+0000 7fae4ca00700 1 -- 172.21.15.7:0/4004599956 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7fae4009c8c0 con 0x7fae4810ee90 2024-06-13T06:52:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:14.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 msgr2=0x7fae2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 0x7fae2c0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae38011490 tx=0x7fae38009260 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 msgr2=0x7fae4819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae40006070 tx=0x7fae40004c60 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 shutdown_connections 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4810ee90 0x7fae4819f6a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae480746c0 0x7fae4819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fae2c0a2cb0 0x7fae2c0a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 --2- 172.21.15.7:0/4004599956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae48074fe0 0x7fae4819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 >> 172.21.15.7:0/4004599956 conn(0x7fae480795f0 msgr2=0x7fae480ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.508+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 shutdown_connections 2024-06-13T06:52:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:14.509+0000 7fae4de00700 1 -- 172.21.15.7:0/4004599956 wait complete. 2024-06-13T06:52:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:52:15.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:14 smithi007 ceph-mon[32077]: pgmap v395: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:15.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:14 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4004599956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:52:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:14 smithi189 ceph-mon[36616]: pgmap v395: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:14 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4004599956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:52:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:14 smithi175 ceph-mon[36886]: pgmap v395: 129 pgs: 129 active+clean; 634 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:14 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4004599956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:52:15.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:52:15.837 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:16.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:15 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:16.251 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:15 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:16.272 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:15 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:52:17.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: pgmap v396: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-06-13T06:52:17.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:17 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: pgmap v396: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:17 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: pgmap v396: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:17 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 -- 172.21.15.7:0/2082407513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c10a6d0 msgr2=0x7f256c10aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 --2- 172.21.15.7:0/2082407513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c10a6d0 0x7f256c10aab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 -- 172.21.15.7:0/2082407513 shutdown_connections 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 --2- 172.21.15.7:0/2082407513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c105090 0x7f256c1115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 --2- 172.21.15.7:0/2082407513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c10a6d0 0x7f256c10aab0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 --2- 172.21.15.7:0/2082407513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 -- 172.21.15.7:0/2082407513 >> 172.21.15.7:0/2082407513 conn(0x7f256c1001c0 msgr2=0x7f256c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 -- 172.21.15.7:0/2082407513 shutdown_connections 2024-06-13T06:52:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.017+0000 7f2570e00700 1 -- 172.21.15.7:0/2082407513 wait complete. 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.019+0000 7f2570e00700 1 Processor -- start 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.019+0000 7f2570e00700 1 -- start start 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c105090 0x7f256c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 0x7f256c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f256c114a00 con 0x7f256c1046d0 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f256c114880 con 0x7f256c10a6d0 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2570e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f256c114b80 con 0x7f256c105090 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2569e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2569e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59674/0 (socket says 172.21.15.7:59674) 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2569e00700 1 -- 172.21.15.7:0/1100554430 learned_addr learned my addr 172.21.15.7:0/1100554430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f2569400700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c105090 0x7f256c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.020+0000 7f256a800700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 0x7f256c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 -- 172.21.15.7:0/1100554430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c105090 msgr2=0x7f256c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c105090 0x7f256c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 -- 172.21.15.7:0/1100554430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 msgr2=0x7f256c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 0x7f256c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 -- 172.21.15.7:0/1100554430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c19fe20 con 0x7f256c1046d0 2024-06-13T06:52:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f256a800700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 0x7f256c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2569e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c027070 con 0x7f256c1046d0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c1a00b0 con 0x7f256c1046d0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.021+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c003d70 con 0x7f256c1046d0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.022+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c00f460 con 0x7f256c1046d0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.022+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c1a7990 con 0x7f256c1046d0 2024-06-13T06:52:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.023+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c10cc60 con 0x7f256c1046d0 2024-06-13T06:52:18.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.025+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f255c02a070 con 0x7f256c1046d0 2024-06-13T06:52:18.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.026+0000 7f2562a00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 0x7f25500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:18.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.026+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f255c0d0290 con 0x7f256c1046d0 2024-06-13T06:52:18.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.027+0000 7f2569400700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 0x7f25500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:18.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.027+0000 7f2569400700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 0x7f25500a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f256c109c60 tx=0x7f2558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:18.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.031+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f255c013070 con 0x7f256c1046d0 2024-06-13T06:52:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.269+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f256c19c170 con 0x7f256c1046d0 2024-06-13T06:52:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.273+0000 7f2562a00700 1 -- 172.21.15.7:0/1100554430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f255c09cd80 con 0x7f256c1046d0 2024-06-13T06:52:18.274 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:18.274 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:52:18.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 msgr2=0x7f25500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:18.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 0x7f25500a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f256c109c60 tx=0x7f2558009260 comp rx=0 tx=0).stop 2024-06-13T06:52:18.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 msgr2=0x7f256c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f255c000c00 tx=0x7f255c00bab0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 shutdown_connections 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f25500a2d30 0x7f25500a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.276+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f256c10a6d0 0x7f256c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.277+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1046d0 0x7f256c19ae10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.277+0000 7f2570e00700 1 --2- 172.21.15.7:0/1100554430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f256c105090 0x7f256c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.277+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 >> 172.21.15.7:0/1100554430 conn(0x7f256c1001c0 msgr2=0x7f256c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:18.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.277+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 shutdown_connections 2024-06-13T06:52:18.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:18.277+0000 7f2570e00700 1 -- 172.21.15.7:0/1100554430 wait complete. 2024-06-13T06:52:18.602 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:18 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1100554430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:18.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:18 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1100554430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:18.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:18 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1100554430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:19.451 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-13T06:52:19.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:52:19.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:19 smithi007 ceph-mon[32077]: pgmap v397: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:19 smithi189 ceph-mon[36616]: pgmap v397: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:19 smithi175 ceph-mon[36886]: pgmap v397: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-06-13T06:52:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:21.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:21 smithi007 ceph-mon[32077]: pgmap v398: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-06-13T06:52:21.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:21.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:21 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:21 smithi189 ceph-mon[36616]: pgmap v398: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-06-13T06:52:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:21 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:21 smithi175 ceph-mon[36886]: pgmap v398: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-06-13T06:52:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:21 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 -- 172.21.15.7:0/4102242262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 msgr2=0x7f1c0810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 --2- 172.21.15.7:0/4102242262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0810f3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b0d0 tx=0x7f1c0000b3e0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 -- 172.21.15.7:0/4102242262 shutdown_connections 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 --2- 172.21.15.7:0/4102242262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0810f3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 --2- 172.21.15.7:0/4102242262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c08102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 --2- 172.21.15.7:0/4102242262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 0x7f1c08107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 -- 172.21.15.7:0/4102242262 >> 172.21.15.7:0/4102242262 conn(0x7f1c080795f0 msgr2=0x7f1c08079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 -- 172.21.15.7:0/4102242262 shutdown_connections 2024-06-13T06:52:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.041+0000 7f1c0de00700 1 -- 172.21.15.7:0/4102242262 wait complete. 2024-06-13T06:52:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.042+0000 7f1c0de00700 1 Processor -- start 2024-06-13T06:52:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.042+0000 7f1c0de00700 1 -- start start 2024-06-13T06:52:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 0x7f1c0819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081127d0 con 0x7f1c08107840 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08112650 con 0x7f1c08102740 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c0de00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08112950 con 0x7f1c081030f0 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c06a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c06a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53690/0 (socket says 172.21.15.7:53690) 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c06a00700 1 -- 172.21.15.7:0/2219055080 learned_addr learned my addr 172.21.15.7:0/2219055080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1c07400700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.043+0000 7f1bffe00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 0x7f1c0819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 -- 172.21.15.7:0/2219055080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 msgr2=0x7f1c0819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 0x7f1c0819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 -- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 msgr2=0x7f1c0819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 -- 172.21.15.7:0/2219055080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0819fe30 con 0x7f1c08102740 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c06a00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800e9a0 tx=0x7f1bf800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800ca10 con 0x7f1c08102740 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf800cbb0 con 0x7f1c08102740 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a0120 con 0x7f1c08102740 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.044+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8018680 con 0x7f1c08102740 2024-06-13T06:52:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.045+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a79a0 con 0x7f1c08102740 2024-06-13T06:52:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.045+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c08069180 con 0x7f1c08102740 2024-06-13T06:52:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.048+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1bf8010430 con 0x7f1c08102740 2024-06-13T06:52:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.049+0000 7f1c0ca00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 0x7f1bec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.049+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1bf80cfc60 con 0x7f1c08102740 2024-06-13T06:52:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.049+0000 7f1bffe00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 0x7f1bec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:22.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.050+0000 7f1bffe00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 0x7f1bec0a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1c0810f800 tx=0x7f1bf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:22.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.051+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1bf809c6c0 con 0x7f1c08102740 2024-06-13T06:52:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.279+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c08100a60 con 0x7f1bec0a2cb0 2024-06-13T06:52:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.315+0000 7f1c0ca00700 1 -- 172.21.15.7:0/2219055080 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340648 (secure 0 0 0) 0x7f1c08100a60 con 0x7f1bec0a2cb0 2024-06-13T06:52:22.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 msgr2=0x7f1bec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 0x7f1bec0a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1c0810f800 tx=0x7f1bf0009260 comp rx=0 tx=0).stop 2024-06-13T06:52:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 msgr2=0x7f1c0819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800e9a0 tx=0x7f1bf800ecb0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 shutdown_connections 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c08107840 0x7f1c0819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1bec0a2cb0 0x7f1bec0a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c08102740 0x7f1c0819ae20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 --2- 172.21.15.7:0/2219055080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c081030f0 0x7f1c0819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 >> 172.21.15.7:0/2219055080 conn(0x7f1c080795f0 msgr2=0x7f1c08100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 shutdown_connections 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:22.322+0000 7f1c0de00700 1 -- 172.21.15.7:0/2219055080 wait complete. 2024-06-13T06:52:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:52:22.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:22.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:22.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:22.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:22.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:22.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:22.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:52:22.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:52:22.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:52:23.143 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-06-13T06:52:21.659787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658660,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":362,"num_read_kb":301,"num_write":352,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378912,"kb_used_data":13120,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562912,"statfs":{"total":1151940427776,"available":1151552421888,"internally_reserved":0,"allocated":13434880,"data_stored":8595009,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33715,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":80,"num_write":19,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005676"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884567+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:51:00.884567+0000","last_peered":"2024-06-13T06:51:00.884567+0000","last_clean":"2024-06-13T06:51:00.884567+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:51:00.884567+0000","last_undegraded":"2024-06-13T06:51:00.884567+0000","last_fullsized":"2024-06-13T06:51:00.884567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889046+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:51:00.889046+0000","last_peered":"2024-06-13T06:51:00.889046+0000","last_clean":"2024-06-13T06:51:00.889046+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:51:00.889046+0000","last_undegraded":"2024-06-13T06:51:00.889046+0000","last_fullsized":"2024-06-13T06:51:00.889046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.885309+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:51:00.885309+0000","last_peered":"2024-06-13T06:51:00.885309+0000","last_clean":"2024-06-13T06:51:00.885309+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:51:00.885309+0000","last_undegraded":"2024-06-13T06:51:00.885309+0000","last_fullsized":"2024-06-13T06:51:00.885309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257890+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:51:01.257890+0000","last_peered":"2024-06-13T06:51:01.257890+0000","last_clean":"2024-06-13T06:51:01.257890+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:51:01.257890+0000","last_undegraded":"2024-06-13T06:51:01.257890+0000","last_fullsized":"2024-06-13T06:51:01.257890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254250+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:51:01.254250+0000","last_peered":"2024-06-13T06:51:01.254250+0000","last_clean":"2024-06-13T06:51:01.254250+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:51:01.254250+0000","last_undegraded":"2024-06-13T06:51:01.254250+0000","last_fullsized":"2024-06-13T06:51:01.254250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253432+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:51:01.253432+0000","last_peered":"2024-06-13T06:51:01.253432+0000","last_clean":"2024-06-13T06:51:01.253432+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:51:01.253432+0000","last_undegraded":"2024-06-13T06:51:01.253432+0000","last_fullsized":"2024-06-13T06:51:01.253432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882061+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:51:00.882061+0000","last_peered":"2024-06-13T06:51:00.882061+0000","last_clean":"2024-06-13T06:51:00.882061+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:51:00.882061+0000","last_undegraded":"2024-06-13T06:51:00.882061+0000","last_fullsized":"2024-06-13T06:51:00.882061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230293+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:51:01.230293+0000","last_peered":"2024-06-13T06:51:01.230293+0000","last_clean":"2024-06-13T06:51:01.230293+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:51:01.230293+0000","last_undegraded":"2024-06-13T06:51:01.230293+0000","last_fullsized":"2024-06-13T06:51:01.230293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943973+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:51:00.943973+0000","last_peered":"2024-06-13T06:51:00.943973+0000","last_clean":"2024-06-13T06:51:00.943973+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:51:00.943973+0000","last_undegraded":"2024-06-13T06:51:00.943973+0000","last_fullsized":"2024-06-13T06:51:00.943973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254002+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:51:01.254002+0000","last_peered":"2024-06-13T06:51:01.254002+0000","last_clean":"2024-06-13T06:51:01.254002+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:51:01.254002+0000","last_undegraded":"2024-06-13T06:51:01.254002+0000","last_fullsized":"2024-06-13T06:51:01.254002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882378+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:51:00.882378+0000","last_peered":"2024-06-13T06:51:00.882378+0000","last_clean":"2024-06-13T06:51:00.882378+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:51:00.882378+0000","last_undegraded":"2024-06-13T06:51:00.882378+0000","last_fullsized":"2024-06-13T06:51:00.882378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257838+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:51:01.257838+0000","last_peered":"2024-06-13T06:51:01.257838+0000","last_clean":"2024-06-13T06:51:01.257838+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:51:01.257838+0000","last_undegraded":"2024-06-13T06:51:01.257838+0000","last_fullsized":"2024-06-13T06:51:01.257838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960929+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:51:00.960929+0000","last_peered":"2024-06-13T06:51:00.960929+0000","last_clean":"2024-06-13T06:51:00.960929+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:51:00.960929+0000","last_undegraded":"2024-06-13T06:51:00.960929+0000","last_fullsized":"2024-06-13T06:51:00.960929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884779+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:51:00.884779+0000","last_peered":"2024-06-13T06:51:00.884779+0000","last_clean":"2024-06-13T06:51:00.884779+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:51:00.884779+0000","last_undegraded":"2024-06-13T06:51:00.884779+0000","last_fullsized":"2024-06-13T06:51:00.884779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883642+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:51:00.883642+0000","last_peered":"2024-06-13T06:51:00.883642+0000","last_clean":"2024-06-13T06:51:00.883642+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:51:00.883642+0000","last_undegraded":"2024-06-13T06:51:00.883642+0000","last_fullsized":"2024-06-13T06:51:00.883642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943121+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:51:00.943121+0000","last_peered":"2024-06-13T06:51:00.943121+0000","last_clean":"2024-06-13T06:51:00.943121+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:51:00.943121+0000","last_undegraded":"2024-06-13T06:51:00.943121+0000","last_fullsized":"2024-06-13T06:51:00.943121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884305+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:51:00.884305+0000","last_peered":"2024-06-13T06:51:00.884305+0000","last_clean":"2024-06-13T06:51:00.884305+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:51:00.884305+0000","last_undegraded":"2024-06-13T06:51:00.884305+0000","last_fullsized":"2024-06-13T06:51:00.884305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892374+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:51:00.892374+0000","last_peered":"2024-06-13T06:51:00.892374+0000","last_clean":"2024-06-13T06:51:00.892374+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:51:00.892374+0000","last_undegraded":"2024-06-13T06:51:00.892374+0000","last_fullsized":"2024-06-13T06:51:00.892374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882200+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:51:00.882200+0000","last_peered":"2024-06-13T06:51:00.882200+0000","last_clean":"2024-06-13T06:51:00.882200+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:51:00.882200+0000","last_undegraded":"2024-06-13T06:51:00.882200+0000","last_fullsized":"2024-06-13T06:51:00.882200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881672+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:51:00.881672+0000","last_peered":"2024-06-13T06:51:00.881672+0000","last_clean":"2024-06-13T06:51:00.881672+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:51:00.881672+0000","last_undegraded":"2024-06-13T06:51:00.881672+0000","last_fullsized":"2024-06-13T06:51:00.881672+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835279+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:51:45.835279+0000","last_peered":"2024-06-13T06:51:45.835279+0000","last_clean":"2024-06-13T06:51:45.835279+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:51:45.835279+0000","last_undegraded":"2024-06-13T06:51:45.835279+0000","last_fullsized":"2024-06-13T06:51:45.835279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231029+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:51:01.231029+0000","last_peered":"2024-06-13T06:51:01.231029+0000","last_clean":"2024-06-13T06:51:01.231029+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:51:01.231029+0000","last_undegraded":"2024-06-13T06:51:01.231029+0000","last_fullsized":"2024-06-13T06:51:01.231029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891147+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:51:00.891147+0000","last_peered":"2024-06-13T06:51:00.891147+0000","last_clean":"2024-06-13T06:51:00.891147+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:51:00.891147+0000","last_undegraded":"2024-06-13T06:51:00.891147+0000","last_fullsized":"2024-06-13T06:51:00.891147+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803500+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:51:41.803500+0000","last_peered":"2024-06-13T06:51:41.803500+0000","last_clean":"2024-06-13T06:51:41.803500+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:51:41.803500+0000","last_undegraded":"2024-06-13T06:51:41.803500+0000","last_fullsized":"2024-06-13T06:51:41.803500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888998+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:51:00.888998+0000","last_peered":"2024-06-13T06:51:00.888998+0000","last_clean":"2024-06-13T06:51:00.888998+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:51:00.888998+0000","last_undegraded":"2024-06-13T06:51:00.888998+0000","last_fullsized":"2024-06-13T06:51:00.888998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891585+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891585+0000","last_peered":"2024-06-13T06:51:00.891585+0000","last_clean":"2024-06-13T06:51:00.891585+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891585+0000","last_undegraded":"2024-06-13T06:51:00.891585+0000","last_fullsized":"2024-06-13T06:51:00.891585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.797974+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:51:41.797974+0000","last_peered":"2024-06-13T06:51:41.797974+0000","last_clean":"2024-06-13T06:51:41.797974+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:51:41.797974+0000","last_undegraded":"2024-06-13T06:51:41.797974+0000","last_fullsized":"2024-06-13T06:51:41.797974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943555+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:51:00.943555+0000","last_peered":"2024-06-13T06:51:00.943555+0000","last_clean":"2024-06-13T06:51:00.943555+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:51:00.943555+0000","last_undegraded":"2024-06-13T06:51:00.943555+0000","last_fullsized":"2024-06-13T06:51:00.943555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891169+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891169+0000","last_peered":"2024-06-13T06:51:00.891169+0000","last_clean":"2024-06-13T06:51:00.891169+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891169+0000","last_undegraded":"2024-06-13T06:51:00.891169+0000","last_fullsized":"2024-06-13T06:51:00.891169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253960+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:51:01.253960+0000","last_peered":"2024-06-13T06:51:01.253960+0000","last_clean":"2024-06-13T06:51:01.253960+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:51:01.253960+0000","last_undegraded":"2024-06-13T06:51:01.253960+0000","last_fullsized":"2024-06-13T06:51:01.253960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253490+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:51:01.253490+0000","last_peered":"2024-06-13T06:51:01.253490+0000","last_clean":"2024-06-13T06:51:01.253490+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:51:01.253490+0000","last_undegraded":"2024-06-13T06:51:01.253490+0000","last_fullsized":"2024-06-13T06:51:01.253490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881893+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881893+0000","last_peered":"2024-06-13T06:51:00.881893+0000","last_clean":"2024-06-13T06:51:00.881893+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881893+0000","last_undegraded":"2024-06-13T06:51:00.881893+0000","last_fullsized":"2024-06-13T06:51:00.881893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":82,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.881570+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:52:18.881570+0000","last_peered":"2024-06-13T06:52:18.881570+0000","last_clean":"2024-06-13T06:52:18.881570+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:52:18.881570+0000","last_undegraded":"2024-06-13T06:52:18.881570+0000","last_fullsized":"2024-06-13T06:52:18.881570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254226+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:51:01.254226+0000","last_peered":"2024-06-13T06:51:01.254226+0000","last_clean":"2024-06-13T06:51:01.254226+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:51:01.254226+0000","last_undegraded":"2024-06-13T06:51:01.254226+0000","last_fullsized":"2024-06-13T06:51:01.254226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882054+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882054+0000","last_peered":"2024-06-13T06:51:00.882054+0000","last_clean":"2024-06-13T06:51:00.882054+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882054+0000","last_undegraded":"2024-06-13T06:51:00.882054+0000","last_fullsized":"2024-06-13T06:51:00.882054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.800947+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:51:41.800947+0000","last_peered":"2024-06-13T06:51:41.800947+0000","last_clean":"2024-06-13T06:51:41.800947+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:51:41.800947+0000","last_undegraded":"2024-06-13T06:51:41.800947+0000","last_fullsized":"2024-06-13T06:51:41.800947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253297+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:51:01.253297+0000","last_peered":"2024-06-13T06:51:01.253297+0000","last_clean":"2024-06-13T06:51:01.253297+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:51:01.253297+0000","last_undegraded":"2024-06-13T06:51:01.253297+0000","last_fullsized":"2024-06-13T06:51:01.253297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884119+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:51:00.884119+0000","last_peered":"2024-06-13T06:51:00.884119+0000","last_clean":"2024-06-13T06:51:00.884119+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:51:00.884119+0000","last_undegraded":"2024-06-13T06:51:00.884119+0000","last_fullsized":"2024-06-13T06:51:00.884119+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.805631+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:51:41.805631+0000","last_peered":"2024-06-13T06:51:41.805631+0000","last_clean":"2024-06-13T06:51:41.805631+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:51:41.805631+0000","last_undegraded":"2024-06-13T06:51:41.805631+0000","last_fullsized":"2024-06-13T06:51:41.805631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230150+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:51:01.230150+0000","last_peered":"2024-06-13T06:51:01.230150+0000","last_clean":"2024-06-13T06:51:01.230150+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:51:01.230150+0000","last_undegraded":"2024-06-13T06:51:01.230150+0000","last_fullsized":"2024-06-13T06:51:01.230150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888726+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:51:00.888726+0000","last_peered":"2024-06-13T06:51:00.888726+0000","last_clean":"2024-06-13T06:51:00.888726+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:51:00.888726+0000","last_undegraded":"2024-06-13T06:51:00.888726+0000","last_fullsized":"2024-06-13T06:51:00.888726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252905+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:51:01.252905+0000","last_peered":"2024-06-13T06:51:01.252905+0000","last_clean":"2024-06-13T06:51:01.252905+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:51:01.252905+0000","last_undegraded":"2024-06-13T06:51:01.252905+0000","last_fullsized":"2024-06-13T06:51:01.252905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.890749+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:51:00.890749+0000","last_peered":"2024-06-13T06:51:00.890749+0000","last_clean":"2024-06-13T06:51:00.890749+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:51:00.890749+0000","last_undegraded":"2024-06-13T06:51:00.890749+0000","last_fullsized":"2024-06-13T06:51:00.890749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230237+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:51:01.230237+0000","last_peered":"2024-06-13T06:51:01.230237+0000","last_clean":"2024-06-13T06:51:01.230237+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:51:01.230237+0000","last_undegraded":"2024-06-13T06:51:01.230237+0000","last_fullsized":"2024-06-13T06:51:01.230237+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":78,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.880912+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:52:18.880912+0000","last_peered":"2024-06-13T06:52:18.880912+0000","last_clean":"2024-06-13T06:52:18.880912+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:52:18.880912+0000","last_undegraded":"2024-06-13T06:52:18.880912+0000","last_fullsized":"2024-06-13T06:52:18.880912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943956+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:51:00.943956+0000","last_peered":"2024-06-13T06:51:00.943956+0000","last_clean":"2024-06-13T06:51:00.943956+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:51:00.943956+0000","last_undegraded":"2024-06-13T06:51:00.943956+0000","last_fullsized":"2024-06-13T06:51:00.943956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888123+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:51:00.888123+0000","last_peered":"2024-06-13T06:51:00.888123+0000","last_clean":"2024-06-13T06:51:00.888123+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:51:00.888123+0000","last_undegraded":"2024-06-13T06:51:00.888123+0000","last_fullsized":"2024-06-13T06:51:00.888123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779187+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:51:40.779187+0000","last_peered":"2024-06-13T06:51:40.779187+0000","last_clean":"2024-06-13T06:51:40.779187+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:51:40.779187+0000","last_undegraded":"2024-06-13T06:51:40.779187+0000","last_fullsized":"2024-06-13T06:51:40.779187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889997+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:51:00.889997+0000","last_peered":"2024-06-13T06:51:00.889997+0000","last_clean":"2024-06-13T06:51:00.889997+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:51:00.889997+0000","last_undegraded":"2024-06-13T06:51:00.889997+0000","last_fullsized":"2024-06-13T06:51:00.889997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891775+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:51:00.891775+0000","last_peered":"2024-06-13T06:51:00.891775+0000","last_clean":"2024-06-13T06:51:00.891775+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:51:00.891775+0000","last_undegraded":"2024-06-13T06:51:00.891775+0000","last_fullsized":"2024-06-13T06:51:00.891775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781228+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:51:40.781228+0000","last_peered":"2024-06-13T06:51:40.781228+0000","last_clean":"2024-06-13T06:51:40.781228+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:51:40.781228+0000","last_undegraded":"2024-06-13T06:51:40.781228+0000","last_fullsized":"2024-06-13T06:51:40.781228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943664+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:51:00.943664+0000","last_peered":"2024-06-13T06:51:00.943664+0000","last_clean":"2024-06-13T06:51:00.943664+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:51:00.943664+0000","last_undegraded":"2024-06-13T06:51:00.943664+0000","last_fullsized":"2024-06-13T06:51:00.943664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891096+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:51:00.891096+0000","last_peered":"2024-06-13T06:51:00.891096+0000","last_clean":"2024-06-13T06:51:00.891096+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:51:00.891096+0000","last_undegraded":"2024-06-13T06:51:00.891096+0000","last_fullsized":"2024-06-13T06:51:00.891096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":153,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:19.888114+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:52:19.888114+0000","last_peered":"2024-06-13T06:52:19.888114+0000","last_clean":"2024-06-13T06:52:19.888114+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:52:19.888114+0000","last_undegraded":"2024-06-13T06:52:19.888114+0000","last_fullsized":"2024-06-13T06:52:19.888114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64058,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":79,"num_read_kb":102,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254365+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:51:01.254365+0000","last_peered":"2024-06-13T06:51:01.254365+0000","last_clean":"2024-06-13T06:51:01.254365+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:51:01.254365+0000","last_undegraded":"2024-06-13T06:51:01.254365+0000","last_fullsized":"2024-06-13T06:51:01.254365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253801+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:51:01.253801+0000","last_peered":"2024-06-13T06:51:01.253801+0000","last_clean":"2024-06-13T06:51:01.253801+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:51:01.253801+0000","last_undegraded":"2024-06-13T06:51:01.253801+0000","last_fullsized":"2024-06-13T06:51:01.253801+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803271+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:51:41.803271+0000","last_peered":"2024-06-13T06:51:41.803271+0000","last_clean":"2024-06-13T06:51:41.803271+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:51:41.803271+0000","last_undegraded":"2024-06-13T06:51:41.803271+0000","last_fullsized":"2024-06-13T06:51:41.803271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960254+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:51:00.960254+0000","last_peered":"2024-06-13T06:51:00.960254+0000","last_clean":"2024-06-13T06:51:00.960254+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:51:00.960254+0000","last_undegraded":"2024-06-13T06:51:00.960254+0000","last_fullsized":"2024-06-13T06:51:00.960254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883856+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:51:00.883856+0000","last_peered":"2024-06-13T06:51:00.883856+0000","last_clean":"2024-06-13T06:51:00.883856+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:51:00.883856+0000","last_undegraded":"2024-06-13T06:51:00.883856+0000","last_fullsized":"2024-06-13T06:51:00.883856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780954+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:51:40.780954+0000","last_peered":"2024-06-13T06:51:40.780954+0000","last_clean":"2024-06-13T06:51:40.780954+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:51:40.780954+0000","last_undegraded":"2024-06-13T06:51:40.780954+0000","last_fullsized":"2024-06-13T06:51:40.780954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882401+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:51:00.882401+0000","last_peered":"2024-06-13T06:51:00.882401+0000","last_clean":"2024-06-13T06:51:00.882401+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:51:00.882401+0000","last_undegraded":"2024-06-13T06:51:00.882401+0000","last_fullsized":"2024-06-13T06:51:00.882401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892656+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:51:00.892656+0000","last_peered":"2024-06-13T06:51:00.892656+0000","last_clean":"2024-06-13T06:51:00.892656+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:51:00.892656+0000","last_undegraded":"2024-06-13T06:51:00.892656+0000","last_fullsized":"2024-06-13T06:51:00.892656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":79,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:15.897692+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:52:15.897692+0000","last_peered":"2024-06-13T06:52:15.897692+0000","last_clean":"2024-06-13T06:52:15.897692+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:52:15.897692+0000","last_undegraded":"2024-06-13T06:52:15.897692+0000","last_fullsized":"2024-06-13T06:52:15.897692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881815+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:51:00.881815+0000","last_peered":"2024-06-13T06:51:00.881815+0000","last_clean":"2024-06-13T06:51:00.881815+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:51:00.881815+0000","last_undegraded":"2024-06-13T06:51:00.881815+0000","last_fullsized":"2024-06-13T06:51:00.881815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960960+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:51:00.960960+0000","last_peered":"2024-06-13T06:51:00.960960+0000","last_clean":"2024-06-13T06:51:00.960960+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:51:00.960960+0000","last_undegraded":"2024-06-13T06:51:00.960960+0000","last_fullsized":"2024-06-13T06:51:00.960960+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803134+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:51:41.803134+0000","last_peered":"2024-06-13T06:51:41.803134+0000","last_clean":"2024-06-13T06:51:41.803134+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:51:41.803134+0000","last_undegraded":"2024-06-13T06:51:41.803134+0000","last_fullsized":"2024-06-13T06:51:41.803134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253554+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:51:01.253554+0000","last_peered":"2024-06-13T06:51:01.253554+0000","last_clean":"2024-06-13T06:51:01.253554+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:51:01.253554+0000","last_undegraded":"2024-06-13T06:51:01.253554+0000","last_fullsized":"2024-06-13T06:51:01.253554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253431+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:51:01.253431+0000","last_peered":"2024-06-13T06:51:01.253431+0000","last_clean":"2024-06-13T06:51:01.253431+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:51:01.253431+0000","last_undegraded":"2024-06-13T06:51:01.253431+0000","last_fullsized":"2024-06-13T06:51:01.253431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:42.797466+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:51:42.797466+0000","last_peered":"2024-06-13T06:51:42.797466+0000","last_clean":"2024-06-13T06:51:42.797466+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:51:42.797466+0000","last_undegraded":"2024-06-13T06:51:42.797466+0000","last_fullsized":"2024-06-13T06:51:42.797466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884048+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:51:00.884048+0000","last_peered":"2024-06-13T06:51:00.884048+0000","last_clean":"2024-06-13T06:51:00.884048+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:51:00.884048+0000","last_undegraded":"2024-06-13T06:51:00.884048+0000","last_fullsized":"2024-06-13T06:51:00.884048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254044+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:51:01.254044+0000","last_peered":"2024-06-13T06:51:01.254044+0000","last_clean":"2024-06-13T06:51:01.254044+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:51:01.254044+0000","last_undegraded":"2024-06-13T06:51:01.254044+0000","last_fullsized":"2024-06-13T06:51:01.254044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.896197+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:51:00.896197+0000","last_peered":"2024-06-13T06:51:00.896197+0000","last_clean":"2024-06-13T06:51:00.896197+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:51:00.896197+0000","last_undegraded":"2024-06-13T06:51:00.896197+0000","last_fullsized":"2024-06-13T06:51:00.896197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253743+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:51:01.253743+0000","last_peered":"2024-06-13T06:51:01.253743+0000","last_clean":"2024-06-13T06:51:01.253743+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:51:01.253743+0000","last_undegraded":"2024-06-13T06:51:01.253743+0000","last_fullsized":"2024-06-13T06:51:01.253743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960207+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:51:00.960207+0000","last_peered":"2024-06-13T06:51:00.960207+0000","last_clean":"2024-06-13T06:51:00.960207+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:51:00.960207+0000","last_undegraded":"2024-06-13T06:51:00.960207+0000","last_fullsized":"2024-06-13T06:51:00.960207+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799920+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:51:41.799920+0000","last_peered":"2024-06-13T06:51:41.799920+0000","last_clean":"2024-06-13T06:51:41.799920+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:51:41.799920+0000","last_undegraded":"2024-06-13T06:51:41.799920+0000","last_fullsized":"2024-06-13T06:51:41.799920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253558+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:51:01.253558+0000","last_peered":"2024-06-13T06:51:01.253558+0000","last_clean":"2024-06-13T06:51:01.253558+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:51:01.253558+0000","last_undegraded":"2024-06-13T06:51:01.253558+0000","last_fullsized":"2024-06-13T06:51:01.253558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961326+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:51:00.961326+0000","last_peered":"2024-06-13T06:51:00.961326+0000","last_clean":"2024-06-13T06:51:00.961326+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:51:00.961326+0000","last_undegraded":"2024-06-13T06:51:00.961326+0000","last_fullsized":"2024-06-13T06:51:00.961326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253297+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:51:01.253297+0000","last_peered":"2024-06-13T06:51:01.253297+0000","last_clean":"2024-06-13T06:51:01.253297+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:51:01.253297+0000","last_undegraded":"2024-06-13T06:51:01.253297+0000","last_fullsized":"2024-06-13T06:51:01.253297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":81,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:19.904051+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:52:19.904051+0000","last_peered":"2024-06-13T06:52:19.904051+0000","last_clean":"2024-06-13T06:52:19.904051+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:52:19.904051+0000","last_undegraded":"2024-06-13T06:52:19.904051+0000","last_fullsized":"2024-06-13T06:52:19.904051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256415+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:51:01.256415+0000","last_peered":"2024-06-13T06:51:01.256415+0000","last_clean":"2024-06-13T06:51:01.256415+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:51:01.256415+0000","last_undegraded":"2024-06-13T06:51:01.256415+0000","last_fullsized":"2024-06-13T06:51:01.256415+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230499+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:51:01.230499+0000","last_peered":"2024-06-13T06:51:01.230499+0000","last_clean":"2024-06-13T06:51:01.230499+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:51:01.230499+0000","last_undegraded":"2024-06-13T06:51:01.230499+0000","last_fullsized":"2024-06-13T06:51:01.230499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831145+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:51:45.831145+0000","last_peered":"2024-06-13T06:51:45.831145+0000","last_clean":"2024-06-13T06:51:45.831145+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:51:45.831145+0000","last_undegraded":"2024-06-13T06:51:45.831145+0000","last_fullsized":"2024-06-13T06:51:45.831145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881706+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:51:00.881706+0000","last_peered":"2024-06-13T06:51:00.881706+0000","last_clean":"2024-06-13T06:51:00.881706+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:51:00.881706+0000","last_undegraded":"2024-06-13T06:51:00.881706+0000","last_fullsized":"2024-06-13T06:51:00.881706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254422+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:51:01.254422+0000","last_peered":"2024-06-13T06:51:01.254422+0000","last_clean":"2024-06-13T06:51:01.254422+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:51:01.254422+0000","last_undegraded":"2024-06-13T06:51:01.254422+0000","last_fullsized":"2024-06-13T06:51:01.254422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.959715+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:51:00.959715+0000","last_peered":"2024-06-13T06:51:00.959715+0000","last_clean":"2024-06-13T06:51:00.959715+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:51:00.959715+0000","last_undegraded":"2024-06-13T06:51:00.959715+0000","last_fullsized":"2024-06-13T06:51:00.959715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.798470+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:51:41.798470+0000","last_peered":"2024-06-13T06:51:41.798470+0000","last_clean":"2024-06-13T06:51:41.798470+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:51:41.798470+0000","last_undegraded":"2024-06-13T06:51:41.798470+0000","last_fullsized":"2024-06-13T06:51:41.798470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257244+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:51:01.257244+0000","last_peered":"2024-06-13T06:51:01.257244+0000","last_clean":"2024-06-13T06:51:01.257244+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:51:01.257244+0000","last_undegraded":"2024-06-13T06:51:01.257244+0000","last_fullsized":"2024-06-13T06:51:01.257244+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831109+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:51:45.831109+0000","last_peered":"2024-06-13T06:51:45.831109+0000","last_clean":"2024-06-13T06:51:45.831109+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:51:45.831109+0000","last_undegraded":"2024-06-13T06:51:45.831109+0000","last_fullsized":"2024-06-13T06:51:45.831109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960300+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:51:00.960300+0000","last_peered":"2024-06-13T06:51:00.960300+0000","last_clean":"2024-06-13T06:51:00.960300+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:51:00.960300+0000","last_undegraded":"2024-06-13T06:51:00.960300+0000","last_fullsized":"2024-06-13T06:51:00.960300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230124+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:51:01.230124+0000","last_peered":"2024-06-13T06:51:01.230124+0000","last_clean":"2024-06-13T06:51:01.230124+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:51:01.230124+0000","last_undegraded":"2024-06-13T06:51:01.230124+0000","last_fullsized":"2024-06-13T06:51:01.230124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'11","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:12.525920+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:52:12.525920+0000","last_peered":"2024-06-13T06:52:12.525920+0000","last_clean":"2024-06-13T06:52:12.525920+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:52:12.525920+0000","last_undegraded":"2024-06-13T06:52:12.525920+0000","last_fullsized":"2024-06-13T06:52:12.525920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231171+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:51:01.231171+0000","last_peered":"2024-06-13T06:51:01.231171+0000","last_clean":"2024-06-13T06:51:01.231171+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:51:01.231171+0000","last_undegraded":"2024-06-13T06:51:01.231171+0000","last_fullsized":"2024-06-13T06:51:01.231171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881533+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:51:00.881533+0000","last_peered":"2024-06-13T06:51:00.881533+0000","last_clean":"2024-06-13T06:51:00.881533+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:51:00.881533+0000","last_undegraded":"2024-06-13T06:51:00.881533+0000","last_fullsized":"2024-06-13T06:51:00.881533+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779752+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:51:40.779752+0000","last_peered":"2024-06-13T06:51:40.779752+0000","last_clean":"2024-06-13T06:51:40.779752+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:51:40.779752+0000","last_undegraded":"2024-06-13T06:51:40.779752+0000","last_fullsized":"2024-06-13T06:51:40.779752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889312+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:51:00.889312+0000","last_peered":"2024-06-13T06:51:00.889312+0000","last_clean":"2024-06-13T06:51:00.889312+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:51:00.889312+0000","last_undegraded":"2024-06-13T06:51:00.889312+0000","last_fullsized":"2024-06-13T06:51:00.889312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256069+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:51:01.256069+0000","last_peered":"2024-06-13T06:51:01.256069+0000","last_clean":"2024-06-13T06:51:01.256069+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:51:01.256069+0000","last_undegraded":"2024-06-13T06:51:01.256069+0000","last_fullsized":"2024-06-13T06:51:01.256069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779635+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:51:40.779635+0000","last_peered":"2024-06-13T06:51:40.779635+0000","last_clean":"2024-06-13T06:51:40.779635+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:51:40.779635+0000","last_undegraded":"2024-06-13T06:51:40.779635+0000","last_fullsized":"2024-06-13T06:51:40.779635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254263+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:51:01.254263+0000","last_peered":"2024-06-13T06:51:01.254263+0000","last_clean":"2024-06-13T06:51:01.254263+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:51:01.254263+0000","last_undegraded":"2024-06-13T06:51:01.254263+0000","last_fullsized":"2024-06-13T06:51:01.254263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892441+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:51:00.892441+0000","last_peered":"2024-06-13T06:51:00.892441+0000","last_clean":"2024-06-13T06:51:00.892441+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:51:00.892441+0000","last_undegraded":"2024-06-13T06:51:00.892441+0000","last_fullsized":"2024-06-13T06:51:00.892441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780399+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:51:40.780399+0000","last_peered":"2024-06-13T06:51:40.780399+0000","last_clean":"2024-06-13T06:51:40.780399+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:51:40.780399+0000","last_undegraded":"2024-06-13T06:51:40.780399+0000","last_fullsized":"2024-06-13T06:51:40.780399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883660+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:51:00.883660+0000","last_peered":"2024-06-13T06:51:00.883660+0000","last_clean":"2024-06-13T06:51:00.883660+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:51:00.883660+0000","last_undegraded":"2024-06-13T06:51:00.883660+0000","last_fullsized":"2024-06-13T06:51:00.883660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891361+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:51:00.891361+0000","last_peered":"2024-06-13T06:51:00.891361+0000","last_clean":"2024-06-13T06:51:00.891361+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:51:00.891361+0000","last_undegraded":"2024-06-13T06:51:00.891361+0000","last_fullsized":"2024-06-13T06:51:00.891361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257403+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:51:01.257403+0000","last_peered":"2024-06-13T06:51:01.257403+0000","last_clean":"2024-06-13T06:51:01.257403+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:51:01.257403+0000","last_undegraded":"2024-06-13T06:51:01.257403+0000","last_fullsized":"2024-06-13T06:51:01.257403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230515+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:51:01.230515+0000","last_peered":"2024-06-13T06:51:01.230515+0000","last_clean":"2024-06-13T06:51:01.230515+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:51:01.230515+0000","last_undegraded":"2024-06-13T06:51:01.230515+0000","last_fullsized":"2024-06-13T06:51:01.230515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891418+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:51:00.891418+0000","last_peered":"2024-06-13T06:51:00.891418+0000","last_clean":"2024-06-13T06:51:00.891418+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:51:00.891418+0000","last_undegraded":"2024-06-13T06:51:00.891418+0000","last_fullsized":"2024-06-13T06:51:00.891418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779907+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:51:40.779907+0000","last_peered":"2024-06-13T06:51:40.779907+0000","last_clean":"2024-06-13T06:51:40.779907+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:51:40.779907+0000","last_undegraded":"2024-06-13T06:51:40.779907+0000","last_fullsized":"2024-06-13T06:51:40.779907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881820+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:51:00.881820+0000","last_peered":"2024-06-13T06:51:00.881820+0000","last_clean":"2024-06-13T06:51:00.881820+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:51:00.881820+0000","last_undegraded":"2024-06-13T06:51:00.881820+0000","last_fullsized":"2024-06-13T06:51:00.881820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943311+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:51:00.943311+0000","last_peered":"2024-06-13T06:51:00.943311+0000","last_clean":"2024-06-13T06:51:00.943311+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:51:00.943311+0000","last_undegraded":"2024-06-13T06:51:00.943311+0000","last_fullsized":"2024-06-13T06:51:00.943311+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803102+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:51:41.803102+0000","last_peered":"2024-06-13T06:51:41.803102+0000","last_clean":"2024-06-13T06:51:41.803102+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:51:41.803102+0000","last_undegraded":"2024-06-13T06:51:41.803102+0000","last_fullsized":"2024-06-13T06:51:41.803102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882588+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:51:00.882588+0000","last_peered":"2024-06-13T06:51:00.882588+0000","last_clean":"2024-06-13T06:51:00.882588+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:51:00.882588+0000","last_undegraded":"2024-06-13T06:51:00.882588+0000","last_fullsized":"2024-06-13T06:51:00.882588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257334+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:51:01.257334+0000","last_peered":"2024-06-13T06:51:01.257334+0000","last_clean":"2024-06-13T06:51:01.257334+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:51:01.257334+0000","last_undegraded":"2024-06-13T06:51:01.257334+0000","last_fullsized":"2024-06-13T06:51:01.257334+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253593+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:51:01.253593+0000","last_peered":"2024-06-13T06:51:01.253593+0000","last_clean":"2024-06-13T06:51:01.253593+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:51:01.253593+0000","last_undegraded":"2024-06-13T06:51:01.253593+0000","last_fullsized":"2024-06-13T06:51:01.253593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882003+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:51:00.882003+0000","last_peered":"2024-06-13T06:51:00.882003+0000","last_clean":"2024-06-13T06:51:00.882003+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:51:00.882003+0000","last_undegraded":"2024-06-13T06:51:00.882003+0000","last_fullsized":"2024-06-13T06:51:00.882003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252411+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:51:01.252411+0000","last_peered":"2024-06-13T06:51:01.252411+0000","last_clean":"2024-06-13T06:51:01.252411+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:51:01.252411+0000","last_undegraded":"2024-06-13T06:51:01.252411+0000","last_fullsized":"2024-06-13T06:51:01.252411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960914+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:51:00.960914+0000","last_peered":"2024-06-13T06:51:00.960914+0000","last_clean":"2024-06-13T06:51:00.960914+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:51:00.960914+0000","last_undegraded":"2024-06-13T06:51:00.960914+0000","last_fullsized":"2024-06-13T06:51:00.960914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882351+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:51:00.882351+0000","last_peered":"2024-06-13T06:51:00.882351+0000","last_clean":"2024-06-13T06:51:00.882351+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:51:00.882351+0000","last_undegraded":"2024-06-13T06:51:00.882351+0000","last_fullsized":"2024-06-13T06:51:00.882351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883807+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:51:00.883807+0000","last_peered":"2024-06-13T06:51:00.883807+0000","last_clean":"2024-06-13T06:51:00.883807+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:51:00.883807+0000","last_undegraded":"2024-06-13T06:51:00.883807+0000","last_fullsized":"2024-06-13T06:51:00.883807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882010+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:51:00.882010+0000","last_peered":"2024-06-13T06:51:00.882010+0000","last_clean":"2024-06-13T06:51:00.882010+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:51:00.882010+0000","last_undegraded":"2024-06-13T06:51:00.882010+0000","last_fullsized":"2024-06-13T06:51:00.882010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881687+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:51:00.881687+0000","last_peered":"2024-06-13T06:51:00.881687+0000","last_clean":"2024-06-13T06:51:00.881687+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:51:00.881687+0000","last_undegraded":"2024-06-13T06:51:00.881687+0000","last_fullsized":"2024-06-13T06:51:00.881687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.886650+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:51:00.886650+0000","last_peered":"2024-06-13T06:51:00.886650+0000","last_clean":"2024-06-13T06:51:00.886650+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:51:00.886650+0000","last_undegraded":"2024-06-13T06:51:00.886650+0000","last_fullsized":"2024-06-13T06:51:00.886650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960365+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:51:00.960365+0000","last_peered":"2024-06-13T06:51:00.960365+0000","last_clean":"2024-06-13T06:51:00.960365+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:51:00.960365+0000","last_undegraded":"2024-06-13T06:51:00.960365+0000","last_fullsized":"2024-06-13T06:51:00.960365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257598+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:51:01.257598+0000","last_peered":"2024-06-13T06:51:01.257598+0000","last_clean":"2024-06-13T06:51:01.257598+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:51:01.257598+0000","last_undegraded":"2024-06-13T06:51:01.257598+0000","last_fullsized":"2024-06-13T06:51:01.257598+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882036+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:51:00.882036+0000","last_peered":"2024-06-13T06:51:00.882036+0000","last_clean":"2024-06-13T06:51:00.882036+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:51:00.882036+0000","last_undegraded":"2024-06-13T06:51:00.882036+0000","last_fullsized":"2024-06-13T06:51:00.882036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231261+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:51:01.231261+0000","last_peered":"2024-06-13T06:51:01.231261+0000","last_clean":"2024-06-13T06:51:01.231261+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:51:01.231261+0000","last_undegraded":"2024-06-13T06:51:01.231261+0000","last_fullsized":"2024-06-13T06:51:01.231261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252971+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:51:01.252971+0000","last_peered":"2024-06-13T06:51:01.252971+0000","last_clean":"2024-06-13T06:51:01.252971+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:51:01.252971+0000","last_undegraded":"2024-06-13T06:51:01.252971+0000","last_fullsized":"2024-06-13T06:51:01.252971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230433+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:51:01.230433+0000","last_peered":"2024-06-13T06:51:01.230433+0000","last_clean":"2024-06-13T06:51:01.230433+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:51:01.230433+0000","last_undegraded":"2024-06-13T06:51:01.230433+0000","last_fullsized":"2024-06-13T06:51:01.230433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943496+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:51:00.943496+0000","last_peered":"2024-06-13T06:51:00.943496+0000","last_clean":"2024-06-13T06:51:00.943496+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:51:00.943496+0000","last_undegraded":"2024-06-13T06:51:00.943496+0000","last_fullsized":"2024-06-13T06:51:00.943496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960812+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:51:00.960812+0000","last_peered":"2024-06-13T06:51:00.960812+0000","last_clean":"2024-06-13T06:51:00.960812+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:51:00.960812+0000","last_undegraded":"2024-06-13T06:51:00.960812+0000","last_fullsized":"2024-06-13T06:51:00.960812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884052+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:51:00.884052+0000","last_peered":"2024-06-13T06:51:00.884052+0000","last_clean":"2024-06-13T06:51:00.884052+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:51:00.884052+0000","last_undegraded":"2024-06-13T06:51:00.884052+0000","last_fullsized":"2024-06-13T06:51:00.884052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68058,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":236,"num_read_kb":192,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":1209759,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150011,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":613327,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32312,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712840,"statfs":{"total":95995035648,"available":95961948160,"internally_reserved":0,"allocated":1163264,"data_stored":610412,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554399,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:52:23.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:52:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:23 smithi007 ceph-mon[32077]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-13T06:52:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:23 smithi007 ceph-mon[32077]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:23 smithi189 ceph-mon[36616]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-13T06:52:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:23 smithi189 ceph-mon[36616]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:23 smithi175 ceph-mon[36886]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-06-13T06:52:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:23 smithi175 ceph-mon[36886]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:25.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:25 smithi007 ceph-mon[32077]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-13T06:52:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:25 smithi189 ceph-mon[36616]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-13T06:52:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:25 smithi175 ceph-mon[36886]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 -- 172.21.15.7:0/2963900891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e24107840 msgr2=0x7f6e2410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/2963900891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e24107840 0x7f6e2410f3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00b0d0 tx=0x7f6e1c00b3e0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 -- 172.21.15.7:0/2963900891 shutdown_connections 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/2963900891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e24107840 0x7f6e2410f3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/2963900891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24102740 0x7f6e24102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/2963900891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e241030f0 0x7f6e24107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 -- 172.21.15.7:0/2963900891 >> 172.21.15.7:0/2963900891 conn(0x7f6e240795f0 msgr2=0x7f6e24079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 -- 172.21.15.7:0/2963900891 shutdown_connections 2024-06-13T06:52:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.805+0000 7f6e2aa00700 1 -- 172.21.15.7:0/2963900891 wait complete. 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.807+0000 7f6e2aa00700 1 Processor -- start 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.807+0000 7f6e2aa00700 1 -- start start 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.807+0000 7f6e2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 0x7f6e2410e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.807+0000 7f6e2aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.807+0000 7f6e2aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24107840 0x7f6e241a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e2aa00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e241127b0 con 0x7f6e241030f0 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e2aa00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24112630 con 0x7f6e24102740 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e2aa00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24112930 con 0x7f6e24107840 2024-06-13T06:52:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60656/0 (socket says 172.21.15.7:60656) 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 -- 172.21.15.7:0/830093733 learned_addr learned my addr 172.21.15.7:0/830093733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 0x7f6e2410e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e28a00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24107840 0x7f6e241a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 -- 172.21.15.7:0/830093733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24107840 msgr2=0x7f6e241a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24107840 0x7f6e241a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 -- 172.21.15.7:0/830093733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 msgr2=0x7f6e2410e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 0x7f6e2410e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23400700 1 -- 172.21.15.7:0/830093733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241a4180 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.808+0000 7f6e23e00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 0x7f6e2410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e23400700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e9a0 tx=0x7f6e0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00ca10 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c00cbb0 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241a4470 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c018680 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.809+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e241abcf0 con 0x7f6e241030f0 2024-06-13T06:52:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.811+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e24069180 con 0x7f6e241030f0 2024-06-13T06:52:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.812+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6e0c010430 con 0x7f6e241030f0 2024-06-13T06:52:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.814+0000 7f6e20c00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 0x7f6e080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.814+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6e0c0cf7b0 con 0x7f6e241030f0 2024-06-13T06:52:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.814+0000 7f6e23e00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 0x7f6e080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.815+0000 7f6e23e00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 0x7f6e080a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009680 tx=0x7f6e14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:25.818+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e0c09c210 con 0x7f6e241030f0 2024-06-13T06:52:26.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.046+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e24100c00 con 0x7f6e080a2cb0 2024-06-13T06:52:26.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.065+0000 7f6e20c00700 1 -- 172.21.15.7:0/830093733 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340646 (secure 0 0 0) 0x7f6e24100c00 con 0x7f6e080a2cb0 2024-06-13T06:52:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 msgr2=0x7f6e080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 0x7f6e080a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009680 tx=0x7f6e14009260 comp rx=0 tx=0).stop 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 msgr2=0x7f6e2410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e9a0 tx=0x7f6e0c00ecb0 comp rx=0 tx=0).stop 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 shutdown_connections 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e24107840 0x7f6e241a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6e080a2cb0 0x7f6e080a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e24102740 0x7f6e2410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 --2- 172.21.15.7:0/830093733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e241030f0 0x7f6e2410eb20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.072+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 >> 172.21.15.7:0/830093733 conn(0x7f6e240795f0 msgr2=0x7f6e241004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.073+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 shutdown_connections 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:26.073+0000 7f6e2aa00700 1 -- 172.21.15.7:0/830093733 wait complete. 2024-06-13T06:52:26.079 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:52:26.924 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-06-13T06:52:25.661591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658660,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":387,"num_read_kb":311,"num_write":352,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378916,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562908,"statfs":{"total":1151940427776,"available":1151552417792,"internally_reserved":0,"allocated":13438976,"data_stored":8599274,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":50,"num_write":6,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884567+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:51:00.884567+0000","last_peered":"2024-06-13T06:51:00.884567+0000","last_clean":"2024-06-13T06:51:00.884567+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:51:00.884567+0000","last_undegraded":"2024-06-13T06:51:00.884567+0000","last_fullsized":"2024-06-13T06:51:00.884567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889174+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:51:00.889174+0000","last_peered":"2024-06-13T06:51:00.889174+0000","last_clean":"2024-06-13T06:51:00.889174+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:51:00.889174+0000","last_undegraded":"2024-06-13T06:51:00.889174+0000","last_fullsized":"2024-06-13T06:51:00.889174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.885443+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:51:00.885443+0000","last_peered":"2024-06-13T06:51:00.885443+0000","last_clean":"2024-06-13T06:51:00.885443+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:51:00.885443+0000","last_undegraded":"2024-06-13T06:51:00.885443+0000","last_fullsized":"2024-06-13T06:51:00.885443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257890+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:51:01.257890+0000","last_peered":"2024-06-13T06:51:01.257890+0000","last_clean":"2024-06-13T06:51:01.257890+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:51:01.257890+0000","last_undegraded":"2024-06-13T06:51:01.257890+0000","last_fullsized":"2024-06-13T06:51:01.257890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254250+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:51:01.254250+0000","last_peered":"2024-06-13T06:51:01.254250+0000","last_clean":"2024-06-13T06:51:01.254250+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:51:01.254250+0000","last_undegraded":"2024-06-13T06:51:01.254250+0000","last_fullsized":"2024-06-13T06:51:01.254250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253432+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:51:01.253432+0000","last_peered":"2024-06-13T06:51:01.253432+0000","last_clean":"2024-06-13T06:51:01.253432+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:51:01.253432+0000","last_undegraded":"2024-06-13T06:51:01.253432+0000","last_fullsized":"2024-06-13T06:51:01.253432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882061+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:51:00.882061+0000","last_peered":"2024-06-13T06:51:00.882061+0000","last_clean":"2024-06-13T06:51:00.882061+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:51:00.882061+0000","last_undegraded":"2024-06-13T06:51:00.882061+0000","last_fullsized":"2024-06-13T06:51:00.882061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230293+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:51:01.230293+0000","last_peered":"2024-06-13T06:51:01.230293+0000","last_clean":"2024-06-13T06:51:01.230293+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:51:01.230293+0000","last_undegraded":"2024-06-13T06:51:01.230293+0000","last_fullsized":"2024-06-13T06:51:01.230293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943973+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:51:00.943973+0000","last_peered":"2024-06-13T06:51:00.943973+0000","last_clean":"2024-06-13T06:51:00.943973+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:51:00.943973+0000","last_undegraded":"2024-06-13T06:51:00.943973+0000","last_fullsized":"2024-06-13T06:51:00.943973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254002+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:51:01.254002+0000","last_peered":"2024-06-13T06:51:01.254002+0000","last_clean":"2024-06-13T06:51:01.254002+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:51:01.254002+0000","last_undegraded":"2024-06-13T06:51:01.254002+0000","last_fullsized":"2024-06-13T06:51:01.254002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882378+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:51:00.882378+0000","last_peered":"2024-06-13T06:51:00.882378+0000","last_clean":"2024-06-13T06:51:00.882378+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:51:00.882378+0000","last_undegraded":"2024-06-13T06:51:00.882378+0000","last_fullsized":"2024-06-13T06:51:00.882378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257838+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:51:01.257838+0000","last_peered":"2024-06-13T06:51:01.257838+0000","last_clean":"2024-06-13T06:51:01.257838+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:51:01.257838+0000","last_undegraded":"2024-06-13T06:51:01.257838+0000","last_fullsized":"2024-06-13T06:51:01.257838+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960929+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:51:00.960929+0000","last_peered":"2024-06-13T06:51:00.960929+0000","last_clean":"2024-06-13T06:51:00.960929+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:51:00.960929+0000","last_undegraded":"2024-06-13T06:51:00.960929+0000","last_fullsized":"2024-06-13T06:51:00.960929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884779+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:51:00.884779+0000","last_peered":"2024-06-13T06:51:00.884779+0000","last_clean":"2024-06-13T06:51:00.884779+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:51:00.884779+0000","last_undegraded":"2024-06-13T06:51:00.884779+0000","last_fullsized":"2024-06-13T06:51:00.884779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883642+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:51:00.883642+0000","last_peered":"2024-06-13T06:51:00.883642+0000","last_clean":"2024-06-13T06:51:00.883642+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:51:00.883642+0000","last_undegraded":"2024-06-13T06:51:00.883642+0000","last_fullsized":"2024-06-13T06:51:00.883642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943121+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:51:00.943121+0000","last_peered":"2024-06-13T06:51:00.943121+0000","last_clean":"2024-06-13T06:51:00.943121+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:51:00.943121+0000","last_undegraded":"2024-06-13T06:51:00.943121+0000","last_fullsized":"2024-06-13T06:51:00.943121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884305+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:51:00.884305+0000","last_peered":"2024-06-13T06:51:00.884305+0000","last_clean":"2024-06-13T06:51:00.884305+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:51:00.884305+0000","last_undegraded":"2024-06-13T06:51:00.884305+0000","last_fullsized":"2024-06-13T06:51:00.884305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892374+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:51:00.892374+0000","last_peered":"2024-06-13T06:51:00.892374+0000","last_clean":"2024-06-13T06:51:00.892374+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:51:00.892374+0000","last_undegraded":"2024-06-13T06:51:00.892374+0000","last_fullsized":"2024-06-13T06:51:00.892374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882200+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:51:00.882200+0000","last_peered":"2024-06-13T06:51:00.882200+0000","last_clean":"2024-06-13T06:51:00.882200+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:51:00.882200+0000","last_undegraded":"2024-06-13T06:51:00.882200+0000","last_fullsized":"2024-06-13T06:51:00.882200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881672+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:51:00.881672+0000","last_peered":"2024-06-13T06:51:00.881672+0000","last_clean":"2024-06-13T06:51:00.881672+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:51:00.881672+0000","last_undegraded":"2024-06-13T06:51:00.881672+0000","last_fullsized":"2024-06-13T06:51:00.881672+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835279+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:51:45.835279+0000","last_peered":"2024-06-13T06:51:45.835279+0000","last_clean":"2024-06-13T06:51:45.835279+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:51:45.835279+0000","last_undegraded":"2024-06-13T06:51:45.835279+0000","last_fullsized":"2024-06-13T06:51:45.835279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231029+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:51:01.231029+0000","last_peered":"2024-06-13T06:51:01.231029+0000","last_clean":"2024-06-13T06:51:01.231029+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:51:01.231029+0000","last_undegraded":"2024-06-13T06:51:01.231029+0000","last_fullsized":"2024-06-13T06:51:01.231029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891147+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:51:00.891147+0000","last_peered":"2024-06-13T06:51:00.891147+0000","last_clean":"2024-06-13T06:51:00.891147+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:51:00.891147+0000","last_undegraded":"2024-06-13T06:51:00.891147+0000","last_fullsized":"2024-06-13T06:51:00.891147+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803500+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:51:41.803500+0000","last_peered":"2024-06-13T06:51:41.803500+0000","last_clean":"2024-06-13T06:51:41.803500+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:51:41.803500+0000","last_undegraded":"2024-06-13T06:51:41.803500+0000","last_fullsized":"2024-06-13T06:51:41.803500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891109+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:51:00.891109+0000","last_peered":"2024-06-13T06:51:00.891109+0000","last_clean":"2024-06-13T06:51:00.891109+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:51:00.891109+0000","last_undegraded":"2024-06-13T06:51:00.891109+0000","last_fullsized":"2024-06-13T06:51:00.891109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891585+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891585+0000","last_peered":"2024-06-13T06:51:00.891585+0000","last_clean":"2024-06-13T06:51:00.891585+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891585+0000","last_undegraded":"2024-06-13T06:51:00.891585+0000","last_fullsized":"2024-06-13T06:51:00.891585+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.797974+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:51:41.797974+0000","last_peered":"2024-06-13T06:51:41.797974+0000","last_clean":"2024-06-13T06:51:41.797974+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:51:41.797974+0000","last_undegraded":"2024-06-13T06:51:41.797974+0000","last_fullsized":"2024-06-13T06:51:41.797974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943555+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:51:00.943555+0000","last_peered":"2024-06-13T06:51:00.943555+0000","last_clean":"2024-06-13T06:51:00.943555+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:51:00.943555+0000","last_undegraded":"2024-06-13T06:51:00.943555+0000","last_fullsized":"2024-06-13T06:51:00.943555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891169+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891169+0000","last_peered":"2024-06-13T06:51:00.891169+0000","last_clean":"2024-06-13T06:51:00.891169+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891169+0000","last_undegraded":"2024-06-13T06:51:00.891169+0000","last_fullsized":"2024-06-13T06:51:00.891169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253960+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:51:01.253960+0000","last_peered":"2024-06-13T06:51:01.253960+0000","last_clean":"2024-06-13T06:51:01.253960+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:51:01.253960+0000","last_undegraded":"2024-06-13T06:51:01.253960+0000","last_fullsized":"2024-06-13T06:51:01.253960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253490+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:51:01.253490+0000","last_peered":"2024-06-13T06:51:01.253490+0000","last_clean":"2024-06-13T06:51:01.253490+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:51:01.253490+0000","last_undegraded":"2024-06-13T06:51:01.253490+0000","last_fullsized":"2024-06-13T06:51:01.253490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881893+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881893+0000","last_peered":"2024-06-13T06:51:00.881893+0000","last_clean":"2024-06-13T06:51:00.881893+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881893+0000","last_undegraded":"2024-06-13T06:51:00.881893+0000","last_fullsized":"2024-06-13T06:51:00.881893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":87,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:23.888161+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:52:23.888161+0000","last_peered":"2024-06-13T06:52:23.888161+0000","last_clean":"2024-06-13T06:52:23.888161+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:52:23.888161+0000","last_undegraded":"2024-06-13T06:52:23.888161+0000","last_fullsized":"2024-06-13T06:52:23.888161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254226+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:51:01.254226+0000","last_peered":"2024-06-13T06:51:01.254226+0000","last_clean":"2024-06-13T06:51:01.254226+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:51:01.254226+0000","last_undegraded":"2024-06-13T06:51:01.254226+0000","last_fullsized":"2024-06-13T06:51:01.254226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882054+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882054+0000","last_peered":"2024-06-13T06:51:00.882054+0000","last_clean":"2024-06-13T06:51:00.882054+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882054+0000","last_undegraded":"2024-06-13T06:51:00.882054+0000","last_fullsized":"2024-06-13T06:51:00.882054+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.800947+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:51:41.800947+0000","last_peered":"2024-06-13T06:51:41.800947+0000","last_clean":"2024-06-13T06:51:41.800947+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:51:41.800947+0000","last_undegraded":"2024-06-13T06:51:41.800947+0000","last_fullsized":"2024-06-13T06:51:41.800947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253297+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:51:01.253297+0000","last_peered":"2024-06-13T06:51:01.253297+0000","last_clean":"2024-06-13T06:51:01.253297+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:51:01.253297+0000","last_undegraded":"2024-06-13T06:51:01.253297+0000","last_fullsized":"2024-06-13T06:51:01.253297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884119+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:51:00.884119+0000","last_peered":"2024-06-13T06:51:00.884119+0000","last_clean":"2024-06-13T06:51:00.884119+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:51:00.884119+0000","last_undegraded":"2024-06-13T06:51:00.884119+0000","last_fullsized":"2024-06-13T06:51:00.884119+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.805631+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:51:41.805631+0000","last_peered":"2024-06-13T06:51:41.805631+0000","last_clean":"2024-06-13T06:51:41.805631+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:51:41.805631+0000","last_undegraded":"2024-06-13T06:51:41.805631+0000","last_fullsized":"2024-06-13T06:51:41.805631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230150+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:51:01.230150+0000","last_peered":"2024-06-13T06:51:01.230150+0000","last_clean":"2024-06-13T06:51:01.230150+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:51:01.230150+0000","last_undegraded":"2024-06-13T06:51:01.230150+0000","last_fullsized":"2024-06-13T06:51:01.230150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888726+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:51:00.888726+0000","last_peered":"2024-06-13T06:51:00.888726+0000","last_clean":"2024-06-13T06:51:00.888726+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:51:00.888726+0000","last_undegraded":"2024-06-13T06:51:00.888726+0000","last_fullsized":"2024-06-13T06:51:00.888726+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252905+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:51:01.252905+0000","last_peered":"2024-06-13T06:51:01.252905+0000","last_clean":"2024-06-13T06:51:01.252905+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:51:01.252905+0000","last_undegraded":"2024-06-13T06:51:01.252905+0000","last_fullsized":"2024-06-13T06:51:01.252905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.890749+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:51:00.890749+0000","last_peered":"2024-06-13T06:51:00.890749+0000","last_clean":"2024-06-13T06:51:00.890749+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:51:00.890749+0000","last_undegraded":"2024-06-13T06:51:00.890749+0000","last_fullsized":"2024-06-13T06:51:00.890749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230237+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:51:01.230237+0000","last_peered":"2024-06-13T06:51:01.230237+0000","last_clean":"2024-06-13T06:51:01.230237+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:51:01.230237+0000","last_undegraded":"2024-06-13T06:51:01.230237+0000","last_fullsized":"2024-06-13T06:51:01.230237+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":83,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:23.887573+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:52:23.887573+0000","last_peered":"2024-06-13T06:52:23.887573+0000","last_clean":"2024-06-13T06:52:23.887573+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:52:23.887573+0000","last_undegraded":"2024-06-13T06:52:23.887573+0000","last_fullsized":"2024-06-13T06:52:23.887573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943956+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:51:00.943956+0000","last_peered":"2024-06-13T06:51:00.943956+0000","last_clean":"2024-06-13T06:51:00.943956+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:51:00.943956+0000","last_undegraded":"2024-06-13T06:51:00.943956+0000","last_fullsized":"2024-06-13T06:51:00.943956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888123+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:51:00.888123+0000","last_peered":"2024-06-13T06:51:00.888123+0000","last_clean":"2024-06-13T06:51:00.888123+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:51:00.888123+0000","last_undegraded":"2024-06-13T06:51:00.888123+0000","last_fullsized":"2024-06-13T06:51:00.888123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779187+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:51:40.779187+0000","last_peered":"2024-06-13T06:51:40.779187+0000","last_clean":"2024-06-13T06:51:40.779187+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:51:40.779187+0000","last_undegraded":"2024-06-13T06:51:40.779187+0000","last_fullsized":"2024-06-13T06:51:40.779187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889997+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:51:00.889997+0000","last_peered":"2024-06-13T06:51:00.889997+0000","last_clean":"2024-06-13T06:51:00.889997+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:51:00.889997+0000","last_undegraded":"2024-06-13T06:51:00.889997+0000","last_fullsized":"2024-06-13T06:51:00.889997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891775+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:51:00.891775+0000","last_peered":"2024-06-13T06:51:00.891775+0000","last_clean":"2024-06-13T06:51:00.891775+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:51:00.891775+0000","last_undegraded":"2024-06-13T06:51:00.891775+0000","last_fullsized":"2024-06-13T06:51:00.891775+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781228+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:51:40.781228+0000","last_peered":"2024-06-13T06:51:40.781228+0000","last_clean":"2024-06-13T06:51:40.781228+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:51:40.781228+0000","last_undegraded":"2024-06-13T06:51:40.781228+0000","last_fullsized":"2024-06-13T06:51:40.781228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943664+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:51:00.943664+0000","last_peered":"2024-06-13T06:51:00.943664+0000","last_clean":"2024-06-13T06:51:00.943664+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:51:00.943664+0000","last_undegraded":"2024-06-13T06:51:00.943664+0000","last_fullsized":"2024-06-13T06:51:00.943664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891096+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:51:00.891096+0000","last_peered":"2024-06-13T06:51:00.891096+0000","last_clean":"2024-06-13T06:51:00.891096+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:51:00.891096+0000","last_undegraded":"2024-06-13T06:51:00.891096+0000","last_fullsized":"2024-06-13T06:51:00.891096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":163,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:24.896731+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:52:24.896731+0000","last_peered":"2024-06-13T06:52:24.896731+0000","last_clean":"2024-06-13T06:52:24.896731+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:52:24.896731+0000","last_undegraded":"2024-06-13T06:52:24.896731+0000","last_fullsized":"2024-06-13T06:52:24.896731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64058,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":107,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254365+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:51:01.254365+0000","last_peered":"2024-06-13T06:51:01.254365+0000","last_clean":"2024-06-13T06:51:01.254365+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:51:01.254365+0000","last_undegraded":"2024-06-13T06:51:01.254365+0000","last_fullsized":"2024-06-13T06:51:01.254365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253801+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:51:01.253801+0000","last_peered":"2024-06-13T06:51:01.253801+0000","last_clean":"2024-06-13T06:51:01.253801+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:51:01.253801+0000","last_undegraded":"2024-06-13T06:51:01.253801+0000","last_fullsized":"2024-06-13T06:51:01.253801+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803271+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:51:41.803271+0000","last_peered":"2024-06-13T06:51:41.803271+0000","last_clean":"2024-06-13T06:51:41.803271+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:51:41.803271+0000","last_undegraded":"2024-06-13T06:51:41.803271+0000","last_fullsized":"2024-06-13T06:51:41.803271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960254+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:51:00.960254+0000","last_peered":"2024-06-13T06:51:00.960254+0000","last_clean":"2024-06-13T06:51:00.960254+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:51:00.960254+0000","last_undegraded":"2024-06-13T06:51:00.960254+0000","last_fullsized":"2024-06-13T06:51:00.960254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883856+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:51:00.883856+0000","last_peered":"2024-06-13T06:51:00.883856+0000","last_clean":"2024-06-13T06:51:00.883856+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:51:00.883856+0000","last_undegraded":"2024-06-13T06:51:00.883856+0000","last_fullsized":"2024-06-13T06:51:00.883856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780954+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:51:40.780954+0000","last_peered":"2024-06-13T06:51:40.780954+0000","last_clean":"2024-06-13T06:51:40.780954+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:51:40.780954+0000","last_undegraded":"2024-06-13T06:51:40.780954+0000","last_fullsized":"2024-06-13T06:51:40.780954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882401+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:51:00.882401+0000","last_peered":"2024-06-13T06:51:00.882401+0000","last_clean":"2024-06-13T06:51:00.882401+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:51:00.882401+0000","last_undegraded":"2024-06-13T06:51:00.882401+0000","last_fullsized":"2024-06-13T06:51:00.882401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892656+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:51:00.892656+0000","last_peered":"2024-06-13T06:51:00.892656+0000","last_clean":"2024-06-13T06:51:00.892656+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:51:00.892656+0000","last_undegraded":"2024-06-13T06:51:00.892656+0000","last_fullsized":"2024-06-13T06:51:00.892656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":84,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:20.906934+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:52:20.906934+0000","last_peered":"2024-06-13T06:52:20.906934+0000","last_clean":"2024-06-13T06:52:20.906934+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:52:20.906934+0000","last_undegraded":"2024-06-13T06:52:20.906934+0000","last_fullsized":"2024-06-13T06:52:20.906934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881815+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:51:00.881815+0000","last_peered":"2024-06-13T06:51:00.881815+0000","last_clean":"2024-06-13T06:51:00.881815+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:51:00.881815+0000","last_undegraded":"2024-06-13T06:51:00.881815+0000","last_fullsized":"2024-06-13T06:51:00.881815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960960+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:51:00.960960+0000","last_peered":"2024-06-13T06:51:00.960960+0000","last_clean":"2024-06-13T06:51:00.960960+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:51:00.960960+0000","last_undegraded":"2024-06-13T06:51:00.960960+0000","last_fullsized":"2024-06-13T06:51:00.960960+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803134+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:51:41.803134+0000","last_peered":"2024-06-13T06:51:41.803134+0000","last_clean":"2024-06-13T06:51:41.803134+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:51:41.803134+0000","last_undegraded":"2024-06-13T06:51:41.803134+0000","last_fullsized":"2024-06-13T06:51:41.803134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253554+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:51:01.253554+0000","last_peered":"2024-06-13T06:51:01.253554+0000","last_clean":"2024-06-13T06:51:01.253554+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:51:01.253554+0000","last_undegraded":"2024-06-13T06:51:01.253554+0000","last_fullsized":"2024-06-13T06:51:01.253554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253431+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:51:01.253431+0000","last_peered":"2024-06-13T06:51:01.253431+0000","last_clean":"2024-06-13T06:51:01.253431+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:51:01.253431+0000","last_undegraded":"2024-06-13T06:51:01.253431+0000","last_fullsized":"2024-06-13T06:51:01.253431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:42.797466+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:51:42.797466+0000","last_peered":"2024-06-13T06:51:42.797466+0000","last_clean":"2024-06-13T06:51:42.797466+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:51:42.797466+0000","last_undegraded":"2024-06-13T06:51:42.797466+0000","last_fullsized":"2024-06-13T06:51:42.797466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884048+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:51:00.884048+0000","last_peered":"2024-06-13T06:51:00.884048+0000","last_clean":"2024-06-13T06:51:00.884048+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:51:00.884048+0000","last_undegraded":"2024-06-13T06:51:00.884048+0000","last_fullsized":"2024-06-13T06:51:00.884048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254044+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:51:01.254044+0000","last_peered":"2024-06-13T06:51:01.254044+0000","last_clean":"2024-06-13T06:51:01.254044+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:51:01.254044+0000","last_undegraded":"2024-06-13T06:51:01.254044+0000","last_fullsized":"2024-06-13T06:51:01.254044+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.896197+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:51:00.896197+0000","last_peered":"2024-06-13T06:51:00.896197+0000","last_clean":"2024-06-13T06:51:00.896197+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:51:00.896197+0000","last_undegraded":"2024-06-13T06:51:00.896197+0000","last_fullsized":"2024-06-13T06:51:00.896197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253797+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:51:01.253797+0000","last_peered":"2024-06-13T06:51:01.253797+0000","last_clean":"2024-06-13T06:51:01.253797+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:51:01.253797+0000","last_undegraded":"2024-06-13T06:51:01.253797+0000","last_fullsized":"2024-06-13T06:51:01.253797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960207+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:51:00.960207+0000","last_peered":"2024-06-13T06:51:00.960207+0000","last_clean":"2024-06-13T06:51:00.960207+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:51:00.960207+0000","last_undegraded":"2024-06-13T06:51:00.960207+0000","last_fullsized":"2024-06-13T06:51:00.960207+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799920+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:51:41.799920+0000","last_peered":"2024-06-13T06:51:41.799920+0000","last_clean":"2024-06-13T06:51:41.799920+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:51:41.799920+0000","last_undegraded":"2024-06-13T06:51:41.799920+0000","last_fullsized":"2024-06-13T06:51:41.799920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253734+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:51:01.253734+0000","last_peered":"2024-06-13T06:51:01.253734+0000","last_clean":"2024-06-13T06:51:01.253734+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:51:01.253734+0000","last_undegraded":"2024-06-13T06:51:01.253734+0000","last_fullsized":"2024-06-13T06:51:01.253734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961326+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:51:00.961326+0000","last_peered":"2024-06-13T06:51:00.961326+0000","last_clean":"2024-06-13T06:51:00.961326+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:51:00.961326+0000","last_undegraded":"2024-06-13T06:51:00.961326+0000","last_fullsized":"2024-06-13T06:51:00.961326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253297+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:51:01.253297+0000","last_peered":"2024-06-13T06:51:01.253297+0000","last_clean":"2024-06-13T06:51:01.253297+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:51:01.253297+0000","last_undegraded":"2024-06-13T06:51:01.253297+0000","last_fullsized":"2024-06-13T06:51:01.253297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":81,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:19.904051+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:52:19.904051+0000","last_peered":"2024-06-13T06:52:19.904051+0000","last_clean":"2024-06-13T06:52:19.904051+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:52:19.904051+0000","last_undegraded":"2024-06-13T06:52:19.904051+0000","last_fullsized":"2024-06-13T06:52:19.904051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256415+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:51:01.256415+0000","last_peered":"2024-06-13T06:51:01.256415+0000","last_clean":"2024-06-13T06:51:01.256415+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:51:01.256415+0000","last_undegraded":"2024-06-13T06:51:01.256415+0000","last_fullsized":"2024-06-13T06:51:01.256415+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230552+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:51:01.230552+0000","last_peered":"2024-06-13T06:51:01.230552+0000","last_clean":"2024-06-13T06:51:01.230552+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:51:01.230552+0000","last_undegraded":"2024-06-13T06:51:01.230552+0000","last_fullsized":"2024-06-13T06:51:01.230552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831145+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:51:45.831145+0000","last_peered":"2024-06-13T06:51:45.831145+0000","last_clean":"2024-06-13T06:51:45.831145+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:51:45.831145+0000","last_undegraded":"2024-06-13T06:51:45.831145+0000","last_fullsized":"2024-06-13T06:51:45.831145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881706+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:51:00.881706+0000","last_peered":"2024-06-13T06:51:00.881706+0000","last_clean":"2024-06-13T06:51:00.881706+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:51:00.881706+0000","last_undegraded":"2024-06-13T06:51:00.881706+0000","last_fullsized":"2024-06-13T06:51:00.881706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254422+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:51:01.254422+0000","last_peered":"2024-06-13T06:51:01.254422+0000","last_clean":"2024-06-13T06:51:01.254422+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:51:01.254422+0000","last_undegraded":"2024-06-13T06:51:01.254422+0000","last_fullsized":"2024-06-13T06:51:01.254422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.959715+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:51:00.959715+0000","last_peered":"2024-06-13T06:51:00.959715+0000","last_clean":"2024-06-13T06:51:00.959715+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:51:00.959715+0000","last_undegraded":"2024-06-13T06:51:00.959715+0000","last_fullsized":"2024-06-13T06:51:00.959715+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.798470+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:51:41.798470+0000","last_peered":"2024-06-13T06:51:41.798470+0000","last_clean":"2024-06-13T06:51:41.798470+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:51:41.798470+0000","last_undegraded":"2024-06-13T06:51:41.798470+0000","last_fullsized":"2024-06-13T06:51:41.798470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257244+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:51:01.257244+0000","last_peered":"2024-06-13T06:51:01.257244+0000","last_clean":"2024-06-13T06:51:01.257244+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:51:01.257244+0000","last_undegraded":"2024-06-13T06:51:01.257244+0000","last_fullsized":"2024-06-13T06:51:01.257244+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831109+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:51:45.831109+0000","last_peered":"2024-06-13T06:51:45.831109+0000","last_clean":"2024-06-13T06:51:45.831109+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:51:45.831109+0000","last_undegraded":"2024-06-13T06:51:45.831109+0000","last_fullsized":"2024-06-13T06:51:45.831109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960300+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:51:00.960300+0000","last_peered":"2024-06-13T06:51:00.960300+0000","last_clean":"2024-06-13T06:51:00.960300+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:51:00.960300+0000","last_undegraded":"2024-06-13T06:51:00.960300+0000","last_fullsized":"2024-06-13T06:51:00.960300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230124+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:51:01.230124+0000","last_peered":"2024-06-13T06:51:01.230124+0000","last_clean":"2024-06-13T06:51:01.230124+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:51:01.230124+0000","last_undegraded":"2024-06-13T06:51:01.230124+0000","last_fullsized":"2024-06-13T06:51:01.230124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'11","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:16.525210+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:52:16.525210+0000","last_peered":"2024-06-13T06:52:16.525210+0000","last_clean":"2024-06-13T06:52:16.525210+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:52:16.525210+0000","last_undegraded":"2024-06-13T06:52:16.525210+0000","last_fullsized":"2024-06-13T06:52:16.525210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231171+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:51:01.231171+0000","last_peered":"2024-06-13T06:51:01.231171+0000","last_clean":"2024-06-13T06:51:01.231171+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:51:01.231171+0000","last_undegraded":"2024-06-13T06:51:01.231171+0000","last_fullsized":"2024-06-13T06:51:01.231171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881533+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:51:00.881533+0000","last_peered":"2024-06-13T06:51:00.881533+0000","last_clean":"2024-06-13T06:51:00.881533+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:51:00.881533+0000","last_undegraded":"2024-06-13T06:51:00.881533+0000","last_fullsized":"2024-06-13T06:51:00.881533+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781280+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:51:40.781280+0000","last_peered":"2024-06-13T06:51:40.781280+0000","last_clean":"2024-06-13T06:51:40.781280+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:51:40.781280+0000","last_undegraded":"2024-06-13T06:51:40.781280+0000","last_fullsized":"2024-06-13T06:51:40.781280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.889312+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:51:00.889312+0000","last_peered":"2024-06-13T06:51:00.889312+0000","last_clean":"2024-06-13T06:51:00.889312+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:51:00.889312+0000","last_undegraded":"2024-06-13T06:51:00.889312+0000","last_fullsized":"2024-06-13T06:51:00.889312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256069+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:51:01.256069+0000","last_peered":"2024-06-13T06:51:01.256069+0000","last_clean":"2024-06-13T06:51:01.256069+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:51:01.256069+0000","last_undegraded":"2024-06-13T06:51:01.256069+0000","last_fullsized":"2024-06-13T06:51:01.256069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780226+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:51:40.780226+0000","last_peered":"2024-06-13T06:51:40.780226+0000","last_clean":"2024-06-13T06:51:40.780226+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:51:40.780226+0000","last_undegraded":"2024-06-13T06:51:40.780226+0000","last_fullsized":"2024-06-13T06:51:40.780226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254263+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:51:01.254263+0000","last_peered":"2024-06-13T06:51:01.254263+0000","last_clean":"2024-06-13T06:51:01.254263+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:51:01.254263+0000","last_undegraded":"2024-06-13T06:51:01.254263+0000","last_fullsized":"2024-06-13T06:51:01.254263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892441+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:51:00.892441+0000","last_peered":"2024-06-13T06:51:00.892441+0000","last_clean":"2024-06-13T06:51:00.892441+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:51:00.892441+0000","last_undegraded":"2024-06-13T06:51:00.892441+0000","last_fullsized":"2024-06-13T06:51:00.892441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781263+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:51:40.781263+0000","last_peered":"2024-06-13T06:51:40.781263+0000","last_clean":"2024-06-13T06:51:40.781263+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:51:40.781263+0000","last_undegraded":"2024-06-13T06:51:40.781263+0000","last_fullsized":"2024-06-13T06:51:40.781263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883660+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:51:00.883660+0000","last_peered":"2024-06-13T06:51:00.883660+0000","last_clean":"2024-06-13T06:51:00.883660+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:51:00.883660+0000","last_undegraded":"2024-06-13T06:51:00.883660+0000","last_fullsized":"2024-06-13T06:51:00.883660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891361+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:51:00.891361+0000","last_peered":"2024-06-13T06:51:00.891361+0000","last_clean":"2024-06-13T06:51:00.891361+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:51:00.891361+0000","last_undegraded":"2024-06-13T06:51:00.891361+0000","last_fullsized":"2024-06-13T06:51:00.891361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257403+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:51:01.257403+0000","last_peered":"2024-06-13T06:51:01.257403+0000","last_clean":"2024-06-13T06:51:01.257403+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:51:01.257403+0000","last_undegraded":"2024-06-13T06:51:01.257403+0000","last_fullsized":"2024-06-13T06:51:01.257403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230515+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:51:01.230515+0000","last_peered":"2024-06-13T06:51:01.230515+0000","last_clean":"2024-06-13T06:51:01.230515+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:51:01.230515+0000","last_undegraded":"2024-06-13T06:51:01.230515+0000","last_fullsized":"2024-06-13T06:51:01.230515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891418+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:51:00.891418+0000","last_peered":"2024-06-13T06:51:00.891418+0000","last_clean":"2024-06-13T06:51:00.891418+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:51:00.891418+0000","last_undegraded":"2024-06-13T06:51:00.891418+0000","last_fullsized":"2024-06-13T06:51:00.891418+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779907+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:51:40.779907+0000","last_peered":"2024-06-13T06:51:40.779907+0000","last_clean":"2024-06-13T06:51:40.779907+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:51:40.779907+0000","last_undegraded":"2024-06-13T06:51:40.779907+0000","last_fullsized":"2024-06-13T06:51:40.779907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881820+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:51:00.881820+0000","last_peered":"2024-06-13T06:51:00.881820+0000","last_clean":"2024-06-13T06:51:00.881820+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:51:00.881820+0000","last_undegraded":"2024-06-13T06:51:00.881820+0000","last_fullsized":"2024-06-13T06:51:00.881820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943311+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:51:00.943311+0000","last_peered":"2024-06-13T06:51:00.943311+0000","last_clean":"2024-06-13T06:51:00.943311+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:51:00.943311+0000","last_undegraded":"2024-06-13T06:51:00.943311+0000","last_fullsized":"2024-06-13T06:51:00.943311+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803102+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:51:41.803102+0000","last_peered":"2024-06-13T06:51:41.803102+0000","last_clean":"2024-06-13T06:51:41.803102+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:51:41.803102+0000","last_undegraded":"2024-06-13T06:51:41.803102+0000","last_fullsized":"2024-06-13T06:51:41.803102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882588+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:51:00.882588+0000","last_peered":"2024-06-13T06:51:00.882588+0000","last_clean":"2024-06-13T06:51:00.882588+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:51:00.882588+0000","last_undegraded":"2024-06-13T06:51:00.882588+0000","last_fullsized":"2024-06-13T06:51:00.882588+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257334+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:51:01.257334+0000","last_peered":"2024-06-13T06:51:01.257334+0000","last_clean":"2024-06-13T06:51:01.257334+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:51:01.257334+0000","last_undegraded":"2024-06-13T06:51:01.257334+0000","last_fullsized":"2024-06-13T06:51:01.257334+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253593+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:51:01.253593+0000","last_peered":"2024-06-13T06:51:01.253593+0000","last_clean":"2024-06-13T06:51:01.253593+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:51:01.253593+0000","last_undegraded":"2024-06-13T06:51:01.253593+0000","last_fullsized":"2024-06-13T06:51:01.253593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882003+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:51:00.882003+0000","last_peered":"2024-06-13T06:51:00.882003+0000","last_clean":"2024-06-13T06:51:00.882003+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:51:00.882003+0000","last_undegraded":"2024-06-13T06:51:00.882003+0000","last_fullsized":"2024-06-13T06:51:00.882003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252411+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:51:01.252411+0000","last_peered":"2024-06-13T06:51:01.252411+0000","last_clean":"2024-06-13T06:51:01.252411+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:51:01.252411+0000","last_undegraded":"2024-06-13T06:51:01.252411+0000","last_fullsized":"2024-06-13T06:51:01.252411+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960914+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:51:00.960914+0000","last_peered":"2024-06-13T06:51:00.960914+0000","last_clean":"2024-06-13T06:51:00.960914+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:51:00.960914+0000","last_undegraded":"2024-06-13T06:51:00.960914+0000","last_fullsized":"2024-06-13T06:51:00.960914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882351+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:51:00.882351+0000","last_peered":"2024-06-13T06:51:00.882351+0000","last_clean":"2024-06-13T06:51:00.882351+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:51:00.882351+0000","last_undegraded":"2024-06-13T06:51:00.882351+0000","last_fullsized":"2024-06-13T06:51:00.882351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883807+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:51:00.883807+0000","last_peered":"2024-06-13T06:51:00.883807+0000","last_clean":"2024-06-13T06:51:00.883807+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:51:00.883807+0000","last_undegraded":"2024-06-13T06:51:00.883807+0000","last_fullsized":"2024-06-13T06:51:00.883807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882106+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:51:00.882106+0000","last_peered":"2024-06-13T06:51:00.882106+0000","last_clean":"2024-06-13T06:51:00.882106+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:51:00.882106+0000","last_undegraded":"2024-06-13T06:51:00.882106+0000","last_fullsized":"2024-06-13T06:51:00.882106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881738+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:51:00.881738+0000","last_peered":"2024-06-13T06:51:00.881738+0000","last_clean":"2024-06-13T06:51:00.881738+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:51:00.881738+0000","last_undegraded":"2024-06-13T06:51:00.881738+0000","last_fullsized":"2024-06-13T06:51:00.881738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.886708+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:51:00.886708+0000","last_peered":"2024-06-13T06:51:00.886708+0000","last_clean":"2024-06-13T06:51:00.886708+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:51:00.886708+0000","last_undegraded":"2024-06-13T06:51:00.886708+0000","last_fullsized":"2024-06-13T06:51:00.886708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960365+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:51:00.960365+0000","last_peered":"2024-06-13T06:51:00.960365+0000","last_clean":"2024-06-13T06:51:00.960365+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:51:00.960365+0000","last_undegraded":"2024-06-13T06:51:00.960365+0000","last_fullsized":"2024-06-13T06:51:00.960365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.257598+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:51:01.257598+0000","last_peered":"2024-06-13T06:51:01.257598+0000","last_clean":"2024-06-13T06:51:01.257598+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:51:01.257598+0000","last_undegraded":"2024-06-13T06:51:01.257598+0000","last_fullsized":"2024-06-13T06:51:01.257598+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882036+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:51:00.882036+0000","last_peered":"2024-06-13T06:51:00.882036+0000","last_clean":"2024-06-13T06:51:00.882036+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:51:00.882036+0000","last_undegraded":"2024-06-13T06:51:00.882036+0000","last_fullsized":"2024-06-13T06:51:00.882036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.231261+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:51:01.231261+0000","last_peered":"2024-06-13T06:51:01.231261+0000","last_clean":"2024-06-13T06:51:01.231261+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:51:01.231261+0000","last_undegraded":"2024-06-13T06:51:01.231261+0000","last_fullsized":"2024-06-13T06:51:01.231261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252971+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:51:01.252971+0000","last_peered":"2024-06-13T06:51:01.252971+0000","last_clean":"2024-06-13T06:51:01.252971+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:51:01.252971+0000","last_undegraded":"2024-06-13T06:51:01.252971+0000","last_fullsized":"2024-06-13T06:51:01.252971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230433+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:51:01.230433+0000","last_peered":"2024-06-13T06:51:01.230433+0000","last_clean":"2024-06-13T06:51:01.230433+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:51:01.230433+0000","last_undegraded":"2024-06-13T06:51:01.230433+0000","last_fullsized":"2024-06-13T06:51:01.230433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943496+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:51:00.943496+0000","last_peered":"2024-06-13T06:51:00.943496+0000","last_clean":"2024-06-13T06:51:00.943496+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:51:00.943496+0000","last_undegraded":"2024-06-13T06:51:00.943496+0000","last_fullsized":"2024-06-13T06:51:00.943496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960812+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:51:00.960812+0000","last_peered":"2024-06-13T06:51:00.960812+0000","last_clean":"2024-06-13T06:51:00.960812+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:51:00.960812+0000","last_undegraded":"2024-06-13T06:51:00.960812+0000","last_fullsized":"2024-06-13T06:51:00.960812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884052+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:51:00.884052+0000","last_peered":"2024-06-13T06:51:00.884052+0000","last_clean":"2024-06-13T06:51:00.884052+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:51:00.884052+0000","last_undegraded":"2024-06-13T06:51:00.884052+0000","last_fullsized":"2024-06-13T06:51:00.884052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68058,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":261,"num_read_kb":202,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":1209759,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037820,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150011,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":613327,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":614677,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554399,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:52:26.930 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-13T06:52:26.930 INFO:tasks.cephadm:Setting up client nodes... 2024-06-13T06:52:26.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-13T06:52:27.422 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:27.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:27 smithi007 ceph-mon[32077]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-13T06:52:27.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:27 smithi007 ceph-mon[32077]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:27 smithi189 ceph-mon[36616]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-13T06:52:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:27 smithi189 ceph-mon[36616]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:27 smithi175 ceph-mon[36886]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-13T06:52:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:27 smithi175 ceph-mon[36886]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 -- 172.21.15.7:0/2971867534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 msgr2=0x7fec64078510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 --2- 172.21.15.7:0/2971867534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec64078510 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec50009990 tx=0x7fec50009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 -- 172.21.15.7:0/2971867534 shutdown_connections 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 --2- 172.21.15.7:0/2971867534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64078a50 0x7fec640792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 --2- 172.21.15.7:0/2971867534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64076e40 0x7fec64077260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 --2- 172.21.15.7:0/2971867534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec64078510 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 -- 172.21.15.7:0/2971867534 >> 172.21.15.7:0/2971867534 conn(0x7fec640ffce0 msgr2=0x7fec64102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 -- 172.21.15.7:0/2971867534 shutdown_connections 2024-06-13T06:52:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.571+0000 7fec6c000700 1 -- 172.21.15.7:0/2971867534 wait complete. 2024-06-13T06:52:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:29 smithi007 ceph-mon[32077]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 Processor -- start 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 -- start start 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64076e40 0x7fec6419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec6419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114780 con 0x7fec64078a50 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114600 con 0x7fec64078090 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.573+0000 7fec6c000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec64114900 con 0x7fec64076e40 2024-06-13T06:52:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60674/0 (socket says 172.21.15.7:60674) 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 -- 172.21.15.7:0/2657232533 learned_addr learned my addr 172.21.15.7:0/2657232533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec68c00700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec6419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec69600700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64076e40 0x7fec6419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 -- 172.21.15.7:0/2657232533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64076e40 msgr2=0x7fec6419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64076e40 0x7fec6419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 -- 172.21.15.7:0/2657232533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 msgr2=0x7fec6419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec6419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.574+0000 7fec6a000700 1 -- 172.21.15.7:0/2657232533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641a6900 con 0x7fec64078a50 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec6a000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ef30 tx=0x7fec5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0092e0 con 0x7fec64078a50 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec641a6b90 con 0x7fec64078a50 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c004bb0 con 0x7fec64078a50 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a70f0 con 0x7fec64078a50 2024-06-13T06:52:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.575+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004220 con 0x7fec64078a50 2024-06-13T06:52:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.576+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec64050a00 con 0x7fec64078a50 2024-06-13T06:52:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.580+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fec5c01b030 con 0x7fec64078a50 2024-06-13T06:52:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.582+0000 7fec62000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 0x7fec4c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.582+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fec5c0cf950 con 0x7fec64078a50 2024-06-13T06:52:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.582+0000 7fec69600700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 0x7fec4c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.583+0000 7fec69600700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 0x7fec4c0a5220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6419fb60 tx=0x7fec58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.584+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec5c022020 con 0x7fec64078a50 2024-06-13T06:52:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:29 smithi189 ceph-mon[36616]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:52:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:29 smithi175 ceph-mon[36886]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:52:29.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.907+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fec640630a0 con 0x7fec64078a50 2024-06-13T06:52:29.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.912+0000 7fec62000700 1 -- 172.21.15.7:0/2657232533 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fec5c010040 con 0x7fec64078a50 2024-06-13T06:52:29.913 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-06-13T06:52:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAtl2pmSMVENhAAAAWiX7rVMImBXsDpMIP5FQ== 2024-06-13T06:52:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.916+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 msgr2=0x7fec4c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.916+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 0x7fec4c0a5220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6419fb60 tx=0x7fec58009260 comp rx=0 tx=0).stop 2024-06-13T06:52:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.916+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 msgr2=0x7fec641a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:29.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.916+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00ef30 tx=0x7fec5c00c3a0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 shutdown_connections 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec64078a50 0x7fec641a6230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec64076e40 0x7fec6419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec4c0a2d60 0x7fec4c0a5220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 --2- 172.21.15.7:0/2657232533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec64078090 0x7fec6419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:29.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 >> 172.21.15.7:0/2657232533 conn(0x7fec640ffce0 msgr2=0x7fec641017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:29.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 shutdown_connections 2024-06-13T06:52:29.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:29.917+0000 7fec6c000700 1 -- 172.21.15.7:0/2657232533 wait complete. 2024-06-13T06:52:30.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:52:30.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:52:30.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:52:30.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:52:30.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-13T06:52:30.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2657232533' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-13T06:52:30.743 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:52:30.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-13T06:52:30.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-13T06:52:30.890 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-13T06:52:30.890 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-13T06:52:30.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph mgr dump --format=json 2024-06-13T06:52:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:31.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:31 smithi007 ceph-mon[32077]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:52:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:31 smithi189 ceph-mon[36616]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:52:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:31 smithi175 ceph-mon[36886]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:52:33.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:33 smithi007 ceph-mon[32077]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:52:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:33 smithi189 ceph-mon[36616]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:52:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:33 smithi175 ceph-mon[36886]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 -- 172.21.15.7:0/1867629873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef0104f80 msgr2=0x7f3ef0111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/1867629873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef0104f80 0x7f3ef0111490 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800b0a0 tx=0x7f3ee800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 -- 172.21.15.7:0/1867629873 shutdown_connections 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/1867629873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef0104f80 0x7f3ef0111490 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/1867629873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef010a650 0x7f3ef010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/1867629873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef01045c0 0x7f3ef0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 -- 172.21.15.7:0/1867629873 >> 172.21.15.7:0/1867629873 conn(0x7f3ef01000b0 msgr2=0x7f3ef01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 -- 172.21.15.7:0/1867629873 shutdown_connections 2024-06-13T06:52:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.759+0000 7f3ef5e00700 1 -- 172.21.15.7:0/1867629873 wait complete. 2024-06-13T06:52:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.761+0000 7f3ef5e00700 1 Processor -- start 2024-06-13T06:52:33.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.761+0000 7f3ef5e00700 1 -- start start 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 0x7f3ef01a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef010a650 0x7f3ef019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0114940 con 0x7f3ef0104f80 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef01147c0 con 0x7f3ef01045c0 2024-06-13T06:52:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0114ac0 con 0x7f3ef010a650 2024-06-13T06:52:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3eefe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 0x7f3ef01a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3eefe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58348/0 (socket says 172.21.15.7:58348) 2024-06-13T06:52:33.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3eefe00700 1 -- 172.21.15.7:0/2997032600 learned_addr learned my addr 172.21.15.7:0/2997032600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef5400700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef010a650 0x7f3ef019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 -- 172.21.15.7:0/2997032600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef010a650 msgr2=0x7f3ef019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef010a650 0x7f3ef019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 -- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 msgr2=0x7f3ef01a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3ef4a00700 1 -- 172.21.15.7:0/2997032600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef019b1e0 con 0x7f3ef01045c0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.762+0000 7f3eefe00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3ef4a00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 0x7f3ef01a08c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400e9c0 tx=0x7f3ee400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004220 con 0x7f3ef01045c0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee40043c0 con 0x7f3ef01045c0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003d10 con 0x7f3ef01045c0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef019b4d0 con 0x7f3ef01045c0 2024-06-13T06:52:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.763+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a7790 con 0x7f3ef01045c0 2024-06-13T06:52:33.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.764+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef0069180 con 0x7f3ef01045c0 2024-06-13T06:52:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.766+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3ee400c7c0 con 0x7f3ef01045c0 2024-06-13T06:52:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.768+0000 7f3eed600700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 0x7f3ed80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.768+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3ee40cf740 con 0x7f3ef01045c0 2024-06-13T06:52:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.768+0000 7f3eefe00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 0x7f3ed80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.769+0000 7f3eefe00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 0x7f3ed80a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0109be0 tx=0x7f3edc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:33.772+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ee409c1a0 con 0x7f3ef01045c0 2024-06-13T06:52:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.063+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3ef019bfb0 con 0x7f3ef01045c0 2024-06-13T06:52:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.069+0000 7f3eed600700 1 -- 172.21.15.7:0/2997032600 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263192 (secure 0 0 0) 0x7f3ee409baa0 con 0x7f3ef01045c0 2024-06-13T06:52:34.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:34.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 msgr2=0x7f3ed80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:34.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 0x7f3ed80a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0109be0 tx=0x7f3edc009260 comp rx=0 tx=0).stop 2024-06-13T06:52:34.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 msgr2=0x7f3ef01a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:34.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 0x7f3ef01a08c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400e9c0 tx=0x7f3ee400ecd0 comp rx=0 tx=0).stop 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 shutdown_connections 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef010a650 0x7f3ef019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ed80a2d30 0x7f3ed80a51f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ef01045c0 0x7f3ef01a08c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 --2- 172.21.15.7:0/2997032600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ef0104f80 0x7f3ef01a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 >> 172.21.15.7:0/2997032600 conn(0x7f3ef01000b0 msgr2=0x7f3ef0101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 shutdown_connections 2024-06-13T06:52:34.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:34.077+0000 7f3ef5e00700 1 -- 172.21.15.7:0/2997032600 wait complete. 2024-06-13T06:52:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:34 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2997032600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:52:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:34 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2997032600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:52:34.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:34 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2997032600' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-13T06:52:35.013 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2901233239},{"type":"v1","addr":"172.21.15.7:6801","nonce":2901233239}]},"active_addr":"172.21.15.7:6801/2901233239","active_change":"2024-06-13T06:41:59.456160+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1594541014}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4126457919}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1042620941}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1504089036}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3015303159}]}]} 2024-06-13T06:52:35.018 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-13T06:52:35.018 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-13T06:52:35.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:52:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:35 smithi175 ceph-mon[36886]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:35.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:35 smithi007 ceph-mon[32077]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:35 smithi189 ceph-mon[36616]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:37 smithi175 ceph-mon[36886]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:37 smithi007 ceph-mon[32077]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:37 smithi189 ceph-mon[36616]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 -- 172.21.15.7:0/2371601594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec100f20 msgr2=0x7fdcec101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 --2- 172.21.15.7:0/2371601594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec100f20 0x7fdcec101300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009990 tx=0x7fdcdc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 -- 172.21.15.7:0/2371601594 shutdown_connections 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 --2- 172.21.15.7:0/2371601594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 --2- 172.21.15.7:0/2371601594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec100f20 0x7fdcec101300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 --2- 172.21.15.7:0/2371601594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec1018d0 0x7fdcec1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 -- 172.21.15.7:0/2371601594 >> 172.21.15.7:0/2371601594 conn(0x7fdcec076640 msgr2=0x7fdcec076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 -- 172.21.15.7:0/2371601594 shutdown_connections 2024-06-13T06:52:37.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.841+0000 7fdceb200700 1 -- 172.21.15.7:0/2371601594 wait complete. 2024-06-13T06:52:37.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.842+0000 7fdceb200700 1 Processor -- start 2024-06-13T06:52:37.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.842+0000 7fdceb200700 1 -- start start 2024-06-13T06:52:37.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.842+0000 7fdceb200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 0x7fdcec19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdceb200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 0x7fdcec19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdceb200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdceb200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1127b0 con 0x7fdcec1018d0 2024-06-13T06:52:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdceb200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec112630 con 0x7fdcec100f20 2024-06-13T06:52:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdceb200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec112930 con 0x7fdcec107790 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 0x7fdcec19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 0x7fdcec19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 0x7fdcec19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58354/0 (socket says 172.21.15.7:58354) 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9400700 1 -- 172.21.15.7:0/322132406 learned_addr learned my addr 172.21.15.7:0/322132406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdcea800700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 -- 172.21.15.7:0/322132406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 msgr2=0x7fdcec19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 -- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 msgr2=0x7fdcec19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 0x7fdcec19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdce9e00700 1 -- 172.21.15.7:0/322132406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec19fd70 con 0x7fdcec100f20 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.843+0000 7fdcea800700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdce9e00700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 0x7fdcec19adb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000c00 tx=0x7fdcdc00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc026070 con 0x7fdcec100f20 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc003d70 con 0x7fdcec100f20 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1a0000 con 0x7fdcec100f20 2024-06-13T06:52:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00fe50 con 0x7fdcec100f20 2024-06-13T06:52:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.844+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a78e0 con 0x7fdcec100f20 2024-06-13T06:52:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.845+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec069180 con 0x7fdcec100f20 2024-06-13T06:52:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.848+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdcdc029070 con 0x7fdcec100f20 2024-06-13T06:52:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.849+0000 7fdce2a00700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 0x7fdcd00a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.849+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fdcdc0d05d0 con 0x7fdcec100f20 2024-06-13T06:52:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.850+0000 7fdce9400700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 0x7fdcd00a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.850+0000 7fdce9400700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 0x7fdcd00a5390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009680 tx=0x7fdcd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:37.851+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcdc030020 con 0x7fdcec100f20 2024-06-13T06:52:38.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.079+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcec1a7ab0 con 0x7fdcec100f20 2024-06-13T06:52:38.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.081+0000 7fdce2a00700 1 -- 172.21.15.7:0/322132406 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7fdcdc013070 con 0x7fdcec100f20 2024-06-13T06:52:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:38.082 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.084+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 msgr2=0x7fdcd00a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.084+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 0x7fdcd00a5390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009680 tx=0x7fdcd8009260 comp rx=0 tx=0).stop 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.084+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 msgr2=0x7fdcec19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.084+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 0x7fdcec19adb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc000c00 tx=0x7fdcdc00bab0 comp rx=0 tx=0).stop 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 shutdown_connections 2024-06-13T06:52:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcec107790 0x7fdcec19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdcd00a2ed0 0x7fdcd00a5390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcec100f20 0x7fdcec19adb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 --2- 172.21.15.7:0/322132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcec1018d0 0x7fdcec19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 >> 172.21.15.7:0/322132406 conn(0x7fdcec076640 msgr2=0x7fdcec0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 shutdown_connections 2024-06-13T06:52:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:38.085+0000 7fdceb200700 1 -- 172.21.15.7:0/322132406 wait complete. 2024-06-13T06:52:38.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:38 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/322132406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:38 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/322132406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:38 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/322132406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:38.902 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-13T06:52:38.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:52:39.382 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:39 smithi175 ceph-mon[36886]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:39 smithi007 ceph-mon[32077]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:39 smithi189 ceph-mon[36616]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:41.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:41 smithi007 ceph-mon[32077]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 -- 172.21.15.7:0/2056714271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec1076f0 msgr2=0x7fd5ec10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 --2- 172.21.15.7:0/2056714271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec10f2c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009f90 tx=0x7fd5e400b330 comp rx=0 tx=0).stop 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 -- 172.21.15.7:0/2056714271 shutdown_connections 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 --2- 172.21.15.7:0/2056714271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec10f2c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 --2- 172.21.15.7:0/2056714271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102610 0x7fd5ec1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 --2- 172.21.15.7:0/2056714271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.709+0000 7fd5f1800700 1 -- 172.21.15.7:0/2056714271 >> 172.21.15.7:0/2056714271 conn(0x7fd5ec0fdea0 msgr2=0x7fd5ec1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.710+0000 7fd5f1800700 1 -- 172.21.15.7:0/2056714271 shutdown_connections 2024-06-13T06:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.710+0000 7fd5f1800700 1 -- 172.21.15.7:0/2056714271 wait complete. 2024-06-13T06:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.711+0000 7fd5f1800700 1 Processor -- start 2024-06-13T06:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.711+0000 7fd5f1800700 1 -- start start 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.711+0000 7fd5f1800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec102610 0x7fd5ec19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.711+0000 7fd5f1800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.711+0000 7fd5f1800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5f1800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec112750 con 0x7fd5ec102fc0 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5f1800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1125d0 con 0x7fd5ec102610 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5f1800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1128d0 con 0x7fd5ec1076f0 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec102610 0x7fd5ec19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37334/0 (socket says 172.21.15.7:37334) 2024-06-13T06:52:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 -- 172.21.15.7:0/4012457135 learned_addr learned my addr 172.21.15.7:0/4012457135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5f0e00700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 -- 172.21.15.7:0/4012457135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec1076f0 msgr2=0x7fd5ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 -- 172.21.15.7:0/4012457135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec102610 msgr2=0x7fd5ec19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec102610 0x7fd5ec19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.712+0000 7fd5eb400700 1 -- 172.21.15.7:0/4012457135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec1990f0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5eb400700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e9f0 tx=0x7fd5d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8004a30 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d800cca0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec1993e0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80056c0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.713+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec1a5650 con 0x7fd5ec102fc0 2024-06-13T06:52:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.714+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ec069180 con 0x7fd5ec102fc0 2024-06-13T06:52:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.717+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd5d800c7c0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.718+0000 7fd5e8c00700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 0x7fd5d00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.719+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd5d80cf630 con 0x7fd5ec102fc0 2024-06-13T06:52:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.719+0000 7fd5ebe00700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 0x7fd5d00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.720+0000 7fd5ebe00700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 0x7fd5d00a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec107520 tx=0x7fd5dc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.724+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5d809c090 con 0x7fd5ec102fc0 2024-06-13T06:52:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:41 smithi189 ceph-mon[36616]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:41 smithi175 ceph-mon[36886]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.957+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5ec1a58e0 con 0x7fd5ec102fc0 2024-06-13T06:52:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.959+0000 7fd5e8c00700 1 -- 172.21.15.7:0/4012457135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7fd5d809b990 con 0x7fd5ec102fc0 2024-06-13T06:52:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:52:41.960 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.962+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 msgr2=0x7fd5d00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.962+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 0x7fd5d00a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec107520 tx=0x7fd5dc009310 comp rx=0 tx=0).stop 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.962+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 msgr2=0x7fd5ec19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.962+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e9f0 tx=0x7fd5d800ed00 comp rx=0 tx=0).stop 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 shutdown_connections 2024-06-13T06:52:41.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5ec1076f0 0x7fd5ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5d00a2d30 0x7fd5d00a51f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ec102610 0x7fd5ec19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 --2- 172.21.15.7:0/4012457135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5ec102fc0 0x7fd5ec19ecc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 >> 172.21.15.7:0/4012457135 conn(0x7fd5ec0fdea0 msgr2=0x7fd5ec0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 shutdown_connections 2024-06-13T06:52:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:41.963+0000 7fd5f1800700 1 -- 172.21.15.7:0/4012457135 wait complete. 2024-06-13T06:52:42.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4012457135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:42.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-13T06:52:42.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-13T06:52:42.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-13T06:52:42.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-13T06:52:42.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-13T06:52:42.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-13T06:52:42.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-13T06:52:42.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-13T06:52:42.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-13T06:52:42.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-13T06:52:42.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-13T06:52:42.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-13T06:52:42.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4012457135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:42.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4012457135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:52:43.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:43 smithi007 ceph-mon[32077]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:43.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:43 smithi189 ceph-mon[36616]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:43 smithi175 ceph-mon[36886]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.926 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.958 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.976 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:43.983 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:45.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:45 smithi007 ceph-mon[32077]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:45 smithi189 ceph-mon[36616]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:45 smithi175 ceph-mon[36886]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:47 smithi007 ceph-mon[32077]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:47 smithi189 ceph-mon[36616]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:47 smithi175 ceph-mon[36886]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:49.674 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:49 smithi007 ceph-mon[32077]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:49 smithi189 ceph-mon[36616]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:49 smithi175 ceph-mon[36886]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:51.722 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:51 smithi007 ceph-mon[32077]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:51 smithi189 ceph-mon[36616]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:51 smithi175 ceph-mon[36886]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:52:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:52:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 -- 172.21.15.7:0/2681160874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 msgr2=0x7f40601115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 --2- 172.21.15.7:0/2681160874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 0x7f40601115a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f405800b0a0 tx=0x7f405800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 -- 172.21.15.7:0/2681160874 shutdown_connections 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 --2- 172.21.15.7:0/2681160874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 0x7f40601115a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 --2- 172.21.15.7:0/2681160874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406010a6d0 0x7f406010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 --2- 172.21.15.7:0/2681160874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40601046d0 0x7f4060104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 -- 172.21.15.7:0/2681160874 >> 172.21.15.7:0/2681160874 conn(0x7f40601001c0 msgr2=0x7f40601025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 -- 172.21.15.7:0/2681160874 shutdown_connections 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.856+0000 7f4067000700 1 -- 172.21.15.7:0/2681160874 wait complete. 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.857+0000 7f4067000700 1 Processor -- start 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.857+0000 7f4067000700 1 -- start start 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 0x7f406019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 0x7f406019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f406010a6d0 0x7f406019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4060114ab0 con 0x7f4060105090 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4060114930 con 0x7f40601046d0 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f4067000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4060114c30 con 0x7f406010a6d0 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f405fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 0x7f406019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f405f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 0x7f406019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f405fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 0x7f406019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32792/0 (socket says 172.21.15.7:32792) 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.858+0000 7f405fe00700 1 -- 172.21.15.7:0/2792320197 learned_addr learned my addr 172.21.15.7:0/2792320197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f4065000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f406010a6d0 0x7f406019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 -- 172.21.15.7:0/2792320197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f406010a6d0 msgr2=0x7f406019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f406010a6d0 0x7f406019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 -- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 msgr2=0x7f406019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4060105090 0x7f406019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406019fe20 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.859+0000 7f405fe00700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 0x7f406019ae60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f405000eb30 tx=0x7f405000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.860+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405000cbd0 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.860+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4050004500 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.860+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40601a0110 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.860+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050010430 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.860+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40601a7990 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.861+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4060062c20 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.863+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4050043b10 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.864+0000 7f405cc00700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40440a2cb0 0x7f40440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f40500cf9b0 con 0x7f40601046d0 2024-06-13T06:52:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f405f400700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40440a2cb0 0x7f40440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f405cc00700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] conn(0x7f40440add90 0x7f40440b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40440b0900 con 0x7f40440add90 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f40500cfd60 con 0x7f40601046d0 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.865+0000 7f4065000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] conn(0x7f40440add90 0x7f40440b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.866+0000 7f405f400700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40440a2cb0 0x7f40440a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4060109c20 tx=0x7f404c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.866+0000 7f4065000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] conn(0x7f40440add90 0x7f40440b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.867+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== osd.1 v2:172.21.15.175:6800/2094167604 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40440b0900 con 0x7f40440add90 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.886+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 --> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40600630a0 con 0x7f40440add90 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f405cc00700 1 -- 172.21.15.7:0/2792320197 <== osd.1 v2:172.21.15.175:6800/2094167604 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f40600630a0 con 0x7f40440add90 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] conn(0x7f40440add90 msgr2=0x7f40440b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] conn(0x7f40440add90 0x7f40440b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40440a2cb0 msgr2=0x7f40440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40440a2cb0 0x7f40440a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4060109c20 tx=0x7f404c009260 comp rx=0 tx=0).stop 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 msgr2=0x7f406019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4067000700 1 --2- 172.21.15.7:0/2792320197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40601046d0 0x7f406019ae60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f405000eb30 tx=0x7f405000ee40 comp rx=0 tx=0).stop 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.887+0000 7f4065000700 1 -- 172.21.15.7:0/2792320197 reap_dead start 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.888+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 shutdown_connections 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.888+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 >> 172.21.15.7:0/2792320197 conn(0x7f40601001c0 msgr2=0x7f40601006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.888+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 shutdown_connections 2024-06-13T06:52:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:52.888+0000 7f4067000700 1 -- 172.21.15.7:0/2792320197 wait complete. 2024-06-13T06:52:53.622 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:53 smithi007 ceph-mon[32077]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:53 smithi189 ceph-mon[36616]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:53 smithi175 ceph-mon[36886]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:52:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 -- 172.21.15.7:0/136129695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f41073d0 msgr2=0x7f27f41077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 --2- 172.21.15.7:0/136129695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f41073d0 0x7f27f41077b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27e80098f0 tx=0x7f27e8009c00 comp rx=0 tx=0).stop 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 -- 172.21.15.7:0/136129695 shutdown_connections 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 --2- 172.21.15.7:0/136129695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 0x7f27f410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 --2- 172.21.15.7:0/136129695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f41073d0 0x7f27f41077b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 --2- 172.21.15.7:0/136129695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f4101390 0x7f27f41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 -- 172.21.15.7:0/136129695 >> 172.21.15.7:0/136129695 conn(0x7f27f40782c0 msgr2=0x7f27f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 -- 172.21.15.7:0/136129695 shutdown_connections 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.819+0000 7f27fb800700 1 -- 172.21.15.7:0/136129695 wait complete. 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.820+0000 7f27fb800700 1 Processor -- start 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.820+0000 7f27fb800700 1 -- start start 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 0x7f27f4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 0x7f27f4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41073d0 0x7f27f41a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4111610 con 0x7f27f4101390 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4111490 con 0x7f27f4101d30 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27fb800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4111790 con 0x7f27f41073d0 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27f8e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 0x7f27f4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27f3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 0x7f27f4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27f8e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 0x7f27f4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44812/0 (socket says 172.21.15.7:44812) 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27f8e00700 1 -- 172.21.15.7:0/1110581205 learned_addr learned my addr 172.21.15.7:0/1110581205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.821+0000 7f27f9800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41073d0 0x7f27f41a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 -- 172.21.15.7:0/1110581205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41073d0 msgr2=0x7f27f41a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41073d0 0x7f27f41a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 -- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 msgr2=0x7f27f4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 0x7f27f4199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f41a5050 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f3e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4101d30 0x7f27f4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f9800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41073d0 0x7f27f41a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f8e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 0x7f27f4199a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009230 tx=0x7f27e800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8005140 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e800b560 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f41a5220 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.822+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003a50 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.823+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f41a5670 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.824+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27f4062ba0 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.826+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f27e8029060 con 0x7f27f4101390 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.827+0000 7f27f1600700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f27dc0a2d30 0x7f27dc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:54.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.827+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f27e80cfce0 con 0x7f27f4101390 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f1600700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] conn(0x7f27dc0ade10 0x7f27dc0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27e8001d30 con 0x7f27dc0ade10 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f27e80d2300 con 0x7f27f4101390 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f3e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f27dc0a2d30 0x7f27dc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f9800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] conn(0x7f27dc0ade10 0x7f27dc0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.828+0000 7f27f3e00700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f27dc0a2d30 0x7f27dc0a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27f40789a0 tx=0x7f27e0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.829+0000 7f27f9800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] conn(0x7f27dc0ade10 0x7f27dc0b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.830+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== osd.10 v2:172.21.15.175:6824/2694135540 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27e8001d30 con 0x7f27dc0ade10 2024-06-13T06:52:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.850+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 --> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27f4063050 con 0x7f27dc0ade10 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27f1600700 1 -- 172.21.15.7:0/1110581205 <== osd.10 v2:172.21.15.175:6824/2694135540 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f27f4063050 con 0x7f27dc0ade10 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] conn(0x7f27dc0ade10 msgr2=0x7f27dc0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] conn(0x7f27dc0ade10 0x7f27dc0b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f27dc0a2d30 msgr2=0x7f27dc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f27dc0a2d30 0x7f27dc0a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27f40789a0 tx=0x7f27e0009310 comp rx=0 tx=0).stop 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 msgr2=0x7f27f4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27fb800700 1 --2- 172.21.15.7:0/1110581205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f4101390 0x7f27f4199a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009230 tx=0x7f27e800fa70 comp rx=0 tx=0).stop 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.851+0000 7f27f9800700 1 -- 172.21.15.7:0/1110581205 reap_dead start 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.852+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 shutdown_connections 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.852+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 >> 172.21.15.7:0/1110581205 conn(0x7f27f40782c0 msgr2=0x7f27f4100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.852+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 shutdown_connections 2024-06-13T06:52:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:54.852+0000 7f27fb800700 1 -- 172.21.15.7:0/1110581205 wait complete. 2024-06-13T06:52:55.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:55 smithi007 ceph-mon[32077]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:55.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:55 smithi189 ceph-mon[36616]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:55 smithi175 ceph-mon[36886]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:52:56.774 INFO:teuthology.orchestra.run.smithi007.stdout:313532612662 2024-06-13T06:52:56.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 -- 172.21.15.7:0/3532646384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0981046b0 msgr2=0x7ff098104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 --2- 172.21.15.7:0/3532646384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0981046b0 0x7ff098104b30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff080009990 tx=0x7ff080009ca0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 -- 172.21.15.7:0/3532646384 shutdown_connections 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 --2- 172.21.15.7:0/3532646384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff098111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 --2- 172.21.15.7:0/3532646384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09810a6b0 0x7ff09810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 --2- 172.21.15.7:0/3532646384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0981046b0 0x7ff098104b30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 -- 172.21.15.7:0/3532646384 >> 172.21.15.7:0/3532646384 conn(0x7ff0981001a0 msgr2=0x7ff0981025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 -- 172.21.15.7:0/3532646384 shutdown_connections 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.411+0000 7ff09f600700 1 -- 172.21.15.7:0/3532646384 wait complete. 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.412+0000 7ff09f600700 1 Processor -- start 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 -- start start 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0981046b0 0x7ff09819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff09819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09810a6b0 0x7ff09819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff098114a40 con 0x7ff098105070 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0981148c0 con 0x7ff0981046b0 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09f600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff098114bc0 con 0x7ff09810a6b0 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff09819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09cc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0981046b0 0x7ff09819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff08fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff09819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44828/0 (socket says 172.21.15.7:44828) 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff08fe00700 1 -- 172.21.15.7:0/3971468732 learned_addr learned my addr 172.21.15.7:0/3971468732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.413+0000 7ff09d600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09810a6b0 0x7ff09819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 -- 172.21.15.7:0/3971468732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09810a6b0 msgr2=0x7ff09819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff09810a6b0 0x7ff09819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 -- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 msgr2=0x7ff09819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff09819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09819fe70 con 0x7ff0981046b0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff08fe00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff098105070 0x7ff09819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff09cc00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0981046b0 0x7ff09819aeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff08800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088004220 con 0x7ff0981046b0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088004d10 con 0x7ff0981046b0 2024-06-13T06:52:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.414+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088005020 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.415+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981a0160 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.415+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981a79e0 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.415+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff098062ba0 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.418+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff088004750 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.419+0000 7ff096000700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff07c0a2cb0 0x7ff07c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.419+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff0880cf640 con 0x7ff0981046b0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.420+0000 7ff08fe00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff07c0a2cb0 0x7ff07c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.420+0000 7ff096000700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] conn(0x7ff07c0add90 0x7ff07c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.420+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff07c0b0900 con 0x7ff07c0add90 2024-06-13T06:52:57.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.420+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7ff0880cf9f0 con 0x7ff0981046b0 2024-06-13T06:52:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.420+0000 7ff09d600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] conn(0x7ff07c0add90 0x7ff07c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.421+0000 7ff08fe00700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff07c0a2cb0 0x7ff07c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff098109d00 tx=0x7ff080009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.421+0000 7ff09d600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] conn(0x7ff07c0add90 0x7ff07c0b0230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.422+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== osd.7 v2:172.21.15.175:6816/935680699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff07c0b0900 con 0x7ff07c0add90 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.441+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 --> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff098063020 con 0x7ff07c0add90 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff096000700 1 -- 172.21.15.7:0/3971468732 <== osd.7 v2:172.21.15.175:6816/935680699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff098063020 con 0x7ff07c0add90 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] conn(0x7ff07c0add90 msgr2=0x7ff07c0b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] conn(0x7ff07c0add90 0x7ff07c0b0230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff07c0a2cb0 msgr2=0x7ff07c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff07c0a2cb0 0x7ff07c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff098109d00 tx=0x7ff080009570 comp rx=0 tx=0).stop 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0981046b0 msgr2=0x7ff09819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09f600700 1 --2- 172.21.15.7:0/3971468732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0981046b0 0x7ff09819aeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9f0 tx=0x7ff08800ed00 comp rx=0 tx=0).stop 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.442+0000 7ff09d600700 1 -- 172.21.15.7:0/3971468732 reap_dead start 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.443+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 shutdown_connections 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.443+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 >> 172.21.15.7:0/3971468732 conn(0x7ff0981001a0 msgr2=0x7ff098100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.443+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 shutdown_connections 2024-06-13T06:52:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:56.443+0000 7ff09f600700 1 -- 172.21.15.7:0/3971468732 wait complete. 2024-06-13T06:52:57.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:57 smithi007 ceph-mon[32077]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:57 smithi189 ceph-mon[36616]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:57 smithi175 ceph-mon[36886]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.515+0000 7f36d8600700 1 -- 172.21.15.7:0/1959045390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d01030f0 msgr2=0x7f36d0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.515+0000 7f36d8600700 1 --2- 172.21.15.7:0/1959045390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d01030f0 0x7f36d0107110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0099c0 tx=0x7f36bc009cd0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.516+0000 7f36d8600700 1 -- 172.21.15.7:0/1959045390 shutdown_connections 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.516+0000 7f36d8600700 1 --2- 172.21.15.7:0/1959045390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0107840 0x7f36d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.516+0000 7f36d8600700 1 --2- 172.21.15.7:0/1959045390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.516+0000 7f36d8600700 1 --2- 172.21.15.7:0/1959045390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d01030f0 0x7f36d0107110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.516+0000 7f36d8600700 1 -- 172.21.15.7:0/1959045390 >> 172.21.15.7:0/1959045390 conn(0x7f36d00795f0 msgr2=0x7f36d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.517+0000 7f36d8600700 1 -- 172.21.15.7:0/1959045390 shutdown_connections 2024-06-13T06:52:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.517+0000 7f36d8600700 1 -- 172.21.15.7:0/1959045390 wait complete. 2024-06-13T06:52:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.518+0000 7f36d8600700 1 Processor -- start 2024-06-13T06:52:58.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.518+0000 7f36d8600700 1 -- start start 2024-06-13T06:52:58.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:58.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d01030f0 0x7f36d019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d0107840 0x7f36d0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36d01127a0 con 0x7f36d01030f0 2024-06-13T06:52:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36d0112620 con 0x7f36d0107840 2024-06-13T06:52:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d8600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36d0112920 con 0x7f36d0102740 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d5c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d5c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37644/0 (socket says 172.21.15.7:37644) 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d5c00700 1 -- 172.21.15.7:0/145327245 learned_addr learned my addr 172.21.15.7:0/145327245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d6600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d0107840 0x7f36d0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.519+0000 7f36d5200700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d01030f0 0x7f36d019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 -- 172.21.15.7:0/145327245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d0107840 msgr2=0x7f36d0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d0107840 0x7f36d0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 -- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d01030f0 msgr2=0x7f36d019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d01030f0 0x7f36d019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d0199180 con 0x7f36d0102740 2024-06-13T06:52:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d6600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36d0107840 0x7f36d0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:52:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5200700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d01030f0 0x7f36d019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:52:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d5c00700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d019e860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9a0 tx=0x7f36c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c400cbd0 con 0x7f36d0102740 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c400cd70 con 0x7f36d0102740 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d0199470 con 0x7f36d0102740 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4018750 con 0x7f36d0102740 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.520+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d01a5730 con 0x7f36d0102740 2024-06-13T06:52:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.522+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36d0062c20 con 0x7f36d0102740 2024-06-13T06:52:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.524+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f36c4010430 con 0x7f36d0102740 2024-06-13T06:52:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.525+0000 7f36ce000700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36b80a2d30 0x7f36b80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.525+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36c40cf7b0 con 0x7f36d0102740 2024-06-13T06:52:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.525+0000 7f36d5200700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36b80a2d30 0x7f36b80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.525+0000 7f36ce000700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] conn(0x7f36b80ade10 0x7f36b80b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:52:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.526+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36c4012ec0 con 0x7f36b80ade10 2024-06-13T06:52:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.526+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f36c40d6050 con 0x7f36d0102740 2024-06-13T06:52:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.526+0000 7f36d6600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] conn(0x7f36b80ade10 0x7f36b80b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:52:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.526+0000 7f36d5200700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36b80a2d30 0x7f36b80a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f36d01070e0 tx=0x7f36bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.527+0000 7f36d6600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] conn(0x7f36b80ade10 0x7f36b80b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:52:58.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.528+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== osd.5 v2:172.21.15.189:6808/1526104805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36c4012ec0 con 0x7f36b80ade10 2024-06-13T06:52:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.546+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 --> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36d0063020 con 0x7f36b80ade10 2024-06-13T06:52:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.546+0000 7f36ce000700 1 -- 172.21.15.7:0/145327245 <== osd.5 v2:172.21.15.189:6808/1526104805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36d0063020 con 0x7f36b80ade10 2024-06-13T06:52:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] conn(0x7f36b80ade10 msgr2=0x7f36b80b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] conn(0x7f36b80ade10 0x7f36b80b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36b80a2d30 msgr2=0x7f36b80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36b80a2d30 0x7f36b80a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f36d01070e0 tx=0x7f36bc005fd0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 msgr2=0x7f36d019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 --2- 172.21.15.7:0/145327245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36d0102740 0x7f36d019e860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36c400e9a0 tx=0x7f36c400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d6600700 1 -- 172.21.15.7:0/145327245 reap_dead start 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 shutdown_connections 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.547+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 >> 172.21.15.7:0/145327245 conn(0x7f36d00795f0 msgr2=0x7f36d0100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.548+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 shutdown_connections 2024-06-13T06:52:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.548+0000 7f36d8600700 1 -- 172.21.15.7:0/145327245 wait complete. 2024-06-13T06:52:59.357 INFO:teuthology.orchestra.run.smithi007.stdout:55834574954 2024-06-13T06:52:59.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-13T06:52:59.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:52:59 smithi007 ceph-mon[32077]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:52:59 smithi189 ceph-mon[36616]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:52:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:52:59 smithi175 ceph-mon[36886]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 -- 172.21.15.7:0/899452373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 msgr2=0x7f4ddc108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 --2- 172.21.15.7:0/899452373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc108f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009990 tx=0x7f4dc4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 -- 172.21.15.7:0/899452373 shutdown_connections 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 --2- 172.21.15.7:0/899452373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 0x7f4ddc115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 --2- 172.21.15.7:0/899452373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 0x7f4ddc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 --2- 172.21.15.7:0/899452373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc108f40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 -- 172.21.15.7:0/899452373 >> 172.21.15.7:0/899452373 conn(0x7f4ddc075c60 msgr2=0x7f4ddc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 -- 172.21.15.7:0/899452373 shutdown_connections 2024-06-13T06:53:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.467+0000 7f4de1200700 1 -- 172.21.15.7:0/899452373 wait complete. 2024-06-13T06:53:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.468+0000 7f4de1200700 1 Processor -- start 2024-06-13T06:53:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.468+0000 7f4de1200700 1 -- start start 2024-06-13T06:53:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 0x7f4ddc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 0x7f4ddc1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc118ea0 con 0x7f4ddc109480 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc118d20 con 0x7f4ddc108ac0 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4de1200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc119020 con 0x7f4ddc10eac0 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4dd9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4dd9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32900/0 (socket says 172.21.15.7:32900) 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4dd9e00700 1 -- 172.21.15.7:0/3888051250 learned_addr learned my addr 172.21.15.7:0/3888051250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4dd9400700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 0x7f4ddc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.469+0000 7f4dda800700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 0x7f4ddc1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 -- 172.21.15.7:0/3888051250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 msgr2=0x7f4ddc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 -- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 msgr2=0x7f4ddc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 0x7f4ddc19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc1a42c0 con 0x7f4ddc10eac0 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dd9e00700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ddc108ac0 0x7f4ddc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dd9400700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc109480 0x7f4ddc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dda800700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 0x7f4ddc1a3b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400efb0 tx=0x7f4dd400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4003a00 con 0x7f4ddc10eac0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd4003ba0 con 0x7f4ddc10eac0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc1a45b0 con 0x7f4ddc10eac0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.470+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4004850 con 0x7f4ddc10eac0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.471+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1abe30 con 0x7f4ddc10eac0 2024-06-13T06:53:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.472+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ddc062ba0 con 0x7f4ddc10eac0 2024-06-13T06:53:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.474+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4dd4012050 con 0x7f4ddc10eac0 2024-06-13T06:53:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.475+0000 7f4dd2a00700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4dc00a2cb0 0x7f4dc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.475+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4dd40cf110 con 0x7f4ddc10eac0 2024-06-13T06:53:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.475+0000 7f4dd9e00700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4dc00a2cb0 0x7f4dc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.476+0000 7f4dd2a00700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] conn(0x7f4dc00add90 0x7f4dc00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.476+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4dc00b0900 con 0x7f4dc00add90 2024-06-13T06:53:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.476+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f4dd40cf4c0 con 0x7f4ddc10eac0 2024-06-13T06:53:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.476+0000 7f4dd9400700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] conn(0x7f4dc00add90 0x7f4dc00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.476+0000 7f4dd9e00700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4dc00a2cb0 0x7f4dc00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc011550 tx=0x7f4dcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.477+0000 7f4dd9400700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] conn(0x7f4dc00add90 0x7f4dc00b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.478+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== osd.8 v2:172.21.15.189:6816/1210852331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4dc00b0900 con 0x7f4dc00add90 2024-06-13T06:53:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.496+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 --> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ddc063020 con 0x7f4dc00add90 2024-06-13T06:53:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.497+0000 7f4dd2a00700 1 -- 172.21.15.7:0/3888051250 <== osd.8 v2:172.21.15.189:6816/1210852331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ddc063020 con 0x7f4dc00add90 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] conn(0x7f4dc00add90 msgr2=0x7f4dc00b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] conn(0x7f4dc00add90 0x7f4dc00b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4dc00a2cb0 msgr2=0x7f4dc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4dc00a2cb0 0x7f4dc00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc011550 tx=0x7f4dcc009260 comp rx=0 tx=0).stop 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 msgr2=0x7f4ddc1a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 --2- 172.21.15.7:0/3888051250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ddc10eac0 0x7f4ddc1a3b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400efb0 tx=0x7f4dd400c3a0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4dda800700 1 -- 172.21.15.7:0/3888051250 reap_dead start 2024-06-13T06:53:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 shutdown_connections 2024-06-13T06:53:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 >> 172.21.15.7:0/3888051250 conn(0x7f4ddc075c60 msgr2=0x7f4ddc076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 shutdown_connections 2024-06-13T06:53:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:00.498+0000 7f4de1200700 1 -- 172.21.15.7:0/3888051250 wait complete. 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 -- 172.21.15.7:0/3118202429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8105030 msgr2=0x7f8fd8111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 --2- 172.21.15.7:0/3118202429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8105030 0x7f8fd8111540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009f90 tx=0x7f8fd000b330 comp rx=0 tx=0).stop 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 -- 172.21.15.7:0/3118202429 shutdown_connections 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 --2- 172.21.15.7:0/3118202429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8105030 0x7f8fd8111540 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 --2- 172.21.15.7:0/3118202429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd810a670 0x7f8fd810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 --2- 172.21.15.7:0/3118202429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd8104670 0x7f8fd8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 -- 172.21.15.7:0/3118202429 >> 172.21.15.7:0/3118202429 conn(0x7f8fd8100140 msgr2=0x7f8fd8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 -- 172.21.15.7:0/3118202429 shutdown_connections 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.680+0000 7f8fdea00700 1 -- 172.21.15.7:0/3118202429 wait complete. 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.681+0000 7f8fdea00700 1 Processor -- start 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 -- start start 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 0x7f8fd819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd8105030 0x7f8fd819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd810a670 0x7f8fd819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114a10 con 0x7f8fd810a670 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114890 con 0x7f8fd8104670 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdea00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114b90 con 0x7f8fd8105030 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fd7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 0x7f8fd819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fdca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd810a670 0x7f8fd819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fd7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 0x7f8fd819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32880/0 (socket says 172.21.15.7:32880) 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fd7e00700 1 -- 172.21.15.7:0/1599108594 learned_addr learned my addr 172.21.15.7:0/1599108594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.682+0000 7f8fd7400700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd8105030 0x7f8fd819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 -- 172.21.15.7:0/1599108594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd8105030 msgr2=0x7f8fd819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd8105030 0x7f8fd819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 -- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd810a670 msgr2=0x7f8fd819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd810a670 0x7f8fd819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd819fe30 con 0x7f8fd8104670 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fdca00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd810a670 0x7f8fd819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd7e00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 0x7f8fd819ae20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800e9c0 tx=0x7f8fc800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800cbd0 con 0x7f8fd8104670 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc800ee50 con 0x7f8fd8104670 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.683+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a0120 con 0x7f8fd8104670 2024-06-13T06:53:00.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.684+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005680 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.684+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a79a0 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.684+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fd8062c20 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.687+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8fc8004750 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.688+0000 7f8fd4c00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8fbc0a2d30 0x7f8fbc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.688+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8fc80cfb10 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.688+0000 7f8fd7400700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8fbc0a2d30 0x7f8fbc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.689+0000 7f8fd4c00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] conn(0x7f8fbc0ade10 0x7f8fbc0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.689+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fc800cdc0 con 0x7f8fbc0ade10 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.689+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f8fc80cfec0 con 0x7f8fd8104670 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.689+0000 7f8fdca00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] conn(0x7f8fbc0ade10 0x7f8fbc0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.689+0000 7f8fd7400700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8fbc0a2d30 0x7f8fbc0a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8109c90 tx=0x7f8fc4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.690+0000 7f8fdca00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] conn(0x7f8fbc0ade10 0x7f8fbc0b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.691+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== osd.0 v2:172.21.15.7:6802/625935948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fc800cdc0 con 0x7f8fbc0ade10 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.709+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 --> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fd80630a0 con 0x7f8fbc0ade10 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.710+0000 7f8fd4c00700 1 -- 172.21.15.7:0/1599108594 <== osd.0 v2:172.21.15.7:6802/625935948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8fd80630a0 con 0x7f8fbc0ade10 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.710+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] conn(0x7f8fbc0ade10 msgr2=0x7f8fbc0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.710+0000 7f8fdea00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] conn(0x7f8fbc0ade10 0x7f8fbc0b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8fbc0a2d30 msgr2=0x7f8fbc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8fbc0a2d30 0x7f8fbc0a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8109c90 tx=0x7f8fc4009310 comp rx=0 tx=0).stop 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 msgr2=0x7f8fd819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 --2- 172.21.15.7:0/1599108594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fd8104670 0x7f8fd819ae20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800e9c0 tx=0x7f8fc800ecd0 comp rx=0 tx=0).stop 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdca00700 1 -- 172.21.15.7:0/1599108594 reap_dead start 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 shutdown_connections 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 >> 172.21.15.7:0/1599108594 conn(0x7f8fd8100140 msgr2=0x7f8fd81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 shutdown_connections 2024-06-13T06:53:00.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:52:58.711+0000 7f8fdea00700 1 -- 172.21.15.7:0/1599108594 wait complete. 2024-06-13T06:53:01.748 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:01 smithi007 ceph-mon[32077]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:01 smithi189 ceph-mon[36616]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:01 smithi175 ceph-mon[36886]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 -- 172.21.15.7:0/2607792316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b64108530 msgr2=0x7f1b64108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2607792316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b64108530 0x7f1b64108910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b540098f0 tx=0x7f1b54009c00 comp rx=0 tx=0).stop 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 -- 172.21.15.7:0/2607792316 shutdown_connections 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2607792316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6406b520 0x7f1b6410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2607792316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b64108530 0x7f1b64108910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2607792316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b6406ab60 0x7f1b6406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 -- 172.21.15.7:0/2607792316 >> 172.21.15.7:0/2607792316 conn(0x7f1b640783d0 msgr2=0x7f1b640787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.772+0000 7f1b6a400700 1 -- 172.21.15.7:0/2607792316 shutdown_connections 2024-06-13T06:53:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.773+0000 7f1b6a400700 1 -- 172.21.15.7:0/2607792316 wait complete. 2024-06-13T06:53:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.774+0000 7f1b6a400700 1 Processor -- start 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 -- start start 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b6406ab60 0x7f1b6419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 0x7f1b6419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b64108530 0x7f1b6419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b641127e0 con 0x7f1b6406ab60 2024-06-13T06:53:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b64112660 con 0x7f1b6406b520 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.775+0000 7f1b6a400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b64112960 con 0x7f1b64108530 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b63400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b6406ab60 0x7f1b6419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 0x7f1b6419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 0x7f1b6419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51798/0 (socket says 172.21.15.7:51798) 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 -- 172.21.15.7:0/2050576993 learned_addr learned my addr 172.21.15.7:0/2050576993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b63e00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b64108530 0x7f1b6419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:02.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 -- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b64108530 msgr2=0x7f1b6419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b64108530 0x7f1b6419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 -- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b6406ab60 msgr2=0x7f1b6419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b6406ab60 0x7f1b6419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.776+0000 7f1b62a00700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6419fef0 con 0x7f1b6406b520 2024-06-13T06:53:02.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b62a00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 0x7f1b6419b420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9c0 tx=0x7f1b5000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50004a30 con 0x7f1b6406b520 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b5000cca0 con 0x7f1b6406b520 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50005610 con 0x7f1b6406b520 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b641a01e0 con 0x7f1b6406b520 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.777+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b641a7a60 con 0x7f1b6406b520 2024-06-13T06:53:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.778+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b6410e100 con 0x7f1b6406b520 2024-06-13T06:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.781+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1b50004cf0 con 0x7f1b6406b520 2024-06-13T06:53:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.782+0000 7f1b5be00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1b500cfab0 con 0x7f1b6406b520 2024-06-13T06:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b63400700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b5be00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] conn(0x7f1b480add90 0x7f1b480b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b5000ce50 con 0x7f1b480add90 2024-06-13T06:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f1b500cfe60 con 0x7f1b6406b520 2024-06-13T06:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.783+0000 7f1b63e00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] conn(0x7f1b480add90 0x7f1b480b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.784+0000 7f1b63400700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009810 tx=0x7f1b54019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.784+0000 7f1b63e00700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] conn(0x7f1b480add90 0x7f1b480b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.785+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== osd.2 v2:172.21.15.189:6800/2765218208 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b5000ce50 con 0x7f1b480add90 2024-06-13T06:53:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.803+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 --> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b64103bf0 con 0x7f1b480add90 2024-06-13T06:53:02.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.804+0000 7f1b5be00700 1 -- 172.21.15.7:0/2050576993 <== osd.2 v2:172.21.15.189:6800/2765218208 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b64103bf0 con 0x7f1b480add90 2024-06-13T06:53:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] conn(0x7f1b480add90 msgr2=0x7f1b480b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] conn(0x7f1b480add90 0x7f1b480b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 msgr2=0x7f1b480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009810 tx=0x7f1b54019040 comp rx=0 tx=0).stop 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 msgr2=0x7f1b6419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 --2- 172.21.15.7:0/2050576993 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b6406b520 0x7f1b6419b420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e9c0 tx=0x7f1b5000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b63e00700 1 -- 172.21.15.7:0/2050576993 reap_dead start 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 shutdown_connections 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.805+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 >> 172.21.15.7:0/2050576993 conn(0x7f1b640783d0 msgr2=0x7f1b6410d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.806+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 shutdown_connections 2024-06-13T06:53:02.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.806+0000 7f1b6a400700 1 -- 172.21.15.7:0/2050576993 wait complete. 2024-06-13T06:53:03.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:03 smithi007 ceph-mon[32077]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:03 smithi189 ceph-mon[36616]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:223338299464 2024-06-13T06:53:03.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-13T06:53:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:03 smithi175 ceph-mon[36886]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 -- 172.21.15.7:0/1796609826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 msgr2=0x7f03641029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 --2- 172.21.15.7:0/1796609826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 0x7f03641029a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f034c009990 tx=0x7f034c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 -- 172.21.15.7:0/1796609826 shutdown_connections 2024-06-13T06:53:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 --2- 172.21.15.7:0/1796609826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0364102ee0 0x7f036410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 --2- 172.21.15.7:0/1796609826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364107450 0x7f0364107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 --2- 172.21.15.7:0/1796609826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 0x7f03641029a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.570+0000 7f036ac00700 1 -- 172.21.15.7:0/1796609826 >> 172.21.15.7:0/1796609826 conn(0x7f03640795f0 msgr2=0x7f0364079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.571+0000 7f036ac00700 1 -- 172.21.15.7:0/1796609826 shutdown_connections 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.571+0000 7f036ac00700 1 -- 172.21.15.7:0/1796609826 wait complete. 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 Processor -- start 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 -- start start 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 0x7f036419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 0x7f036419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0364107450 0x7f036419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0364112850 con 0x7f0364102ee0 2024-06-13T06:53:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03641126d0 con 0x7f0364102540 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.572+0000 7f036ac00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03641129d0 con 0x7f0364107450 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 0x7f036419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 0x7f036419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39048/0 (socket says 172.21.15.7:39048) 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 -- 172.21.15.7:0/3983620985 learned_addr learned my addr 172.21.15.7:0/3983620985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f0363e00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 0x7f036419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f0368c00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0364107450 0x7f036419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 -- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0364107450 msgr2=0x7f036419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0364107450 0x7f036419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 -- 172.21.15.7:0/3983620985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 msgr2=0x7f036419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0364102540 0x7f036419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.573+0000 7f035b400700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036419fe70 con 0x7f0364102ee0 2024-06-13T06:53:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f035b400700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 0x7f036419b3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c027070 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034c0056d0 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641a0100 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c022e10 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.574+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641a79e0 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.575+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0364100df0 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.577+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f034c02a070 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0361600700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03480a2cb0 0x7f03480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f034c0d0aa0 con 0x7f0364102ee0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0363e00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03480a2cb0 0x7f03480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0361600700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] conn(0x7f03480add90 0x7f03480b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03480b0900 con 0x7f03480add90 2024-06-13T06:53:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.579+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f034c0d0e50 con 0x7f0364102ee0 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.580+0000 7f0368c00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] conn(0x7f03480add90 0x7f03480b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.580+0000 7f0363e00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03480a2cb0 0x7f03480a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0364103720 tx=0x7f035400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.581+0000 7f0368c00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] conn(0x7f03480add90 0x7f03480b0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.582+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== osd.11 v2:172.21.15.189:6824/1625559024 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03480b0900 con 0x7f03480add90 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.600+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 --> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0364103c00 con 0x7f03480add90 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.601+0000 7f0361600700 1 -- 172.21.15.7:0/3983620985 <== osd.11 v2:172.21.15.189:6824/1625559024 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0364103c00 con 0x7f03480add90 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] conn(0x7f03480add90 msgr2=0x7f03480b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] conn(0x7f03480add90 0x7f03480b0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03480a2cb0 msgr2=0x7f03480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03480a2cb0 0x7f03480a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0364103720 tx=0x7f035400b040 comp rx=0 tx=0).stop 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 msgr2=0x7f036419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 --2- 172.21.15.7:0/3983620985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0364102ee0 0x7f036419b3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c00bab0 comp rx=0 tx=0).stop 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f0368c00700 1 -- 172.21.15.7:0/3983620985 reap_dead start 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 shutdown_connections 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 >> 172.21.15.7:0/3983620985 conn(0x7f03640795f0 msgr2=0x7f0364100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 shutdown_connections 2024-06-13T06:53:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:02.602+0000 7f036ac00700 1 -- 172.21.15.7:0/3983620985 wait complete. 2024-06-13T06:53:05.033 INFO:teuthology.orchestra.run.smithi007.stdout:34359738479 2024-06-13T06:53:05.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-13T06:53:05.119 INFO:teuthology.orchestra.run.smithi007.stdout:257698037828 2024-06-13T06:53:05.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-13T06:53:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:05 smithi007 ceph-mon[32077]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:05 smithi189 ceph-mon[36616]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:05.860 INFO:teuthology.orchestra.run.smithi007.stdout:167503724629 2024-06-13T06:53:05.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-13T06:53:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:05 smithi175 ceph-mon[36886]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:07.595 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:07 smithi007 ceph-mon[32077]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 -- 172.21.15.7:0/1405994990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 msgr2=0x7fb7b81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 --2- 172.21.15.7:0/1405994990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b81029a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40099c0 tx=0x7fb7a4009cd0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 -- 172.21.15.7:0/1405994990 shutdown_connections 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 --2- 172.21.15.7:0/1405994990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b8102ee0 0x7fb7b810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 --2- 172.21.15.7:0/1405994990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 0x7fb7b81089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 --2- 172.21.15.7:0/1405994990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b81029a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 -- 172.21.15.7:0/1405994990 >> 172.21.15.7:0/1405994990 conn(0x7fb7b8079680 msgr2=0x7fb7b8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 -- 172.21.15.7:0/1405994990 shutdown_connections 2024-06-13T06:53:07.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.604+0000 7fb7be600700 1 -- 172.21.15.7:0/1405994990 wait complete. 2024-06-13T06:53:07.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.605+0000 7fb7be600700 1 Processor -- start 2024-06-13T06:53:07.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 -- start start 2024-06-13T06:53:07.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:07.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b8102ee0 0x7fb7b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:07.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 0x7fb7b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:07.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8112860 con 0x7fb7b8102520 2024-06-13T06:53:07.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81126e0 con 0x7fb7b81085c0 2024-06-13T06:53:07.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7be600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81129e0 con 0x7fb7b8102ee0 2024-06-13T06:53:07.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.606+0000 7fb7b7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:07.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39086/0 (socket says 172.21.15.7:39086) 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 -- 172.21.15.7:0/1862940563 learned_addr learned my addr 172.21.15.7:0/1862940563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b6a00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b8102ee0 0x7fb7b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7e00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 0x7fb7b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 -- 172.21.15.7:0/1862940563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b8102ee0 msgr2=0x7fb7b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b8102ee0 0x7fb7b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 -- 172.21.15.7:0/1862940563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 msgr2=0x7fb7b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 0x7fb7b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7400700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b81a7190 con 0x7fb7b8102520 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.607+0000 7fb7b7e00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b81085c0 0x7fb7b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:07.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7b7400700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b8074790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9a0 tx=0x7fb7a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800ca10 con 0x7fb7b8102520 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a800cbb0 con 0x7fb7b8102520 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b81a7480 con 0x7fb7b8102520 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8018680 con 0x7fb7b8102520 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.608+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b81a79e0 con 0x7fb7b8102520 2024-06-13T06:53:07.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.609+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7b8062ba0 con 0x7fb7b8102520 2024-06-13T06:53:07.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.613+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb7a8010430 con 0x7fb7b8102520 2024-06-13T06:53:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.614+0000 7fb7afe00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb79c0a2d30 0x7fb79c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.614+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb7a80d0240 con 0x7fb7b8102520 2024-06-13T06:53:07.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.614+0000 7fb7b6a00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb79c0a2d30 0x7fb79c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.614+0000 7fb7afe00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] conn(0x7fb79c0ade10 0x7fb79c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.614+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb79c0b0980 con 0x7fb79c0ade10 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.615+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fb7a80d05f0 con 0x7fb7b8102520 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.615+0000 7fb7b7e00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] conn(0x7fb79c0ade10 0x7fb79c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.615+0000 7fb7b6a00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb79c0a2d30 0x7fb79c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80761d0 tx=0x7fb7a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.615+0000 7fb7b7e00700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] conn(0x7fb79c0ade10 0x7fb79c0b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.617+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== osd.3 v2:172.21.15.7:6810/2396372699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb79c0b0980 con 0x7fb79c0ade10 2024-06-13T06:53:07.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.635+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 --> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7b8062fa0 con 0x7fb79c0ade10 2024-06-13T06:53:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7afe00700 1 -- 172.21.15.7:0/1862940563 <== osd.3 v2:172.21.15.7:6810/2396372699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7b8062fa0 con 0x7fb79c0ade10 2024-06-13T06:53:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] conn(0x7fb79c0ade10 msgr2=0x7fb79c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7be600700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] conn(0x7fb79c0ade10 0x7fb79c0b02b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb79c0a2d30 msgr2=0x7fb79c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7be600700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb79c0a2d30 0x7fb79c0a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80761d0 tx=0x7fb7a4005fb0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.636+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 msgr2=0x7fb7b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7be600700 1 --2- 172.21.15.7:0/1862940563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102520 0x7fb7b8074790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9a0 tx=0x7fb7a800ecb0 comp rx=0 tx=0).stop 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7b7e00700 1 -- 172.21.15.7:0/1862940563 reap_dead start 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 shutdown_connections 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 >> 172.21.15.7:0/1862940563 conn(0x7fb7b8079680 msgr2=0x7fb7b8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 shutdown_connections 2024-06-13T06:53:07.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:07.637+0000 7fb7be600700 1 -- 172.21.15.7:0/1862940563 wait complete. 2024-06-13T06:53:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:07 smithi189 ceph-mon[36616]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:07 smithi175 ceph-mon[36886]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:08.110 INFO:teuthology.orchestra.run.smithi007.stdout:347892351027 2024-06-13T06:53:08.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-13T06:53:09.667 INFO:teuthology.orchestra.run.smithi007.stdout:111669149792 2024-06-13T06:53:09.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-13T06:53:09.724 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:09 smithi007 ceph-mon[32077]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 -- 172.21.15.7:0/1495948259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c88108500 msgr2=0x7f9c881088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1495948259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c88108500 0x7f9c881088e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009990 tx=0x7f9c78009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 -- 172.21.15.7:0/1495948259 shutdown_connections 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1495948259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c8806ab30 0x7f9c8810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1495948259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c88108500 0x7f9c881088e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1495948259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 -- 172.21.15.7:0/1495948259 >> 172.21.15.7:0/1495948259 conn(0x7f9c88078340 msgr2=0x7f9c88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 -- 172.21.15.7:0/1495948259 shutdown_connections 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.593+0000 7f9c8de00700 1 -- 172.21.15.7:0/1495948259 wait complete. 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.594+0000 7f9c8de00700 1 Processor -- start 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.594+0000 7f9c8de00700 1 -- start start 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8806ab30 0x7f9c8819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c88108500 0x7f9c8819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881127b0 con 0x7f9c8806ab30 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112630 con 0x7f9c8806a170 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c8de00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88112930 con 0x7f9c88108500 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:09.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51824/0 (socket says 172.21.15.7:51824) 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c86000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8806ab30 0x7f9c8819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c86a00700 1 -- 172.21.15.7:0/1540998559 learned_addr learned my addr 172.21.15.7:0/1540998559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.595+0000 7f9c87400700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c88108500 0x7f9c8819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 -- 172.21.15.7:0/1540998559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c88108500 msgr2=0x7f9c8819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c88108500 0x7f9c8819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 -- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8806ab30 msgr2=0x7f9c8819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8806ab30 0x7f9c8819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8819fe70 con 0x7f9c8806a170 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86000700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8806ab30 0x7f9c8819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.596+0000 7f9c86a00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8819ae60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005910 tx=0x7f9c7800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.597+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78027070 con 0x7f9c8806a170 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.597+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c78003d70 con 0x7f9c8806a170 2024-06-13T06:53:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.597+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800f460 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.597+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881a0100 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.597+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a79e0 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.599+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c68001030 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.602+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9c7802a070 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.603+0000 7f9c8ca00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c6c0a2cb0 0x7f9c6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9c780d0a90 con 0x7f9c8806a170 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c86000700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c6c0a2cb0 0x7f9c6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c8ca00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] conn(0x7f9c6c0add90 0x7f9c6c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c6c0b0900 con 0x7f9c6c0add90 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f9c780d0e40 con 0x7f9c8806a170 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.604+0000 7f9c87400700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] conn(0x7f9c6c0add90 0x7f9c6c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.605+0000 7f9c86000700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c6c0a2cb0 0x7f9c6c0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c88103440 tx=0x7f9c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.605+0000 7f9c87400700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] conn(0x7f9c6c0add90 0x7f9c6c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.607+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== osd.6 v2:172.21.15.7:6818/3095237375 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c6c0b0900 con 0x7f9c6c0add90 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.631+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 --> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c68002e60 con 0x7f9c6c0add90 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8ca00700 1 -- 172.21.15.7:0/1540998559 <== osd.6 v2:172.21.15.7:6818/3095237375 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c68002e60 con 0x7f9c6c0add90 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] conn(0x7f9c6c0add90 msgr2=0x7f9c6c0b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] conn(0x7f9c6c0add90 0x7f9c6c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c6c0a2cb0 msgr2=0x7f9c6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c6c0a2cb0 0x7f9c6c0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c88103440 tx=0x7f9c74009260 comp rx=0 tx=0).stop 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 msgr2=0x7f9c8819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.632+0000 7f9c8de00700 1 --2- 172.21.15.7:0/1540998559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c8806a170 0x7f9c8819ae60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005910 tx=0x7f9c7800bab0 comp rx=0 tx=0).stop 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.633+0000 7f9c87400700 1 -- 172.21.15.7:0/1540998559 reap_dead start 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.633+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 shutdown_connections 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.633+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 >> 172.21.15.7:0/1540998559 conn(0x7f9c88078340 msgr2=0x7f9c8810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.633+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 shutdown_connections 2024-06-13T06:53:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:06.633+0000 7f9c8de00700 1 -- 172.21.15.7:0/1540998559 wait complete. 2024-06-13T06:53:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:09 smithi189 ceph-mon[36616]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:09 smithi175 ceph-mon[36886]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:10.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 -- 172.21.15.7:0/1510370894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059806aac0 msgr2=0x7f059810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 --2- 172.21.15.7:0/1510370894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059806aac0 0x7f059810ea00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05840098f0 tx=0x7f0584009c00 comp rx=0 tx=0).stop 2024-06-13T06:53:10.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 -- 172.21.15.7:0/1510370894 shutdown_connections 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 --2- 172.21.15.7:0/1510370894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f059810ef40 0x7f0598111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 --2- 172.21.15.7:0/1510370894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806a1a0 0x7f059806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 --2- 172.21.15.7:0/1510370894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059806aac0 0x7f059810ea00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.054+0000 7f05a0000700 1 -- 172.21.15.7:0/1510370894 >> 172.21.15.7:0/1510370894 conn(0x7f0598078340 msgr2=0x7f0598078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.055+0000 7f05a0000700 1 -- 172.21.15.7:0/1510370894 shutdown_connections 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.055+0000 7f05a0000700 1 -- 172.21.15.7:0/1510370894 wait complete. 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 Processor -- start 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 -- start start 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f059806a1a0 0x7f059819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806aac0 0x7f059819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 0x7f059819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05981149f0 con 0x7f059810ef40 2024-06-13T06:53:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0598114870 con 0x7f059806aac0 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.056+0000 7f05a0000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0598114b70 con 0x7f059806a1a0 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 0x7f059819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 0x7f059819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39090/0 (socket says 172.21.15.7:39090) 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 -- 172.21.15.7:0/1419036892 learned_addr learned my addr 172.21.15.7:0/1419036892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059cc00700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806aac0 0x7f059819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059d600700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f059806a1a0 0x7f059819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 -- 172.21.15.7:0/1419036892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f059806a1a0 msgr2=0x7f059819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f059806a1a0 0x7f059819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 -- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806aac0 msgr2=0x7f059819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806aac0 0x7f059819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059e000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819ff10 con 0x7f059810ef40 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.057+0000 7f059cc00700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806aac0 0x7f059819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f059e000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 0x7f059819f7d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f059000c3d0 tx=0x7f059000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900090d0 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a0200 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590009270 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900075c0 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.058+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a7a80 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.059+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0598062c20 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.061+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0590012070 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f0596000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05800a2cb0 0x7f05800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05900cfac0 con 0x7f059810ef40 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f059d600700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05800a2cb0 0x7f05800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f0596000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] conn(0x7f05800add90 0x7f05800b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05800b0900 con 0x7f05800add90 2024-06-13T06:53:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.063+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f05900cfe70 con 0x7f059810ef40 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.064+0000 7f059cc00700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] conn(0x7f05800add90 0x7f05800b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.064+0000 7f059d600700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05800a2cb0 0x7f05800a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f058c009c60 tx=0x7f058c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.064+0000 7f059cc00700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] conn(0x7f05800add90 0x7f05800b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.065+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== osd.4 v2:172.21.15.175:6808/576672935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05800b0900 con 0x7f05800add90 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.084+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 --> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f05980630a0 con 0x7f05800add90 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f0596000700 1 -- 172.21.15.7:0/1419036892 <== osd.4 v2:172.21.15.175:6808/576672935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f05980630a0 con 0x7f05800add90 2024-06-13T06:53:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] conn(0x7f05800add90 msgr2=0x7f05800b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] conn(0x7f05800add90 0x7f05800b0230 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05800a2cb0 msgr2=0x7f05800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05800a2cb0 0x7f05800a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f058c009c60 tx=0x7f058c009260 comp rx=0 tx=0).stop 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 msgr2=0x7f059819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f05a0000700 1 --2- 172.21.15.7:0/1419036892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f059810ef40 0x7f059819f7d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f059000c3d0 tx=0x7f059000c6e0 comp rx=0 tx=0).stop 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.085+0000 7f059e000700 1 -- 172.21.15.7:0/1419036892 reap_dead start 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.086+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 shutdown_connections 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.086+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 >> 172.21.15.7:0/1419036892 conn(0x7f0598078340 msgr2=0x7f0598110410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.086+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 shutdown_connections 2024-06-13T06:53:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:10.086+0000 7f05a0000700 1 -- 172.21.15.7:0/1419036892 wait complete. 2024-06-13T06:53:10.847 INFO:teuthology.orchestra.run.smithi007.stdout:77309411432 2024-06-13T06:53:10.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-13T06:53:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 -- 172.21.15.7:0/996625843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 msgr2=0x7fc9cc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 --2- 172.21.15.7:0/996625843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc1088e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009990 tx=0x7fc9c0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 -- 172.21.15.7:0/996625843 shutdown_connections 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 --2- 172.21.15.7:0/996625843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 --2- 172.21.15.7:0/996625843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc1088e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 --2- 172.21.15.7:0/996625843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9cc06a170 0x7fc9cc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 -- 172.21.15.7:0/996625843 >> 172.21.15.7:0/996625843 conn(0x7fc9cc078340 msgr2=0x7fc9cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 -- 172.21.15.7:0/996625843 shutdown_connections 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.192+0000 7fc9d4800700 1 -- 172.21.15.7:0/996625843 wait complete. 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 Processor -- start 2024-06-13T06:53:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 -- start start 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9cc06a170 0x7fc9cc101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc112760 con 0x7fc9cc06ab30 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1125e0 con 0x7fc9cc108500 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.193+0000 7fc9d4800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1128e0 con 0x7fc9cc06a170 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57396/0 (socket says 172.21.15.7:57396) 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 -- 172.21.15.7:0/3499189512 learned_addr learned my addr 172.21.15.7:0/3499189512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1e00700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9cc06a170 0x7fc9cc101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d2800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 -- 172.21.15.7:0/3499189512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9cc06a170 msgr2=0x7fc9cc101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9cc06a170 0x7fc9cc101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 -- 172.21.15.7:0/3499189512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 msgr2=0x7fc9cc101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d1400700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc1a7140 con 0x7fc9cc06ab30 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.194+0000 7fc9d2800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9cc108500 0x7fc9cc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9d1400700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc1019a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00d950 tx=0x7fc9bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc016070 con 0x7fc9cc06ab30 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc1a7430 con 0x7fc9cc06ab30 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc009e20 con 0x7fc9cc06ab30 2024-06-13T06:53:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00f620 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.195+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc1a79c0 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.196+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9cc062ba0 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.199+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc9bc019070 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9c6a00700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9b40a2d30 0x7fc9b40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc9bc0cf560 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9d1e00700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9b40a2d30 0x7fc9b40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9c6a00700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] conn(0x7fc9b40ade10 0x7fc9b40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9bc010e50 con 0x7fc9b40ade10 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.201+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc0cf9b0 con 0x7fc9cc06ab30 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.202+0000 7fc9d2800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] conn(0x7fc9b40ade10 0x7fc9b40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.202+0000 7fc9d1e00700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9b40a2d30 0x7fc9b40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00098e0 tx=0x7fc9c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.202+0000 7fc9d2800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] conn(0x7fc9b40ade10 0x7fc9b40b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.203+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== osd.9 v2:172.21.15.7:6826/1288548158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9bc010e50 con 0x7fc9b40ade10 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.222+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 --> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9cc062fa0 con 0x7fc9b40ade10 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.223+0000 7fc9c6a00700 1 -- 172.21.15.7:0/3499189512 <== osd.9 v2:172.21.15.7:6826/1288548158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9cc062fa0 con 0x7fc9b40ade10 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.223+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] conn(0x7fc9b40ade10 msgr2=0x7fc9b40b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.223+0000 7fc9d4800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] conn(0x7fc9b40ade10 0x7fc9b40b02b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.223+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9b40a2d30 msgr2=0x7fc9b40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.223+0000 7fc9d4800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9b40a2d30 0x7fc9b40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00098e0 tx=0x7fc9c0005960 comp rx=0 tx=0).stop 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 msgr2=0x7fc9cc1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 --2- 172.21.15.7:0/3499189512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9cc06ab30 0x7fc9cc1019a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00d950 tx=0x7fc9bc00dc60 comp rx=0 tx=0).stop 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d2800700 1 -- 172.21.15.7:0/3499189512 reap_dead start 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 shutdown_connections 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 >> 172.21.15.7:0/3499189512 conn(0x7fc9cc078340 msgr2=0x7fc9cc0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 shutdown_connections 2024-06-13T06:53:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:11.224+0000 7fc9d4800700 1 -- 172.21.15.7:0/3499189512 wait complete. 2024-06-13T06:53:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:11 smithi189 ceph-mon[36616]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:11.850 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:11 smithi007 ceph-mon[32077]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:11 smithi175 ceph-mon[36886]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:12.306 INFO:teuthology.orchestra.run.smithi007.stdout:201863462992 2024-06-13T06:53:12.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-13T06:53:12.366 INFO:teuthology.orchestra.run.smithi007.stdout:146028888156 2024-06-13T06:53:12.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-13T06:53:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:12.878 INFO:teuthology.orchestra.run.smithi007.stdout:292057776190 2024-06-13T06:53:12.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-13T06:53:13.575 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:13 smithi007 ceph-mon[32077]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:13 smithi189 ceph-mon[36616]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:13 smithi175 ceph-mon[36886]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:15.149 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:15.149 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:15.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:15 smithi007 ceph-mon[32077]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:15 smithi189 ceph-mon[36616]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:15 smithi175 ceph-mon[36886]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 -- 172.21.15.7:0/3328767756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260107840 msgr2=0x7f726010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 --2- 172.21.15.7:0/3328767756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260107840 0x7f726010f3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f725800b0d0 tx=0x7f725800b3e0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 -- 172.21.15.7:0/3328767756 shutdown_connections 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 --2- 172.21.15.7:0/3328767756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260107840 0x7f726010f3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 --2- 172.21.15.7:0/3328767756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7260102740 0x7f7260102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 --2- 172.21.15.7:0/3328767756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72601030f0 0x7f7260107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 -- 172.21.15.7:0/3328767756 >> 172.21.15.7:0/3328767756 conn(0x7f72600795f0 msgr2=0x7f7260079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 -- 172.21.15.7:0/3328767756 shutdown_connections 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.408+0000 7f7266e00700 1 -- 172.21.15.7:0/3328767756 wait complete. 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.409+0000 7f7266e00700 1 Processor -- start 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.409+0000 7f7266e00700 1 -- start start 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72601030f0 0x7f726019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7260107840 0x7f72601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72601127b0 con 0x7f7260102740 2024-06-13T06:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7260112630 con 0x7f72601030f0 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7266e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7260112930 con 0x7f7260107840 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f725fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f725fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57400/0 (socket says 172.21.15.7:57400) 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f725fe00700 1 -- 172.21.15.7:0/1064251021 learned_addr learned my addr 172.21.15.7:0/1064251021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7257400700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72601030f0 0x7f726019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.410+0000 7f7264e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7260107840 0x7f72601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 -- 172.21.15.7:0/1064251021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7260107840 msgr2=0x7f72601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7260107840 0x7f72601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 -- 172.21.15.7:0/1064251021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72601030f0 msgr2=0x7f726019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72601030f0 0x7f726019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 -- 172.21.15.7:0/1064251021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7260199150 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stdout:55834574959 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725fe00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f725000eb30 tx=0x7f725000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000cbd0 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7260199440 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7250004500 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250010430 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.411+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601a56b0 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.413+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7260069180 con 0x7f7260102740 2024-06-13T06:53:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.415+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7250003f00 con 0x7f7260102740 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.417+0000 7f725d600700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 0x7f72400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.417+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f72500cf910 con 0x7f7260102740 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.418+0000 7f7257400700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 0x7f72400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.418+0000 7f7257400700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 0x7f72400a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7260101f80 tx=0x7f7248009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.420+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f725001f340 con 0x7f7260102740 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.656+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f726002dca0 con 0x7f7260102740 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.657+0000 7f725d600700 1 -- 172.21.15.7:0/1064251021 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f725009c400 con 0x7f7260102740 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 msgr2=0x7f72400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 0x7f72400a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7260101f80 tx=0x7f7248009260 comp rx=0 tx=0).stop 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 msgr2=0x7f726019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f725000eb30 tx=0x7f725000ee40 comp rx=0 tx=0).stop 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 shutdown_connections 2024-06-13T06:53:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7260107840 0x7f72601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7260102740 0x7f726019e7e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f72400a2d30 0x7f72400a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 --2- 172.21.15.7:0/1064251021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72601030f0 0x7f726019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 >> 172.21.15.7:0/1064251021 conn(0x7f72600795f0 msgr2=0x7f7260100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 shutdown_connections 2024-06-13T06:53:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:16.660+0000 7f7266e00700 1 -- 172.21.15.7:0/1064251021 wait complete. 2024-06-13T06:53:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:17.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:17 smithi007 ceph-mon[32077]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:17.771 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:17 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1064251021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:53:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:17 smithi189 ceph-mon[36616]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:17 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1064251021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:53:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:17 smithi175 ceph-mon[36886]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:17 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1064251021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-13T06:53:19.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:19 smithi007 ceph-mon[32077]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:19.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:19 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1313275383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:53:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:19 smithi189 ceph-mon[36616]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:19 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1313275383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:53:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:19 smithi175 ceph-mon[36886]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:19 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1313275383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-13T06:53:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 -- 172.21.15.7:0/345319468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc102ea0 msgr2=0x7f19dc10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 --2- 172.21.15.7:0/345319468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc102ea0 0x7f19dc10f380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009f90 tx=0x7f19d400b330 comp rx=0 tx=0).stop 2024-06-13T06:53:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 -- 172.21.15.7:0/345319468 shutdown_connections 2024-06-13T06:53:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 --2- 172.21.15.7:0/345319468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc102ea0 0x7f19dc10f380 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 --2- 172.21.15.7:0/345319468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1085f0 0x7f19dc1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 --2- 172.21.15.7:0/345319468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc1024e0 0x7f19dc102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 -- 172.21.15.7:0/345319468 >> 172.21.15.7:0/345319468 conn(0x7f19dc079640 msgr2=0x7f19dc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 -- 172.21.15.7:0/345319468 shutdown_connections 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.314+0000 7f19e1a00700 1 -- 172.21.15.7:0/345319468 wait complete. 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.315+0000 7f19e1a00700 1 Processor -- start 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.315+0000 7f19e1a00700 1 -- start start 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1024e0 0x7f19dc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1085f0 0x7f19dc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc1127b0 con 0x7f19dc102ea0 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc112630 con 0x7f19dc1085f0 2024-06-13T06:53:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19e1a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc112930 con 0x7f19dc1024e0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19d2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19d2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57418/0 (socket says 172.21.15.7:57418) 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19d2000700 1 -- 172.21.15.7:0/1313275383 learned_addr learned my addr 172.21.15.7:0/1313275383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19daa00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1024e0 0x7f19dc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.316+0000 7f19db400700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1085f0 0x7f19dc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 -- 172.21.15.7:0/1313275383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1024e0 msgr2=0x7f19dc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1024e0 0x7f19dc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 -- 172.21.15.7:0/1313275383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1085f0 msgr2=0x7f19dc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1085f0 0x7f19dc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 -- 172.21.15.7:0/1313275383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc199210 con 0x7f19dc102ea0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d2000700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19c400ec10 tx=0x7f19c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c400cc90 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19c400ce30 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc199500 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4010690 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.317+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc0746c0 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.319+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19dc069180 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.321+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f19c4003680 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.322+0000 7f19d3e00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 0x7f19c00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.322+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f19c40cf860 con 0x7f19dc102ea0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.323+0000 7f19daa00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 0x7f19c00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.324+0000 7f19daa00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 0x7f19c00a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19dc108e70 tx=0x7f19cc012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:20.299 INFO:teuthology.orchestra.run.smithi007.stdout:223338299468 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.327+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f19c409c2c0 con 0x7f19dc102ea0 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.582+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f19dc199fc0 con 0x7f19dc102ea0 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.583+0000 7f19d3e00700 1 -- 172.21.15.7:0/1313275383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f19dc199fc0 con 0x7f19dc102ea0 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 msgr2=0x7f19c00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 0x7f19c00a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19dc108e70 tx=0x7f19cc012040 comp rx=0 tx=0).stop 2024-06-13T06:53:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 msgr2=0x7f19dc19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19c400ec10 tx=0x7f19c400ef20 comp rx=0 tx=0).stop 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 shutdown_connections 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19c00a2d30 0x7f19c00a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19dc1085f0 0x7f19dc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19dc1024e0 0x7f19dc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 --2- 172.21.15.7:0/1313275383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19dc102ea0 0x7f19dc19ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 >> 172.21.15.7:0/1313275383 conn(0x7f19dc079640 msgr2=0x7f19dc100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.586+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 shutdown_connections 2024-06-13T06:53:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:18.587+0000 7f19e1a00700 1 -- 172.21.15.7:0/1313275383 wait complete. 2024-06-13T06:53:21.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:21 smithi007 ceph-mon[32077]: pgmap v428: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:21 smithi189 ceph-mon[36616]: pgmap v428: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:21 smithi175 ceph-mon[36886]: pgmap v428: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:22.292 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299464 got 223338299468 for osd.7 2024-06-13T06:53:22.293 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.789+0000 7fa652600700 1 -- 172.21.15.7:0/2952725439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102540 msgr2=0x7fa64c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.789+0000 7fa652600700 1 --2- 172.21.15.7:0/2952725439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102540 0x7fa64c1029a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa638009990 tx=0x7fa638009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 -- 172.21.15.7:0/2952725439 shutdown_connections 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 --2- 172.21.15.7:0/2952725439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102ee0 0x7fa64c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 --2- 172.21.15.7:0/2952725439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c107450 0x7fa64c107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 --2- 172.21.15.7:0/2952725439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102540 0x7fa64c1029a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 -- 172.21.15.7:0/2952725439 >> 172.21.15.7:0/2952725439 conn(0x7fa64c0795f0 msgr2=0x7fa64c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 -- 172.21.15.7:0/2952725439 shutdown_connections 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.790+0000 7fa652600700 1 -- 172.21.15.7:0/2952725439 wait complete. 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 Processor -- start 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 -- start start 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 0x7fa64c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102ee0 0x7fa64c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1127d0 con 0x7fa64c107450 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c112650 con 0x7fa64c102ee0 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.791+0000 7fa652600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c112950 con 0x7fa64c102540 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 0x7fa64c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40648/0 (socket says 172.21.15.7:40648) 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64be00700 1 -- 172.21.15.7:0/2433126265 learned_addr learned my addr 172.21.15.7:0/2433126265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64aa00700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102ee0 0x7fa64c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 -- 172.21.15.7:0/2433126265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102ee0 msgr2=0x7fa64c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102ee0 0x7fa64c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 -- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 msgr2=0x7fa64c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64b400700 1 -- 172.21.15.7:0/2433126265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c19fec0 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.792+0000 7fa64be00700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa64b400700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 0x7fa64c19aeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9c0 tx=0x7fa63c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c004220 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63c0043c0 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c1a01b0 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003d10 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.793+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a7a30 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.794+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c069180 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.796+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa63c00c7c0 con 0x7fa64c102540 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.797+0000 7fa643e00700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 0x7fa6300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.798+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa63c0cf710 con 0x7fa64c102540 2024-06-13T06:53:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.798+0000 7fa64aa00700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 0x7fa6300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.799+0000 7fa64aa00700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 0x7fa6300a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098e0 tx=0x7fa638005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:21.802+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa63c09c170 con 0x7fa64c102540 2024-06-13T06:53:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.034+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa64c19c210 con 0x7fa64c102540 2024-06-13T06:53:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.035+0000 7fa643e00700 1 -- 172.21.15.7:0/2433126265 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa63c09ba70 con 0x7fa64c102540 2024-06-13T06:53:22.556 INFO:teuthology.orchestra.run.smithi007.stdout:77309411435 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 msgr2=0x7fa6300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 0x7fa6300a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098e0 tx=0x7fa638005960 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 msgr2=0x7fa64c19aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 0x7fa64c19aeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9c0 tx=0x7fa63c00ecd0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 shutdown_connections 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c107450 0x7fa64c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102540 0x7fa64c19aeb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2cb0 0x7fa6300a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 --2- 172.21.15.7:0/2433126265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c102ee0 0x7fa64c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 >> 172.21.15.7:0/2433126265 conn(0x7fa64c0795f0 msgr2=0x7fa64c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 shutdown_connections 2024-06-13T06:53:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:22.038+0000 7fa652600700 1 -- 172.21.15.7:0/2433126265 wait complete. 2024-06-13T06:53:22.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:22 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2433126265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:53:22.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:22 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:22 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2433126265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:53:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:22 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:22.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:22 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2433126265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-13T06:53:22.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:22 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:53:23.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:23.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:23.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:23.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:23.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:23.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: pgmap v429: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:53:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:53:23.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574959 for osd.1 2024-06-13T06:53:23.948 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:24.589 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:24.594 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:24.703 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:24 smithi007 ceph-mon[32077]: pgmap v430: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:24.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:24 smithi189 ceph-mon[36616]: pgmap v430: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:24 smithi175 ceph-mon[36886]: pgmap v430: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:25.124 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411432 got 77309411435 for osd.2 2024-06-13T06:53:25.124 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:26.960 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:26 smithi007 ceph-mon[32077]: pgmap v431: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:26 smithi189 ceph-mon[36616]: pgmap v431: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:27.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:26 smithi175 ceph-mon[36886]: pgmap v431: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:28.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:28 smithi007 ceph-mon[32077]: pgmap v432: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:28 smithi189 ceph-mon[36616]: pgmap v432: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:28 smithi175 ceph-mon[36886]: pgmap v432: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:31.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:30 smithi007 ceph-mon[32077]: pgmap v433: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:31.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3769542929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:53:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:30 smithi189 ceph-mon[36616]: pgmap v433: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:31.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3769542929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:53:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:30 smithi175 ceph-mon[36886]: pgmap v433: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:31.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3769542929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 -- 172.21.15.7:0/1376985003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 msgr2=0x7fa64c107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 --2- 172.21.15.7:0/1376985003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c107020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6380099c0 tx=0x7fa638009cd0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 -- 172.21.15.7:0/1376985003 shutdown_connections 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 --2- 172.21.15.7:0/1376985003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 --2- 172.21.15.7:0/1376985003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102620 0x7fa64c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 --2- 172.21.15.7:0/1376985003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c107020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 -- 172.21.15.7:0/1376985003 >> 172.21.15.7:0/1376985003 conn(0x7fa64c0fdf10 msgr2=0x7fa64c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 -- 172.21.15.7:0/1376985003 shutdown_connections 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.653+0000 7fa651400700 1 -- 172.21.15.7:0/1376985003 wait complete. 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.654+0000 7fa651400700 1 Processor -- start 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 -- start start 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102620 0x7fa64c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c112730 con 0x7fa64c102620 2024-06-13T06:53:31.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1125b0 con 0x7fa64c107700 2024-06-13T06:53:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa651400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1128b0 con 0x7fa64c102fd0 2024-06-13T06:53:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.655+0000 7fa64b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57238/0 (socket says 172.21.15.7:57238) 2024-06-13T06:53:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 -- 172.21.15.7:0/2020273327 learned_addr learned my addr 172.21.15.7:0/2020273327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa650a00700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 -- 172.21.15.7:0/2020273327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 msgr2=0x7fa64c10e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 -- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102620 msgr2=0x7fa64c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102620 0x7fa64c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 -- 172.21.15.7:0/2020273327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c10ecd0 con 0x7fa64c102fd0 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa650a00700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.656+0000 7fa64b400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa638005f50 tx=0x7fa638003b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.657+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638003680 con 0x7fa64c102fd0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.657+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c10ef60 con 0x7fa64c102fd0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.657+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c19b3e0 con 0x7fa64c102fd0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.657+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa638003e50 con 0x7fa64c102fd0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.657+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63800fe50 con 0x7fa64c102fd0 2024-06-13T06:53:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.658+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c103b00 con 0x7fa64c102fd0 2024-06-13T06:53:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.661+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa63800f580 con 0x7fa64c102fd0 2024-06-13T06:53:31.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.663+0000 7fa648c00700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 0x7fa6300a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.663+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa6380d0220 con 0x7fa64c102fd0 2024-06-13T06:53:31.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.663+0000 7fa64be00700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 0x7fa6300a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.664+0000 7fa64be00700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 0x7fa6300a54a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005950 tx=0x7fa63c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:31.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.664+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa638013070 con 0x7fa64c102fd0 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stdout:146028888160 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 -- 172.21.15.7:0/52744284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890107700 msgr2=0x7f989010f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 --2- 172.21.15.7:0/52744284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890107700 0x7f989010f2d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9888009fc0 tx=0x7f988800b330 comp rx=0 tx=0).stop 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 -- 172.21.15.7:0/52744284 shutdown_connections 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 --2- 172.21.15.7:0/52744284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890107700 0x7f989010f2d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 --2- 172.21.15.7:0/52744284 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 --2- 172.21.15.7:0/52744284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890102fd0 0x7f9890107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 -- 172.21.15.7:0/52744284 >> 172.21.15.7:0/52744284 conn(0x7f98900fdf10 msgr2=0x7f9890100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 -- 172.21.15.7:0/52744284 shutdown_connections 2024-06-13T06:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.450+0000 7f9895e00700 1 -- 172.21.15.7:0/52744284 wait complete. 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.451+0000 7f9895e00700 1 Processor -- start 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.451+0000 7f9895e00700 1 -- start start 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890102fd0 0x7f9890074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 0x7f9890075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9890112760 con 0x7f9890102fd0 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98901125e0 con 0x7f9890102620 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98901128e0 con 0x7f9890107700 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40562/0 (socket says 172.21.15.7:40562) 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 -- 172.21.15.7:0/4091640195 learned_addr learned my addr 172.21.15.7:0/4091640195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f988fe00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890102fd0 0x7f9890074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895400700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 0x7f9890075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 -- 172.21.15.7:0/4091640195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 msgr2=0x7f9890075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 0x7f9890075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 -- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890102fd0 msgr2=0x7f9890074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890102fd0 0x7f9890074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9894a00700 1 -- 172.21.15.7:0/4091640195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a7080 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.452+0000 7f9895400700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 0x7f9890075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f9894a00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f988400ec10 tx=0x7f988400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884004220 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988400cd50 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884003d60 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901988f0 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.453+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9890198de0 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.454+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9890069180 con 0x7f9890102620 2024-06-13T06:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.457+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f988400c870 con 0x7f9890102620 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.458+0000 7f988d600700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 0x7f98780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.459+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f98840d7f50 con 0x7f9890102620 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.459+0000 7f988fe00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 0x7f98780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.460+0000 7f988fe00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 0x7f98780a5160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9890108c30 tx=0x7f987c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.461+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f988402d020 con 0x7f9890102620 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.733+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f989002dca0 con 0x7f9890102620 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.734+0000 7f988d600700 1 -- 172.21.15.7:0/4091640195 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98840a4b10 con 0x7f9890102620 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.736+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 msgr2=0x7f98780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.736+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 0x7f98780a5160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9890108c30 tx=0x7f987c009260 comp rx=0 tx=0).stop 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.736+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 msgr2=0x7f9890074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.736+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f988400ec10 tx=0x7f988400ef20 comp rx=0 tx=0).stop 2024-06-13T06:53:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 shutdown_connections 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9890107700 0x7f9890075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f98780a2ca0 0x7f98780a5160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9890102620 0x7f9890074790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 --2- 172.21.15.7:0/4091640195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9890102fd0 0x7f9890074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 >> 172.21.15.7:0/4091640195 conn(0x7f98900fdf10 msgr2=0x7f98900fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 shutdown_connections 2024-06-13T06:53:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.737+0000 7f9895e00700 1 -- 172.21.15.7:0/4091640195 wait complete. 2024-06-13T06:53:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.909+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa64c197d40 con 0x7fa64c102fd0 2024-06-13T06:53:31.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:31 smithi007 ceph-mon[32077]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-13T06:53:31.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:31 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4091640195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:53:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.909+0000 7fa648c00700 1 -- 172.21.15.7:0/2020273327 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa63809ce70 con 0x7fa64c102fd0 2024-06-13T06:53:31.910 INFO:teuthology.orchestra.run.smithi007.stdout:257698037834 2024-06-13T06:53:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 msgr2=0x7fa6300a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 0x7fa6300a54a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005950 tx=0x7fa63c0058e0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 msgr2=0x7fa64c19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa638005f50 tx=0x7fa638003b40 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 shutdown_connections 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6300a2fe0 0x7fa6300a54a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64c107700 0x7fa64c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.912+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64c102620 0x7fa64c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.913+0000 7fa651400700 1 --2- 172.21.15.7:0/2020273327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa64c102fd0 0x7fa64c19fdc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.913+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 >> 172.21.15.7:0/2020273327 conn(0x7fa64c0fdf10 msgr2=0x7fa64c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.913+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 shutdown_connections 2024-06-13T06:53:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.913+0000 7fa651400700 1 -- 172.21.15.7:0/2020273327 wait complete. 2024-06-13T06:53:32.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:31 smithi189 ceph-mon[36616]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-13T06:53:32.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:31 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4091640195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:53:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:31 smithi175 ceph-mon[36886]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-13T06:53:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:31 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4091640195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 -- 172.21.15.7:0/3176383970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 msgr2=0x7ff06c107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 --2- 172.21.15.7:0/3176383970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c107000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff058009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 -- 172.21.15.7:0/3176383970 shutdown_connections 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 --2- 172.21.15.7:0/3176383970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 --2- 172.21.15.7:0/3176383970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 --2- 172.21.15.7:0/3176383970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c107000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 -- 172.21.15.7:0/3176383970 >> 172.21.15.7:0/3176383970 conn(0x7ff06c0fdeb0 msgr2=0x7ff06c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 -- 172.21.15.7:0/3176383970 shutdown_connections 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.258+0000 7ff070a00700 1 -- 172.21.15.7:0/3176383970 wait complete. 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.259+0000 7ff070a00700 1 Processor -- start 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.259+0000 7ff070a00700 1 -- start start 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1126a0 con 0x7ff06c102600 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c112520 con 0x7ff06c102fb0 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff070a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c112820 con 0x7ff06c1076e0 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff06a800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff069e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff06a800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40668/0 (socket says 172.21.15.7:40668) 2024-06-13T06:53:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff06a800700 1 -- 172.21.15.7:0/3769542929 learned_addr learned my addr 172.21.15.7:0/3769542929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.260+0000 7ff06b200700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 -- 172.21.15.7:0/3769542929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 msgr2=0x7ff06c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 -- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 msgr2=0x7ff06c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 -- 172.21.15.7:0/3769542929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c199120 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff06b200700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff069e00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c19ed40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff058005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058026070 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff058021410 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1993b0 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.261+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058017650 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.262+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c1a56d0 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.262+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c069180 con 0x7ff06c102fb0 2024-06-13T06:53:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.265+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff058029070 con 0x7ff06c102fb0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.266+0000 7ff063400700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 0x7ff0500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.266+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff0580d03b0 con 0x7ff06c102fb0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.266+0000 7ff06a800700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 0x7ff0500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.267+0000 7ff06a800700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 0x7ff0500a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff06c199b00 tx=0x7ff05c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.268+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff058013070 con 0x7ff06c102fb0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.509+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff06c02dca0 con 0x7ff06c102fb0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.510+0000 7ff063400700 1 -- 172.21.15.7:0/3769542929 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff05809d020 con 0x7ff06c102fb0 2024-06-13T06:53:32.445 INFO:teuthology.orchestra.run.smithi007.stdout:313532612669 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 msgr2=0x7ff0500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 0x7ff0500a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff06c199b00 tx=0x7ff05c009260 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 msgr2=0x7ff06c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c19ed40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff058005610 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 shutdown_connections 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff06c1076e0 0x7ff06c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c102600 0x7ff06c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff0500a2cb0 0x7ff0500a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 --2- 172.21.15.7:0/3769542929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06c102fb0 0x7ff06c19ed40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.513+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 >> 172.21.15.7:0/3769542929 conn(0x7ff06c0fdeb0 msgr2=0x7ff06c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.514+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 shutdown_connections 2024-06-13T06:53:32.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:30.514+0000 7ff070a00700 1 -- 172.21.15.7:0/3769542929 wait complete. 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.503+0000 7f5a3f200700 1 -- 172.21.15.7:0/4068728179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 msgr2=0x7f5a40107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.503+0000 7f5a3f200700 1 --2- 172.21.15.7:0/4068728179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a40107090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009990 tx=0x7f5a2c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 -- 172.21.15.7:0/4068728179 shutdown_connections 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 --2- 172.21.15.7:0/4068728179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 0x7f5a4010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 --2- 172.21.15.7:0/4068728179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 --2- 172.21.15.7:0/4068728179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a40107090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 -- 172.21.15.7:0/4068728179 >> 172.21.15.7:0/4068728179 conn(0x7f5a400781d0 msgr2=0x7f5a400785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 -- 172.21.15.7:0/4068728179 shutdown_connections 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.504+0000 7f5a3f200700 1 -- 172.21.15.7:0/4068728179 wait complete. 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.505+0000 7f5a3f200700 1 Processor -- start 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.505+0000 7f5a3f200700 1 -- start start 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.505+0000 7f5a3f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.505+0000 7f5a3f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a4019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.505+0000 7f5a3f200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 0x7f5a4019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401126d0 con 0x7f5a4006ac60 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3f200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40112550 con 0x7f5a401077c0 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40112850 con 0x7f5a4006b610 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41662/0 (socket says 172.21.15.7:41662) 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 -- 172.21.15.7:0/3328689773 learned_addr learned my addr 172.21.15.7:0/3328689773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3d400700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a4019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3e800700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 0x7f5a4019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 -- 172.21.15.7:0/3328689773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 msgr2=0x7f5a4019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a4019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 -- 172.21.15.7:0/3328689773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 msgr2=0x7f5a4019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 0x7f5a4019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.506+0000 7f5a3de00700 1 -- 172.21.15.7:0/3328689773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019fd80 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a3de00700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000eb30 tx=0x7f5a3000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000cbd0 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a401a0070 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a78f0 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a30004500 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.507+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30010430 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.508+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a20005330 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.510+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5a30003680 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.512+0000 7f5a36a00700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 0x7f5a240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.512+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5a300cf5c0 con 0x7f5a4006ac60 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.512+0000 7f5a3d400700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 0x7f5a240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.513+0000 7f5a3d400700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 0x7f5a240a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006010 tx=0x7f5a2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.517+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a3009c180 con 0x7f5a4006ac60 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 -- 172.21.15.7:0/1741681351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 msgr2=0x7f7f14102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1741681351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14102910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009920 tx=0x7f7f00009c30 comp rx=0 tx=0).stop 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 -- 172.21.15.7:0/1741681351 shutdown_connections 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1741681351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f14102e50 0x7f7f1410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1741681351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1741681351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14102910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.576+0000 7f7f1be00700 1 -- 172.21.15.7:0/1741681351 >> 172.21.15.7:0/1741681351 conn(0x7f7f140fdf40 msgr2=0x7f7f14100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.577+0000 7f7f1be00700 1 -- 172.21.15.7:0/1741681351 shutdown_connections 2024-06-13T06:53:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.577+0000 7f7f1be00700 1 -- 172.21.15.7:0/1741681351 wait complete. 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.578+0000 7f7f1be00700 1 Processor -- start 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.578+0000 7f7f1be00700 1 -- start start 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.578+0000 7f7f1be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.578+0000 7f7f1be00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f14102e50 0x7f7f14074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.578+0000 7f7f1be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f1be00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f141127f0 con 0x7f7f14108540 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f1be00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14112670 con 0x7f7f14102490 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f1be00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14112970 con 0x7f7f14102e50 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40580/0 (socket says 172.21.15.7:40580) 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 -- 172.21.15.7:0/1873089175 learned_addr learned my addr 172.21.15.7:0/1873089175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19e00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 -- 172.21.15.7:0/1873089175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f14102e50 msgr2=0x7f7f14074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f14102e50 0x7f7f14074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 -- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 msgr2=0x7f7f14075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19400700 1 -- 172.21.15.7:0/1873089175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f141a7180 con 0x7f7f14102490 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.579+0000 7f7f19e00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stdout:111669149797 2024-06-13T06:53:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f19400700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e3c0 tx=0x7f7f0800e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080090d0 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141a7410 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f08009270 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08004800 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.580+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141a79a0 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.581+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f14069180 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.583+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7f0801b030 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.584+0000 7f7f12000700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 0x7f7efc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.585+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7f080cf1d0 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.585+0000 7f7f18a00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 0x7f7efc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.586+0000 7f7f18a00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 0x7f7efc0a5160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009680 tx=0x7f7f00009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.588+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f08022060 con 0x7f7f14102490 2024-06-13T06:53:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.836+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7f1402df00 con 0x7f7f14102490 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.837+0000 7f7f12000700 1 -- 172.21.15.7:0/1873089175 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f08010040 con 0x7f7f14102490 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 msgr2=0x7f7efc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 0x7f7efc0a5160 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009680 tx=0x7f7f00009f80 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 msgr2=0x7f7f14074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e3c0 tx=0x7f7f0800e780 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 shutdown_connections 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f14108540 0x7f7f14075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7efc0a2ca0 0x7f7efc0a5160 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f14102490 0x7f7f14074790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 --2- 172.21.15.7:0/1873089175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f14102e50 0x7f7f14074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 >> 172.21.15.7:0/1873089175 conn(0x7f7f140fdf40 msgr2=0x7f7f140ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 shutdown_connections 2024-06-13T06:53:32.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:31.840+0000 7f7f1be00700 1 -- 172.21.15.7:0/1873089175 wait complete. 2024-06-13T06:53:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.754+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5a20005620 con 0x7f5a4006ac60 2024-06-13T06:53:32.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.755+0000 7f5a36a00700 1 -- 172.21.15.7:0/3328689773 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a3009ba80 con 0x7f5a4006ac60 2024-06-13T06:53:32.755 INFO:teuthology.orchestra.run.smithi007.stdout:201863462997 2024-06-13T06:53:32.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 msgr2=0x7f5a240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 0x7f5a240a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006010 tx=0x7f5a2c005960 comp rx=0 tx=0).stop 2024-06-13T06:53:32.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 msgr2=0x7f5a4019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:32.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000eb30 tx=0x7f5a3000ee40 comp rx=0 tx=0).stop 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 shutdown_connections 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a240a2ca0 0x7f5a240a5160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a401077c0 0x7f5a4019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4006ac60 0x7f5a4019ad70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 --2- 172.21.15.7:0/3328689773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4006b610 0x7f5a4019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 >> 172.21.15.7:0/3328689773 conn(0x7f5a400781d0 msgr2=0x7f5a4010d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 shutdown_connections 2024-06-13T06:53:32.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:32.758+0000 7f5a3f200700 1 -- 172.21.15.7:0/3328689773 wait complete. 2024-06-13T06:53:33.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:32 smithi007 ceph-mon[32077]: pgmap v434: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T06:53:33.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1873089175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:53:33.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2020273327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:53:33.040 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3328689773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:53:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:32 smithi189 ceph-mon[36616]: pgmap v434: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T06:53:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1873089175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:53:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2020273327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:53:33.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3328689773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:53:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:32 smithi175 ceph-mon[36886]: pgmap v434: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-13T06:53:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1873089175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-13T06:53:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2020273327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-13T06:53:33.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3328689773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-13T06:53:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 -- 172.21.15.7:0/155705565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac102ee0 msgr2=0x7f30ac10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 --2- 172.21.15.7:0/155705565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac102ee0 0x7f30ac10f3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b0a0 tx=0x7f30a400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 -- 172.21.15.7:0/155705565 shutdown_connections 2024-06-13T06:53:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 --2- 172.21.15.7:0/155705565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac102ee0 0x7f30ac10f3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 --2- 172.21.15.7:0/155705565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac107450 0x7f30ac107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 --2- 172.21.15.7:0/155705565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.001+0000 7f30b3200700 1 -- 172.21.15.7:0/155705565 >> 172.21.15.7:0/155705565 conn(0x7f30ac0795f0 msgr2=0x7f30ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.002+0000 7f30b3200700 1 -- 172.21.15.7:0/155705565 shutdown_connections 2024-06-13T06:53:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.002+0000 7f30b3200700 1 -- 172.21.15.7:0/155705565 wait complete. 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 Processor -- start 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 -- start start 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 0x7f30ac19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 0x7f30ac19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac112830 con 0x7f30ac102ee0 2024-06-13T06:53:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1126b0 con 0x7f30ac107450 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.003+0000 7f30b3200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1129b0 con 0x7f30ac102540 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30ab400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 0x7f30ac19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57286/0 (socket says 172.21.15.7:57286) 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 -- 172.21.15.7:0/1737178528 learned_addr learned my addr 172.21.15.7:0/1737178528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30b1200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 0x7f30ac19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 -- 172.21.15.7:0/1737178528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 msgr2=0x7f30ac19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 0x7f30ac19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 -- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 msgr2=0x7f30ac19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 0x7f30ac19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30abe00700 1 -- 172.21.15.7:0/1737178528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac19fe90 con 0x7f30ac102540 2024-06-13T06:53:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.004+0000 7f30ab400700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 0x7f30ac19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30b1200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 0x7f30ac19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30abe00700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e970 tx=0x7f309c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004220 con 0x7f30ac102540 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac1a0180 con 0x7f30ac102540 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309c0043c0 con 0x7f30ac102540 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c003dc0 con 0x7f30ac102540 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.005+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a7a00 con 0x7f30ac102540 2024-06-13T06:53:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.007+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ac069180 con 0x7f30ac102540 2024-06-13T06:53:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.010+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f309c00c7e0 con 0x7f30ac102540 2024-06-13T06:53:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.012+0000 7f30a8c00700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 0x7f30900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.012+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f309c0cf4a0 con 0x7f30ac102540 2024-06-13T06:53:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.012+0000 7f30ab400700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 0x7f30900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.013+0000 7f30ab400700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 0x7f30900a5160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30ac1023a0 tx=0x7f3098009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.016+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f309c09c060 con 0x7f30ac102540 2024-06-13T06:53:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.291+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f30ac19c1e0 con 0x7f30ac102540 2024-06-13T06:53:34.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.291+0000 7f30a8c00700 1 -- 172.21.15.7:0/1737178528 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f309c09b960 con 0x7f30ac102540 2024-06-13T06:53:34.292 INFO:teuthology.orchestra.run.smithi007.stdout:292057776195 2024-06-13T06:53:34.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.294+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 msgr2=0x7f30900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 0x7f30900a5160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30ac1023a0 tx=0x7f3098009260 comp rx=0 tx=0).stop 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 msgr2=0x7f30ac19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e970 tx=0x7f309c00ec80 comp rx=0 tx=0).stop 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 shutdown_connections 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f30900a2ca0 0x7f30900a5160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30ac107450 0x7f30ac19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac102540 0x7f30ac19ae80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 --2- 172.21.15.7:0/1737178528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30ac102ee0 0x7f30ac19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:34.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 >> 172.21.15.7:0/1737178528 conn(0x7f30ac0795f0 msgr2=0x7f30ac1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:34.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 shutdown_connections 2024-06-13T06:53:34.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.295+0000 7f30b3200700 1 -- 172.21.15.7:0/1737178528 wait complete. 2024-06-13T06:53:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:34 smithi189 ceph-mon[36616]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:34 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2512927304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:53:35.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:34 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1737178528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:53:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:34 smithi175 ceph-mon[36886]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:34 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2512927304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:53:35.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:34 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1737178528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:53:35.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:34 smithi007 ceph-mon[32077]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:35.134 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:34 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2512927304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-13T06:53:35.135 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:34 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1737178528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-13T06:53:35.323 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149792 got 111669149797 for osd.3 2024-06-13T06:53:35.323 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 -- 172.21.15.7:0/1901071220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 msgr2=0x7f08601157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 --2- 172.21.15.7:0/1901071220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601157b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f085800b0a0 tx=0x7f085800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 -- 172.21.15.7:0/1901071220 shutdown_connections 2024-06-13T06:53:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 --2- 172.21.15.7:0/1901071220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601157b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 --2- 172.21.15.7:0/1901071220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860074830 0x7f0860074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 --2- 172.21.15.7:0/1901071220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860075150 0x7f0860112e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 -- 172.21.15.7:0/1901071220 >> 172.21.15.7:0/1901071220 conn(0x7f08600fdea0 msgr2=0x7f08601002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 -- 172.21.15.7:0/1901071220 shutdown_connections 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.903+0000 7f0866c00700 1 -- 172.21.15.7:0/1901071220 wait complete. 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.904+0000 7f0866c00700 1 Processor -- start 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.904+0000 7f0866c00700 1 -- start start 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 0x7f086019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0860118cd0 con 0x7f08601133c0 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0860118b50 con 0x7f0860075150 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0860118e50 con 0x7f0860074830 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0864e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0864e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40644/0 (socket says 172.21.15.7:40644) 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0864e00700 1 -- 172.21.15.7:0/2512927304 learned_addr learned my addr 172.21.15.7:0/2512927304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0866200700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.905+0000 7f0865800700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 0x7f086019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 -- 172.21.15.7:0/2512927304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 msgr2=0x7f086019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 0x7f086019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 -- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 msgr2=0x7f08601a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 -- 172.21.15.7:0/2512927304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08601a4200 con 0x7f0860075150 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0865800700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 0x7f086019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0866200700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:53:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:347892351033 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0864e00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e9c0 tx=0x7f084c00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00cbd0 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08601a44f0 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c00cd70 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.906+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c010430 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.907+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08601abd70 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.908+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0860069180 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.909+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f084c004750 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.911+0000 7f085e000700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 0x7f08480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.911+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f084c0cf800 con 0x7f0860075150 2024-06-13T06:53:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.911+0000 7f0865800700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 0x7f08480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.912+0000 7f0865800700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 0x7f08480a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0854009cb0 tx=0x7f0854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:33.915+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f084c09c3c0 con 0x7f0860075150 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.160+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f08601a0550 con 0x7f0860075150 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.161+0000 7f085e000700 1 -- 172.21.15.7:0/2512927304 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f084c09bcc0 con 0x7f0860075150 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.164+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 msgr2=0x7f08480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.164+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 0x7f08480a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0854009cb0 tx=0x7f0854009260 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 msgr2=0x7f086019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e9c0 tx=0x7f084c00ed80 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 shutdown_connections 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08601133c0 0x7f08601a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0860074830 0x7f086019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f08480a2c20 0x7f08480a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 --2- 172.21.15.7:0/2512927304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0860075150 0x7f086019f780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 >> 172.21.15.7:0/2512927304 conn(0x7f08600fdea0 msgr2=0x7f08600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:35.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 shutdown_connections 2024-06-13T06:53:35.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.165+0000 7f0866c00700 1 -- 172.21.15.7:0/2512927304 wait complete. 2024-06-13T06:53:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:35 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1866148124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:53:36.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:35 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1866148124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:53:36.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:35 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1866148124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-13T06:53:36.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037828 got 257698037834 for osd.8 2024-06-13T06:53:36.232 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:36.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888156 got 146028888160 for osd.4 2024-06-13T06:53:36.608 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:36.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 -- 172.21.15.7:0/3071608790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 msgr2=0x7f4e34111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/3071608790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e34111560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b0a0 tx=0x7f4e2c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 -- 172.21.15.7:0/3071608790 shutdown_connections 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/3071608790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e34111560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/3071608790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 0x7f4e3410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/3071608790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e34104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 -- 172.21.15.7:0/3071608790 >> 172.21.15.7:0/3071608790 conn(0x7f4e341001a0 msgr2=0x7f4e341025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 -- 172.21.15.7:0/3071608790 shutdown_connections 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.886+0000 7f4e3bc00700 1 -- 172.21.15.7:0/3071608790 wait complete. 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.887+0000 7f4e3bc00700 1 Processor -- start 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 -- start start 2024-06-13T06:53:36.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e3419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 0x7f4e3419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34114a30 con 0x7f4e3410a690 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e341148b0 con 0x7f4e34104690 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e3bc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34114bb0 con 0x7f4e34105050 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e39200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e33e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e3419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e39200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40684/0 (socket says 172.21.15.7:40684) 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e39200700 1 -- 172.21.15.7:0/1866148124 learned_addr learned my addr 172.21.15.7:0/1866148124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.888+0000 7f4e39c00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 0x7f4e3419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 -- 172.21.15.7:0/1866148124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 msgr2=0x7f4e3419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 -- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 msgr2=0x7f4e3419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 0x7f4e3419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 -- 172.21.15.7:0/1866148124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3419fe40 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e39200700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e33e00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e3419b3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000e9f0 tx=0x7f4e2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004220 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e20004d10 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.889+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20005020 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.890+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e341a0130 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.890+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341a79b0 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.891+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e34069180 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.893+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4e20004750 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.894+0000 7f4e31600700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 0x7f4e1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.894+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4e200cf2c0 con 0x7f4e34105050 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.894+0000 7f4e39200700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 0x7f4e1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.895+0000 7f4e39200700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 0x7f4e1c0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e28011550 tx=0x7f4e28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:34.898+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e2009be80 con 0x7f4e34105050 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.133+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4e3419c190 con 0x7f4e34105050 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.134+0000 7f4e31600700 1 -- 172.21.15.7:0/1866148124 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e200043c0 con 0x7f4e34105050 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stdout:34359738486 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.137+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 msgr2=0x7f4e1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.137+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 0x7f4e1c0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4e28011550 tx=0x7f4e28009260 comp rx=0 tx=0).stop 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 msgr2=0x7f4e3419b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e3419b3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000e9f0 tx=0x7f4e2000ed00 comp rx=0 tx=0).stop 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 shutdown_connections 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3410a690 0x7f4e3419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4e1c0a2c20 0x7f4e1c0a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e34104690 0x7f4e3419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 --2- 172.21.15.7:0/1866148124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e34105050 0x7f4e3419b3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 >> 172.21.15.7:0/1866148124 conn(0x7f4e341001a0 msgr2=0x7f4e341006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 shutdown_connections 2024-06-13T06:53:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.138+0000 7f4e3bc00700 1 -- 172.21.15.7:0/1866148124 wait complete. 2024-06-13T06:53:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:36 smithi189 ceph-mon[36616]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:37.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:36 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2764082863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:53:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:36 smithi175 ceph-mon[36886]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:36 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2764082863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:53:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:36 smithi007 ceph-mon[32077]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:36 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2764082863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-13T06:53:38.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462992 got 201863462997 for osd.6 2024-06-13T06:53:38.279 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:38.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738479 got 34359738486 for osd.0 2024-06-13T06:53:38.327 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.882+0000 7f420e000700 1 -- 172.21.15.7:0/3255955765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 msgr2=0x7f4208101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.882+0000 7f420e000700 1 --2- 172.21.15.7:0/3255955765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f4208101810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41f40098f0 tx=0x7f41f4009c00 comp rx=0 tx=0).stop 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 -- 172.21.15.7:0/3255955765 shutdown_connections 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 --2- 172.21.15.7:0/3255955765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 0x7f420810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 --2- 172.21.15.7:0/3255955765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 --2- 172.21.15.7:0/3255955765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f4208101810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 -- 172.21.15.7:0/3255955765 >> 172.21.15.7:0/3255955765 conn(0x7f4208078340 msgr2=0x7f4208078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 -- 172.21.15.7:0/3255955765 shutdown_connections 2024-06-13T06:53:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.883+0000 7f420e000700 1 -- 172.21.15.7:0/3255955765 wait complete. 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.884+0000 7f420e000700 1 Processor -- start 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 -- start start 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 0x7f420819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42081116c0 con 0x7f4208107400 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111540 con 0x7f4208101d50 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f420e000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4208111840 con 0x7f4208101390 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f4207e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.885+0000 7f4207400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4206a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 0x7f420819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57328/0 (socket says 172.21.15.7:57328) 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207400700 1 -- 172.21.15.7:0/2764082863 learned_addr learned my addr 172.21.15.7:0/2764082863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 -- 172.21.15.7:0/2764082863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 msgr2=0x7f420819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 -- 172.21.15.7:0/2764082863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 msgr2=0x7f420819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 0x7f420819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 -- 172.21.15.7:0/2764082863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4208198100 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207400700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.886+0000 7f4207e00700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081979a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f420000e580 tx=0x7f420000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.887+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200004db0 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.887+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42000090d0 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.887+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42081983f0 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.887+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200004700 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.887+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420806a170 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.888+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4208069180 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.890+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4200003a00 con 0x7f4208107400 2024-06-13T06:53:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.892+0000 7f41ffe00700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 0x7f41ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.893+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f42000d11f0 con 0x7f4208107400 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.893+0000 7f4207400700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 0x7f41ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.893+0000 7f4207400700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 0x7f41ec0a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4208198ae0 tx=0x7f41f8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:35.897+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f420009ddb0 con 0x7f4208107400 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.133+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f420806ab70 con 0x7f4208107400 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.134+0000 7f41ffe00700 1 -- 172.21.15.7:0/2764082863 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f420009d6b0 con 0x7f4208107400 2024-06-13T06:53:38.349 INFO:teuthology.orchestra.run.smithi007.stdout:167503724637 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 msgr2=0x7f41ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 0x7f41ec0a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4208198ae0 tx=0x7f41f8009310 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 msgr2=0x7f42081979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081979a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f420000e580 tx=0x7f420000e890 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 shutdown_connections 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107400 0x7f42081979a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4208101390 0x7f420819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41ec0a2ca0 0x7f41ec0a5160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 --2- 172.21.15.7:0/2764082863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4208101d50 0x7f420819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 >> 172.21.15.7:0/2764082863 conn(0x7f4208078340 msgr2=0x7f42080fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 shutdown_connections 2024-06-13T06:53:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:36.137+0000 7f420e000700 1 -- 172.21.15.7:0/2764082863 wait complete. 2024-06-13T06:53:39.008 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:38 smithi007 ceph-mon[32077]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:39.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:38 smithi189 ceph-mon[36616]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:38 smithi175 ceph-mon[36886]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-13T06:53:39.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724629 got 167503724637 for osd.5 2024-06-13T06:53:39.135 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:39.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612662 got 313532612669 for osd.10 2024-06-13T06:53:39.445 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:39.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351027 got 347892351033 for osd.11 2024-06-13T06:53:39.814 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:40.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776190 got 292057776195 for osd.9 2024-06-13T06:53:40.406 DEBUG:teuthology.parallel:result is None 2024-06-13T06:53:40.406 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-13T06:53:40.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:53:40.874 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:40 smithi189 ceph-mon[36616]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:53:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:40 smithi175 ceph-mon[36886]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:53:41.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:40 smithi007 ceph-mon[32077]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-13T06:53:42.946 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:42 smithi007 ceph-mon[32077]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:43.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:42 smithi189 ceph-mon[36616]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:43.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:42 smithi175 ceph-mon[36886]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-13T06:53:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 -- 172.21.15.7:0/3619062321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290109480 msgr2=0x7fa290115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 --2- 172.21.15.7:0/3619062321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290109480 0x7fa290115990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f90 tx=0x7fa28800b330 comp rx=0 tx=0).stop 2024-06-13T06:53:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 -- 172.21.15.7:0/3619062321 shutdown_connections 2024-06-13T06:53:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 --2- 172.21.15.7:0/3619062321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa290109480 0x7fa290115990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 --2- 172.21.15.7:0/3619062321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa29010eac0 0x7fa29010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 --2- 172.21.15.7:0/3619062321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa290108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 -- 172.21.15.7:0/3619062321 >> 172.21.15.7:0/3619062321 conn(0x7fa290075c60 msgr2=0x7fa2901072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 -- 172.21.15.7:0/3619062321 shutdown_connections 2024-06-13T06:53:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.231+0000 7fa298000700 1 -- 172.21.15.7:0/3619062321 wait complete. 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.232+0000 7fa298000700 1 Processor -- start 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 -- start start 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa29019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 0x7fa2901a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa290118ea0 con 0x7fa290109480 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa290118d20 con 0x7fa290108ac0 2024-06-13T06:53:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa298000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa290119020 con 0x7fa29010eac0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa294c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.233+0000 7fa294c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41180/0 (socket says 172.21.15.7:41180) 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa294c00700 1 -- 172.21.15.7:0/4125753239 learned_addr learned my addr 172.21.15.7:0/4125753239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 0x7fa2901a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa295600700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa29019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 -- 172.21.15.7:0/4125753239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 msgr2=0x7fa29019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa29019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 -- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 msgr2=0x7fa29019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 -- 172.21.15.7:0/4125753239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2901a41f0 con 0x7fa29010eac0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa294c00700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa295600700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa29019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.234+0000 7fa296000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 0x7fa2901a3ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f90 tx=0x7fa28800cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.235+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288021070 con 0x7fa29010eac0 2024-06-13T06:53:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.235+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288007ac0 con 0x7fa29010eac0 2024-06-13T06:53:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.235+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2901a4480 con 0x7fa29010eac0 2024-06-13T06:53:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.235+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288005040 con 0x7fa29010eac0 2024-06-13T06:53:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.235+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901abd60 con 0x7fa29010eac0 2024-06-13T06:53:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.236+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa290069180 con 0x7fa29010eac0 2024-06-13T06:53:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.238+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa288024070 con 0x7fa29010eac0 2024-06-13T06:53:43.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.239+0000 7fa28e000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 0x7fa2780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.240+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa28800f020 con 0x7fa29010eac0 2024-06-13T06:53:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.240+0000 7fa295600700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 0x7fa2780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.241+0000 7fa295600700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 0x7fa2780a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa284005fd0 tx=0x7fa284009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.242+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa28802b020 con 0x7fa29010eac0 2024-06-13T06:53:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.466+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2900628d0 con 0x7fa2780a2c20 2024-06-13T06:53:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.479+0000 7fa28e000700 1 -- 172.21.15.7:0/4125753239 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340763 (secure 0 0 0) 0x7fa2900628d0 con 0x7fa2780a2c20 2024-06-13T06:53:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:53:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.486+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 msgr2=0x7fa2780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.486+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 0x7fa2780a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa284005fd0 tx=0x7fa284009310 comp rx=0 tx=0).stop 2024-06-13T06:53:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.486+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 msgr2=0x7fa2901a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.486+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 0x7fa2901a3ab0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f90 tx=0x7fa28800cfa0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 shutdown_connections 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa29010eac0 0x7fa2901a3ab0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2780a2c20 0x7fa2780a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa290108ac0 0x7fa29019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 --2- 172.21.15.7:0/4125753239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290109480 0x7fa29019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 >> 172.21.15.7:0/4125753239 conn(0x7fa290075c60 msgr2=0x7fa290076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 shutdown_connections 2024-06-13T06:53:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:43.487+0000 7fa298000700 1 -- 172.21.15.7:0/4125753239 wait complete. 2024-06-13T06:53:43.490 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:53:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:43 smithi189 ceph-mon[36616]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:43 smithi175 ceph-mon[36886]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:44.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:43 smithi007 ceph-mon[32077]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:44.294 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-06-13T06:53:41.699014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659050,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":844,"num_read_kb":543,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378916,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562908,"statfs":{"total":1151940427776,"available":1151552417792,"internally_reserved":0,"allocated":13438976,"data_stored":8600444,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005590"},"pg_stats":[{"pgid":"3.3f","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632049+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:53:29.632049+0000","last_peered":"2024-06-13T06:53:29.632049+0000","last_clean":"2024-06-13T06:53:29.632049+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:53:29.632049+0000","last_undegraded":"2024-06-13T06:53:29.632049+0000","last_fullsized":"2024-06-13T06:53:29.632049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626164+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:53:29.626164+0000","last_peered":"2024-06-13T06:53:29.626164+0000","last_clean":"2024-06-13T06:53:29.626164+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:53:29.626164+0000","last_undegraded":"2024-06-13T06:53:29.626164+0000","last_fullsized":"2024-06-13T06:53:29.626164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622717+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:53:29.622717+0000","last_peered":"2024-06-13T06:53:29.622717+0000","last_clean":"2024-06-13T06:53:29.622717+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:53:29.622717+0000","last_undegraded":"2024-06-13T06:53:29.622717+0000","last_fullsized":"2024-06-13T06:53:29.622717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622494+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:53:29.622494+0000","last_peered":"2024-06-13T06:53:29.622494+0000","last_clean":"2024-06-13T06:53:29.622494+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:53:29.622494+0000","last_undegraded":"2024-06-13T06:53:29.622494+0000","last_fullsized":"2024-06-13T06:53:29.622494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633368+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:53:29.633368+0000","last_peered":"2024-06-13T06:53:29.633368+0000","last_clean":"2024-06-13T06:53:29.633368+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:53:29.633368+0000","last_undegraded":"2024-06-13T06:53:29.633368+0000","last_fullsized":"2024-06-13T06:53:29.633368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633444+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:53:29.633444+0000","last_peered":"2024-06-13T06:53:29.633444+0000","last_clean":"2024-06-13T06:53:29.633444+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:53:29.633444+0000","last_undegraded":"2024-06-13T06:53:29.633444+0000","last_fullsized":"2024-06-13T06:53:29.633444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"98'5","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628727+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:53:29.628727+0000","last_peered":"2024-06-13T06:53:29.628727+0000","last_clean":"2024-06-13T06:53:29.628727+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:53:29.628727+0000","last_undegraded":"2024-06-13T06:53:29.628727+0000","last_fullsized":"2024-06-13T06:53:29.628727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637897+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:53:29.637897+0000","last_peered":"2024-06-13T06:53:29.637897+0000","last_clean":"2024-06-13T06:53:29.637897+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:53:29.637897+0000","last_undegraded":"2024-06-13T06:53:29.637897+0000","last_fullsized":"2024-06-13T06:53:29.637897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631145+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:53:29.631145+0000","last_peered":"2024-06-13T06:53:29.631145+0000","last_clean":"2024-06-13T06:53:29.631145+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:53:29.631145+0000","last_undegraded":"2024-06-13T06:53:29.631145+0000","last_fullsized":"2024-06-13T06:53:29.631145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"98'16","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627110+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:53:29.627110+0000","last_peered":"2024-06-13T06:53:29.627110+0000","last_clean":"2024-06-13T06:53:29.627110+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:53:29.627110+0000","last_undegraded":"2024-06-13T06:53:29.627110+0000","last_fullsized":"2024-06-13T06:53:29.627110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632199+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:53:29.632199+0000","last_peered":"2024-06-13T06:53:29.632199+0000","last_clean":"2024-06-13T06:53:29.632199+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:53:29.632199+0000","last_undegraded":"2024-06-13T06:53:29.632199+0000","last_fullsized":"2024-06-13T06:53:29.632199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"98'7","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635014+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:53:29.635014+0000","last_peered":"2024-06-13T06:53:29.635014+0000","last_clean":"2024-06-13T06:53:29.635014+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:53:29.635014+0000","last_undegraded":"2024-06-13T06:53:29.635014+0000","last_fullsized":"2024-06-13T06:53:29.635014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631329+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:53:29.631329+0000","last_peered":"2024-06-13T06:53:29.631329+0000","last_clean":"2024-06-13T06:53:29.631329+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:53:29.631329+0000","last_undegraded":"2024-06-13T06:53:29.631329+0000","last_fullsized":"2024-06-13T06:53:29.631329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628371+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:53:29.628371+0000","last_peered":"2024-06-13T06:53:29.628371+0000","last_clean":"2024-06-13T06:53:29.628371+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:53:29.628371+0000","last_undegraded":"2024-06-13T06:53:29.628371+0000","last_fullsized":"2024-06-13T06:53:29.628371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622830+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:53:29.622830+0000","last_peered":"2024-06-13T06:53:29.622830+0000","last_clean":"2024-06-13T06:53:29.622830+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:53:29.622830+0000","last_undegraded":"2024-06-13T06:53:29.622830+0000","last_fullsized":"2024-06-13T06:53:29.622830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631629+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:53:29.631629+0000","last_peered":"2024-06-13T06:53:29.631629+0000","last_clean":"2024-06-13T06:53:29.631629+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:53:29.631629+0000","last_undegraded":"2024-06-13T06:53:29.631629+0000","last_fullsized":"2024-06-13T06:53:29.631629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632097+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:53:29.632097+0000","last_peered":"2024-06-13T06:53:29.632097+0000","last_clean":"2024-06-13T06:53:29.632097+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:53:29.632097+0000","last_undegraded":"2024-06-13T06:53:29.632097+0000","last_fullsized":"2024-06-13T06:53:29.632097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634479+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:53:29.634479+0000","last_peered":"2024-06-13T06:53:29.634479+0000","last_clean":"2024-06-13T06:53:29.634479+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:53:29.634479+0000","last_undegraded":"2024-06-13T06:53:29.634479+0000","last_fullsized":"2024-06-13T06:53:29.634479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629955+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:53:29.629955+0000","last_peered":"2024-06-13T06:53:29.629955+0000","last_clean":"2024-06-13T06:53:29.629955+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:53:29.629955+0000","last_undegraded":"2024-06-13T06:53:29.629955+0000","last_fullsized":"2024-06-13T06:53:29.629955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881746+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:51:00.881746+0000","last_peered":"2024-06-13T06:51:00.881746+0000","last_clean":"2024-06-13T06:51:00.881746+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:51:00.881746+0000","last_undegraded":"2024-06-13T06:51:00.881746+0000","last_fullsized":"2024-06-13T06:51:00.881746+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835279+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:51:45.835279+0000","last_peered":"2024-06-13T06:51:45.835279+0000","last_clean":"2024-06-13T06:51:45.835279+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:51:45.835279+0000","last_undegraded":"2024-06-13T06:51:45.835279+0000","last_fullsized":"2024-06-13T06:51:45.835279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635321+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:53:29.635321+0000","last_peered":"2024-06-13T06:53:29.635321+0000","last_clean":"2024-06-13T06:53:29.635321+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:53:29.635321+0000","last_undegraded":"2024-06-13T06:53:29.635321+0000","last_fullsized":"2024-06-13T06:53:29.635321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891212+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:51:00.891212+0000","last_peered":"2024-06-13T06:51:00.891212+0000","last_clean":"2024-06-13T06:51:00.891212+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:51:00.891212+0000","last_undegraded":"2024-06-13T06:51:00.891212+0000","last_fullsized":"2024-06-13T06:51:00.891212+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803500+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:51:41.803500+0000","last_peered":"2024-06-13T06:51:41.803500+0000","last_clean":"2024-06-13T06:51:41.803500+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:51:41.803500+0000","last_undegraded":"2024-06-13T06:51:41.803500+0000","last_fullsized":"2024-06-13T06:51:41.803500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635642+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:53:29.635642+0000","last_peered":"2024-06-13T06:53:29.635642+0000","last_clean":"2024-06-13T06:53:29.635642+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:53:29.635642+0000","last_undegraded":"2024-06-13T06:53:29.635642+0000","last_fullsized":"2024-06-13T06:53:29.635642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891648+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891648+0000","last_peered":"2024-06-13T06:51:00.891648+0000","last_clean":"2024-06-13T06:51:00.891648+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891648+0000","last_undegraded":"2024-06-13T06:51:00.891648+0000","last_fullsized":"2024-06-13T06:51:00.891648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799045+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:51:41.799045+0000","last_peered":"2024-06-13T06:51:41.799045+0000","last_clean":"2024-06-13T06:51:41.799045+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:51:41.799045+0000","last_undegraded":"2024-06-13T06:51:41.799045+0000","last_fullsized":"2024-06-13T06:51:41.799045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634597+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:53:29.634597+0000","last_peered":"2024-06-13T06:53:29.634597+0000","last_clean":"2024-06-13T06:53:29.634597+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:53:29.634597+0000","last_undegraded":"2024-06-13T06:53:29.634597+0000","last_fullsized":"2024-06-13T06:53:29.634597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891222+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891222+0000","last_peered":"2024-06-13T06:51:00.891222+0000","last_clean":"2024-06-13T06:51:00.891222+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891222+0000","last_undegraded":"2024-06-13T06:51:00.891222+0000","last_fullsized":"2024-06-13T06:51:00.891222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254033+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:51:01.254033+0000","last_peered":"2024-06-13T06:51:01.254033+0000","last_clean":"2024-06-13T06:51:01.254033+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:51:01.254033+0000","last_undegraded":"2024-06-13T06:51:01.254033+0000","last_fullsized":"2024-06-13T06:51:01.254033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631918+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:53:29.631918+0000","last_peered":"2024-06-13T06:53:29.631918+0000","last_clean":"2024-06-13T06:53:29.631918+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:53:29.631918+0000","last_undegraded":"2024-06-13T06:53:29.631918+0000","last_fullsized":"2024-06-13T06:53:29.631918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881987+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881987+0000","last_peered":"2024-06-13T06:51:00.881987+0000","last_clean":"2024-06-13T06:51:00.881987+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881987+0000","last_undegraded":"2024-06-13T06:51:00.881987+0000","last_fullsized":"2024-06-13T06:51:00.881987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":163,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:39.001168+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:53:39.001168+0000","last_peered":"2024-06-13T06:53:39.001168+0000","last_clean":"2024-06-13T06:53:39.001168+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:53:39.001168+0000","last_undegraded":"2024-06-13T06:53:39.001168+0000","last_fullsized":"2024-06-13T06:53:39.001168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617073+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:53:29.617073+0000","last_peered":"2024-06-13T06:53:29.617073+0000","last_clean":"2024-06-13T06:53:29.617073+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:53:29.617073+0000","last_undegraded":"2024-06-13T06:53:29.617073+0000","last_fullsized":"2024-06-13T06:53:29.617073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882139+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882139+0000","last_peered":"2024-06-13T06:51:00.882139+0000","last_clean":"2024-06-13T06:51:00.882139+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882139+0000","last_undegraded":"2024-06-13T06:51:00.882139+0000","last_fullsized":"2024-06-13T06:51:00.882139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.801075+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:51:41.801075+0000","last_peered":"2024-06-13T06:51:41.801075+0000","last_clean":"2024-06-13T06:51:41.801075+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:51:41.801075+0000","last_undegraded":"2024-06-13T06:51:41.801075+0000","last_fullsized":"2024-06-13T06:51:41.801075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640050+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:53:29.640050+0000","last_peered":"2024-06-13T06:53:29.640050+0000","last_clean":"2024-06-13T06:53:29.640050+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:53:29.640050+0000","last_undegraded":"2024-06-13T06:53:29.640050+0000","last_fullsized":"2024-06-13T06:53:29.640050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884182+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:51:00.884182+0000","last_peered":"2024-06-13T06:51:00.884182+0000","last_clean":"2024-06-13T06:51:00.884182+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:51:00.884182+0000","last_undegraded":"2024-06-13T06:51:00.884182+0000","last_fullsized":"2024-06-13T06:51:00.884182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.805782+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:51:41.805782+0000","last_peered":"2024-06-13T06:51:41.805782+0000","last_clean":"2024-06-13T06:51:41.805782+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:51:41.805782+0000","last_undegraded":"2024-06-13T06:51:41.805782+0000","last_fullsized":"2024-06-13T06:51:41.805782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637137+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:53:29.637137+0000","last_peered":"2024-06-13T06:53:29.637137+0000","last_clean":"2024-06-13T06:53:29.637137+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:53:29.637137+0000","last_undegraded":"2024-06-13T06:53:29.637137+0000","last_fullsized":"2024-06-13T06:53:29.637137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888882+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:51:00.888882+0000","last_peered":"2024-06-13T06:51:00.888882+0000","last_clean":"2024-06-13T06:51:00.888882+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:51:00.888882+0000","last_undegraded":"2024-06-13T06:51:00.888882+0000","last_fullsized":"2024-06-13T06:51:00.888882+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780108+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:51:40.780108+0000","last_peered":"2024-06-13T06:51:40.780108+0000","last_clean":"2024-06-13T06:51:40.780108+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:51:40.780108+0000","last_undegraded":"2024-06-13T06:51:40.780108+0000","last_fullsized":"2024-06-13T06:51:40.780108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639788+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:53:29.639788+0000","last_peered":"2024-06-13T06:53:29.639788+0000","last_clean":"2024-06-13T06:53:29.639788+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:53:29.639788+0000","last_undegraded":"2024-06-13T06:53:29.639788+0000","last_fullsized":"2024-06-13T06:53:29.639788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230296+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:51:01.230296+0000","last_peered":"2024-06-13T06:51:01.230296+0000","last_clean":"2024-06-13T06:51:01.230296+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:51:01.230296+0000","last_undegraded":"2024-06-13T06:51:01.230296+0000","last_fullsized":"2024-06-13T06:51:01.230296+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":159,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:39.000527+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:53:39.000527+0000","last_peered":"2024-06-13T06:53:39.000527+0000","last_clean":"2024-06-13T06:53:39.000527+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:53:39.000527+0000","last_undegraded":"2024-06-13T06:53:39.000527+0000","last_fullsized":"2024-06-13T06:53:39.000527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634415+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:53:29.634415+0000","last_peered":"2024-06-13T06:53:29.634415+0000","last_clean":"2024-06-13T06:53:29.634415+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:53:29.634415+0000","last_undegraded":"2024-06-13T06:53:29.634415+0000","last_fullsized":"2024-06-13T06:53:29.634415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888254+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:51:00.888254+0000","last_peered":"2024-06-13T06:51:00.888254+0000","last_clean":"2024-06-13T06:51:00.888254+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:51:00.888254+0000","last_undegraded":"2024-06-13T06:51:00.888254+0000","last_fullsized":"2024-06-13T06:51:00.888254+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779852+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:51:40.779852+0000","last_peered":"2024-06-13T06:51:40.779852+0000","last_clean":"2024-06-13T06:51:40.779852+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:51:40.779852+0000","last_undegraded":"2024-06-13T06:51:40.779852+0000","last_fullsized":"2024-06-13T06:51:40.779852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628538+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:53:29.628538+0000","last_peered":"2024-06-13T06:53:29.628538+0000","last_clean":"2024-06-13T06:53:29.628538+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:53:29.628538+0000","last_undegraded":"2024-06-13T06:53:29.628538+0000","last_fullsized":"2024-06-13T06:53:29.628538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891857+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:51:00.891857+0000","last_peered":"2024-06-13T06:51:00.891857+0000","last_clean":"2024-06-13T06:51:00.891857+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:51:00.891857+0000","last_undegraded":"2024-06-13T06:51:00.891857+0000","last_fullsized":"2024-06-13T06:51:00.891857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.833439+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:51:45.833439+0000","last_peered":"2024-06-13T06:51:45.833439+0000","last_clean":"2024-06-13T06:51:45.833439+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:51:45.833439+0000","last_undegraded":"2024-06-13T06:51:45.833439+0000","last_fullsized":"2024-06-13T06:51:45.833439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632252+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:53:29.632252+0000","last_peered":"2024-06-13T06:53:29.632252+0000","last_clean":"2024-06-13T06:53:29.632252+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:53:29.632252+0000","last_undegraded":"2024-06-13T06:53:29.632252+0000","last_fullsized":"2024-06-13T06:53:29.632252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891224+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:51:00.891224+0000","last_peered":"2024-06-13T06:51:00.891224+0000","last_clean":"2024-06-13T06:51:00.891224+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:51:00.891224+0000","last_undegraded":"2024-06-13T06:51:00.891224+0000","last_fullsized":"2024-06-13T06:51:00.891224+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'37","reported_seq":318,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:40.012211+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:53:40.012211+0000","last_peered":"2024-06-13T06:53:40.012211+0000","last_clean":"2024-06-13T06:53:40.012211+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:53:40.012211+0000","last_undegraded":"2024-06-13T06:53:40.012211+0000","last_fullsized":"2024-06-13T06:53:40.012211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64307,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.611823+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:53:29.611823+0000","last_peered":"2024-06-13T06:53:29.611823+0000","last_clean":"2024-06-13T06:53:29.611823+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:53:29.611823+0000","last_undegraded":"2024-06-13T06:53:29.611823+0000","last_fullsized":"2024-06-13T06:53:29.611823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253857+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:51:01.253857+0000","last_peered":"2024-06-13T06:51:01.253857+0000","last_clean":"2024-06-13T06:51:01.253857+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:51:01.253857+0000","last_undegraded":"2024-06-13T06:51:01.253857+0000","last_fullsized":"2024-06-13T06:51:01.253857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.830980+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:51:45.830980+0000","last_peered":"2024-06-13T06:51:45.830980+0000","last_clean":"2024-06-13T06:51:45.830980+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:51:45.830980+0000","last_undegraded":"2024-06-13T06:51:45.830980+0000","last_fullsized":"2024-06-13T06:51:45.830980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630741+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:53:29.630741+0000","last_peered":"2024-06-13T06:53:29.630741+0000","last_clean":"2024-06-13T06:53:29.630741+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:53:29.630741+0000","last_undegraded":"2024-06-13T06:53:29.630741+0000","last_fullsized":"2024-06-13T06:53:29.630741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883955+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:51:00.883955+0000","last_peered":"2024-06-13T06:51:00.883955+0000","last_clean":"2024-06-13T06:51:00.883955+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:51:00.883955+0000","last_undegraded":"2024-06-13T06:51:00.883955+0000","last_fullsized":"2024-06-13T06:51:00.883955+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781368+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:51:40.781368+0000","last_peered":"2024-06-13T06:51:40.781368+0000","last_clean":"2024-06-13T06:51:40.781368+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:51:40.781368+0000","last_undegraded":"2024-06-13T06:51:40.781368+0000","last_fullsized":"2024-06-13T06:51:40.781368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"98'15","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632168+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:53:29.632168+0000","last_peered":"2024-06-13T06:53:29.632168+0000","last_clean":"2024-06-13T06:53:29.632168+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:53:29.632168+0000","last_undegraded":"2024-06-13T06:53:29.632168+0000","last_fullsized":"2024-06-13T06:53:29.632168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892716+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:51:00.892716+0000","last_peered":"2024-06-13T06:51:00.892716+0000","last_clean":"2024-06-13T06:51:00.892716+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:51:00.892716+0000","last_undegraded":"2024-06-13T06:51:00.892716+0000","last_fullsized":"2024-06-13T06:51:00.892716+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":162,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:36.072369+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:53:36.072369+0000","last_peered":"2024-06-13T06:53:36.072369+0000","last_clean":"2024-06-13T06:53:36.072369+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:53:36.072369+0000","last_undegraded":"2024-06-13T06:53:36.072369+0000","last_fullsized":"2024-06-13T06:53:36.072369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633544+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:53:29.633544+0000","last_peered":"2024-06-13T06:53:29.633544+0000","last_clean":"2024-06-13T06:53:29.633544+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:53:29.633544+0000","last_undegraded":"2024-06-13T06:53:29.633544+0000","last_fullsized":"2024-06-13T06:53:29.633544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961104+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:51:00.961104+0000","last_peered":"2024-06-13T06:51:00.961104+0000","last_clean":"2024-06-13T06:51:00.961104+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:51:00.961104+0000","last_undegraded":"2024-06-13T06:51:00.961104+0000","last_fullsized":"2024-06-13T06:51:00.961104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":57,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.830961+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:51:45.830961+0000","last_peered":"2024-06-13T06:51:45.830961+0000","last_clean":"2024-06-13T06:51:45.830961+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:51:45.830961+0000","last_undegraded":"2024-06-13T06:51:45.830961+0000","last_fullsized":"2024-06-13T06:51:45.830961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622846+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:53:29.622846+0000","last_peered":"2024-06-13T06:53:29.622846+0000","last_clean":"2024-06-13T06:53:29.622846+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:53:29.622846+0000","last_undegraded":"2024-06-13T06:53:29.622846+0000","last_fullsized":"2024-06-13T06:53:29.622846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253488+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:51:01.253488+0000","last_peered":"2024-06-13T06:51:01.253488+0000","last_clean":"2024-06-13T06:51:01.253488+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:51:01.253488+0000","last_undegraded":"2024-06-13T06:51:01.253488+0000","last_fullsized":"2024-06-13T06:51:01.253488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831263+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:51:45.831263+0000","last_peered":"2024-06-13T06:51:45.831263+0000","last_clean":"2024-06-13T06:51:45.831263+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:51:45.831263+0000","last_undegraded":"2024-06-13T06:51:45.831263+0000","last_fullsized":"2024-06-13T06:51:45.831263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627776+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:53:29.627776+0000","last_peered":"2024-06-13T06:53:29.627776+0000","last_clean":"2024-06-13T06:53:29.627776+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:53:29.627776+0000","last_undegraded":"2024-06-13T06:53:29.627776+0000","last_fullsized":"2024-06-13T06:53:29.627776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254101+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:51:01.254101+0000","last_peered":"2024-06-13T06:51:01.254101+0000","last_clean":"2024-06-13T06:51:01.254101+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:51:01.254101+0000","last_undegraded":"2024-06-13T06:51:01.254101+0000","last_fullsized":"2024-06-13T06:51:01.254101+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.896197+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:51:00.896197+0000","last_peered":"2024-06-13T06:51:00.896197+0000","last_clean":"2024-06-13T06:51:00.896197+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:51:00.896197+0000","last_undegraded":"2024-06-13T06:51:00.896197+0000","last_fullsized":"2024-06-13T06:51:00.896197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636656+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:53:29.636656+0000","last_peered":"2024-06-13T06:53:29.636656+0000","last_clean":"2024-06-13T06:53:29.636656+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:53:29.636656+0000","last_undegraded":"2024-06-13T06:53:29.636656+0000","last_fullsized":"2024-06-13T06:53:29.636656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960360+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:51:00.960360+0000","last_peered":"2024-06-13T06:51:00.960360+0000","last_clean":"2024-06-13T06:51:00.960360+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:51:00.960360+0000","last_undegraded":"2024-06-13T06:51:00.960360+0000","last_fullsized":"2024-06-13T06:51:00.960360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799920+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:51:41.799920+0000","last_peered":"2024-06-13T06:51:41.799920+0000","last_clean":"2024-06-13T06:51:41.799920+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:51:41.799920+0000","last_undegraded":"2024-06-13T06:51:41.799920+0000","last_fullsized":"2024-06-13T06:51:41.799920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623582+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:53:29.623582+0000","last_peered":"2024-06-13T06:53:29.623582+0000","last_clean":"2024-06-13T06:53:29.623582+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:53:29.623582+0000","last_undegraded":"2024-06-13T06:53:29.623582+0000","last_fullsized":"2024-06-13T06:53:29.623582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961435+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:51:00.961435+0000","last_peered":"2024-06-13T06:51:00.961435+0000","last_clean":"2024-06-13T06:51:00.961435+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:51:00.961435+0000","last_undegraded":"2024-06-13T06:51:00.961435+0000","last_fullsized":"2024-06-13T06:51:00.961435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.621714+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:53:29.621714+0000","last_peered":"2024-06-13T06:53:29.621714+0000","last_clean":"2024-06-13T06:53:29.621714+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:53:29.621714+0000","last_undegraded":"2024-06-13T06:53:29.621714+0000","last_fullsized":"2024-06-13T06:53:29.621714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":162,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:40.081839+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:53:40.081839+0000","last_peered":"2024-06-13T06:53:40.081839+0000","last_clean":"2024-06-13T06:53:40.081839+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:53:40.081839+0000","last_undegraded":"2024-06-13T06:53:40.081839+0000","last_fullsized":"2024-06-13T06:53:40.081839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:58.774101+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:51:58.774101+0000","last_peered":"2024-06-13T06:51:58.774101+0000","last_clean":"2024-06-13T06:51:58.774101+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:51:58.774101+0000","last_undegraded":"2024-06-13T06:51:58.774101+0000","last_fullsized":"2024-06-13T06:51:58.774101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636617+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:53:29.636617+0000","last_peered":"2024-06-13T06:53:29.636617+0000","last_clean":"2024-06-13T06:53:29.636617+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:53:29.636617+0000","last_undegraded":"2024-06-13T06:53:29.636617+0000","last_fullsized":"2024-06-13T06:53:29.636617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831145+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:51:45.831145+0000","last_peered":"2024-06-13T06:51:45.831145+0000","last_clean":"2024-06-13T06:51:45.831145+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:51:45.831145+0000","last_undegraded":"2024-06-13T06:51:45.831145+0000","last_fullsized":"2024-06-13T06:51:45.831145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:11.024426+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:51:11.024426+0000","last_peered":"2024-06-13T06:51:11.024426+0000","last_clean":"2024-06-13T06:51:11.024426+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:51:11.024426+0000","last_undegraded":"2024-06-13T06:51:11.024426+0000","last_fullsized":"2024-06-13T06:51:11.024426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254476+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:51:01.254476+0000","last_peered":"2024-06-13T06:51:01.254476+0000","last_clean":"2024-06-13T06:51:01.254476+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:51:01.254476+0000","last_undegraded":"2024-06-13T06:51:01.254476+0000","last_fullsized":"2024-06-13T06:51:01.254476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623385+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:53:29.623385+0000","last_peered":"2024-06-13T06:53:29.623385+0000","last_clean":"2024-06-13T06:53:29.623385+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:53:29.623385+0000","last_undegraded":"2024-06-13T06:53:29.623385+0000","last_fullsized":"2024-06-13T06:53:29.623385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.798470+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:51:41.798470+0000","last_peered":"2024-06-13T06:51:41.798470+0000","last_clean":"2024-06-13T06:51:41.798470+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:51:41.798470+0000","last_undegraded":"2024-06-13T06:51:41.798470+0000","last_fullsized":"2024-06-13T06:51:41.798470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.258410+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:51:01.258410+0000","last_peered":"2024-06-13T06:51:01.258410+0000","last_clean":"2024-06-13T06:51:01.258410+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:51:01.258410+0000","last_undegraded":"2024-06-13T06:51:01.258410+0000","last_fullsized":"2024-06-13T06:51:01.258410+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831109+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:51:45.831109+0000","last_peered":"2024-06-13T06:51:45.831109+0000","last_clean":"2024-06-13T06:51:45.831109+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:51:45.831109+0000","last_undegraded":"2024-06-13T06:51:45.831109+0000","last_fullsized":"2024-06-13T06:51:45.831109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622400+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:53:29.622400+0000","last_peered":"2024-06-13T06:53:29.622400+0000","last_clean":"2024-06-13T06:53:29.622400+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:53:29.622400+0000","last_undegraded":"2024-06-13T06:53:29.622400+0000","last_fullsized":"2024-06-13T06:53:29.622400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230194+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:51:01.230194+0000","last_peered":"2024-06-13T06:51:01.230194+0000","last_clean":"2024-06-13T06:51:01.230194+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:51:01.230194+0000","last_undegraded":"2024-06-13T06:51:01.230194+0000","last_fullsized":"2024-06-13T06:51:01.230194+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'12","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:23.465258+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:52:23.465258+0000","last_peered":"2024-06-13T06:52:23.465258+0000","last_clean":"2024-06-13T06:52:23.465258+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:52:23.465258+0000","last_undegraded":"2024-06-13T06:52:23.465258+0000","last_fullsized":"2024-06-13T06:52:23.465258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631623+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:53:29.631623+0000","last_peered":"2024-06-13T06:53:29.631623+0000","last_clean":"2024-06-13T06:53:29.631623+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:53:29.631623+0000","last_undegraded":"2024-06-13T06:53:29.631623+0000","last_fullsized":"2024-06-13T06:53:29.631623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881608+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:51:00.881608+0000","last_peered":"2024-06-13T06:51:00.881608+0000","last_clean":"2024-06-13T06:51:00.881608+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:51:00.881608+0000","last_undegraded":"2024-06-13T06:51:00.881608+0000","last_fullsized":"2024-06-13T06:51:00.881608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781280+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:51:40.781280+0000","last_peered":"2024-06-13T06:51:40.781280+0000","last_clean":"2024-06-13T06:51:40.781280+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:51:40.781280+0000","last_undegraded":"2024-06-13T06:51:40.781280+0000","last_fullsized":"2024-06-13T06:51:40.781280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635239+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:53:29.635239+0000","last_peered":"2024-06-13T06:53:29.635239+0000","last_clean":"2024-06-13T06:53:29.635239+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:53:29.635239+0000","last_undegraded":"2024-06-13T06:53:29.635239+0000","last_fullsized":"2024-06-13T06:53:29.635239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256834+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:51:01.256834+0000","last_peered":"2024-06-13T06:51:01.256834+0000","last_clean":"2024-06-13T06:51:01.256834+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:51:01.256834+0000","last_undegraded":"2024-06-13T06:51:01.256834+0000","last_fullsized":"2024-06-13T06:51:01.256834+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780226+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:51:40.780226+0000","last_peered":"2024-06-13T06:51:40.780226+0000","last_clean":"2024-06-13T06:51:40.780226+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:51:40.780226+0000","last_undegraded":"2024-06-13T06:51:40.780226+0000","last_fullsized":"2024-06-13T06:51:40.780226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"98'2","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.587970+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:53:29.587970+0000","last_peered":"2024-06-13T06:53:29.587970+0000","last_clean":"2024-06-13T06:53:29.587970+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:53:29.587970+0000","last_undegraded":"2024-06-13T06:53:29.587970+0000","last_fullsized":"2024-06-13T06:53:29.587970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892510+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:51:00.892510+0000","last_peered":"2024-06-13T06:51:00.892510+0000","last_clean":"2024-06-13T06:51:00.892510+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:51:00.892510+0000","last_undegraded":"2024-06-13T06:51:00.892510+0000","last_fullsized":"2024-06-13T06:51:00.892510+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781263+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:51:40.781263+0000","last_peered":"2024-06-13T06:51:40.781263+0000","last_clean":"2024-06-13T06:51:40.781263+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:51:40.781263+0000","last_undegraded":"2024-06-13T06:51:40.781263+0000","last_fullsized":"2024-06-13T06:51:40.781263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.624953+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:53:29.624953+0000","last_peered":"2024-06-13T06:53:29.624953+0000","last_clean":"2024-06-13T06:53:29.624953+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:53:29.624953+0000","last_undegraded":"2024-06-13T06:53:29.624953+0000","last_fullsized":"2024-06-13T06:53:29.624953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:06.674426+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:52:06.674426+0000","last_peered":"2024-06-13T06:52:06.674426+0000","last_clean":"2024-06-13T06:52:06.674426+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:52:06.674426+0000","last_undegraded":"2024-06-13T06:52:06.674426+0000","last_fullsized":"2024-06-13T06:52:06.674426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:42.777256+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:51:42.777256+0000","last_peered":"2024-06-13T06:51:42.777256+0000","last_clean":"2024-06-13T06:51:42.777256+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:51:42.777256+0000","last_undegraded":"2024-06-13T06:51:42.777256+0000","last_fullsized":"2024-06-13T06:51:42.777256+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639123+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:53:29.639123+0000","last_peered":"2024-06-13T06:53:29.639123+0000","last_clean":"2024-06-13T06:53:29.639123+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:53:29.639123+0000","last_undegraded":"2024-06-13T06:53:29.639123+0000","last_fullsized":"2024-06-13T06:53:29.639123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891484+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:51:00.891484+0000","last_peered":"2024-06-13T06:51:00.891484+0000","last_clean":"2024-06-13T06:51:00.891484+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:51:00.891484+0000","last_undegraded":"2024-06-13T06:51:00.891484+0000","last_fullsized":"2024-06-13T06:51:00.891484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831122+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:51:45.831122+0000","last_peered":"2024-06-13T06:51:45.831122+0000","last_clean":"2024-06-13T06:51:45.831122+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:51:45.831122+0000","last_undegraded":"2024-06-13T06:51:45.831122+0000","last_fullsized":"2024-06-13T06:51:45.831122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"98'5","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631856+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:53:29.631856+0000","last_peered":"2024-06-13T06:53:29.631856+0000","last_clean":"2024-06-13T06:53:29.631856+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:53:29.631856+0000","last_undegraded":"2024-06-13T06:53:29.631856+0000","last_fullsized":"2024-06-13T06:53:29.631856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943373+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:51:00.943373+0000","last_peered":"2024-06-13T06:51:00.943373+0000","last_clean":"2024-06-13T06:51:00.943373+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:51:00.943373+0000","last_undegraded":"2024-06-13T06:51:00.943373+0000","last_fullsized":"2024-06-13T06:51:00.943373+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835140+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:51:45.835140+0000","last_peered":"2024-06-13T06:51:45.835140+0000","last_clean":"2024-06-13T06:51:45.835140+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:51:45.835140+0000","last_undegraded":"2024-06-13T06:51:45.835140+0000","last_fullsized":"2024-06-13T06:51:45.835140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"98'7","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632579+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:53:29.632579+0000","last_peered":"2024-06-13T06:53:29.632579+0000","last_clean":"2024-06-13T06:53:29.632579+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:53:29.632579+0000","last_undegraded":"2024-06-13T06:53:29.632579+0000","last_fullsized":"2024-06-13T06:53:29.632579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.258645+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:51:01.258645+0000","last_peered":"2024-06-13T06:51:01.258645+0000","last_clean":"2024-06-13T06:51:01.258645+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:51:01.258645+0000","last_undegraded":"2024-06-13T06:51:01.258645+0000","last_fullsized":"2024-06-13T06:51:01.258645+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831717+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:51:45.831717+0000","last_peered":"2024-06-13T06:51:45.831717+0000","last_clean":"2024-06-13T06:51:45.831717+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:51:45.831717+0000","last_undegraded":"2024-06-13T06:51:45.831717+0000","last_fullsized":"2024-06-13T06:51:45.831717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629662+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:53:29.629662+0000","last_peered":"2024-06-13T06:53:29.629662+0000","last_clean":"2024-06-13T06:53:29.629662+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:53:29.629662+0000","last_undegraded":"2024-06-13T06:53:29.629662+0000","last_fullsized":"2024-06-13T06:53:29.629662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252489+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:51:01.252489+0000","last_peered":"2024-06-13T06:51:01.252489+0000","last_clean":"2024-06-13T06:51:01.252489+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:51:01.252489+0000","last_undegraded":"2024-06-13T06:51:01.252489+0000","last_fullsized":"2024-06-13T06:51:01.252489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960980+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:51:00.960980+0000","last_peered":"2024-06-13T06:51:00.960980+0000","last_clean":"2024-06-13T06:51:00.960980+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:51:00.960980+0000","last_undegraded":"2024-06-13T06:51:00.960980+0000","last_fullsized":"2024-06-13T06:51:00.960980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"98'9","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.620947+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:53:29.620947+0000","last_peered":"2024-06-13T06:53:29.620947+0000","last_clean":"2024-06-13T06:53:29.620947+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:53:29.620947+0000","last_undegraded":"2024-06-13T06:53:29.620947+0000","last_fullsized":"2024-06-13T06:53:29.620947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623417+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:53:29.623417+0000","last_peered":"2024-06-13T06:53:29.623417+0000","last_clean":"2024-06-13T06:53:29.623417+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:53:29.623417+0000","last_undegraded":"2024-06-13T06:53:29.623417+0000","last_fullsized":"2024-06-13T06:53:29.623417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"98'12","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623598+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:53:29.623598+0000","last_peered":"2024-06-13T06:53:29.623598+0000","last_clean":"2024-06-13T06:53:29.623598+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:53:29.623598+0000","last_undegraded":"2024-06-13T06:53:29.623598+0000","last_fullsized":"2024-06-13T06:53:29.623598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629587+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:53:29.629587+0000","last_peered":"2024-06-13T06:53:29.629587+0000","last_clean":"2024-06-13T06:53:29.629587+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:53:29.629587+0000","last_undegraded":"2024-06-13T06:53:29.629587+0000","last_fullsized":"2024-06-13T06:53:29.629587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638021+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:53:29.638021+0000","last_peered":"2024-06-13T06:53:29.638021+0000","last_clean":"2024-06-13T06:53:29.638021+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:53:29.638021+0000","last_undegraded":"2024-06-13T06:53:29.638021+0000","last_fullsized":"2024-06-13T06:53:29.638021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631033+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:53:29.631033+0000","last_peered":"2024-06-13T06:53:29.631033+0000","last_clean":"2024-06-13T06:53:29.631033+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:53:29.631033+0000","last_undegraded":"2024-06-13T06:53:29.631033+0000","last_fullsized":"2024-06-13T06:53:29.631033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617246+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:53:29.617246+0000","last_peered":"2024-06-13T06:53:29.617246+0000","last_clean":"2024-06-13T06:53:29.617246+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:53:29.617246+0000","last_undegraded":"2024-06-13T06:53:29.617246+0000","last_fullsized":"2024-06-13T06:53:29.617246+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629337+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:53:29.629337+0000","last_peered":"2024-06-13T06:53:29.629337+0000","last_clean":"2024-06-13T06:53:29.629337+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:53:29.629337+0000","last_undegraded":"2024-06-13T06:53:29.629337+0000","last_fullsized":"2024-06-13T06:53:29.629337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"98'2","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629570+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:53:29.629570+0000","last_peered":"2024-06-13T06:53:29.629570+0000","last_clean":"2024-06-13T06:53:29.629570+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:53:29.629570+0000","last_undegraded":"2024-06-13T06:53:29.629570+0000","last_fullsized":"2024-06-13T06:53:29.629570+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639896+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:53:29.639896+0000","last_peered":"2024-06-13T06:53:29.639896+0000","last_clean":"2024-06-13T06:53:29.639896+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:53:29.639896+0000","last_undegraded":"2024-06-13T06:53:29.639896+0000","last_fullsized":"2024-06-13T06:53:29.639896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631619+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:53:29.631619+0000","last_peered":"2024-06-13T06:53:29.631619+0000","last_clean":"2024-06-13T06:53:29.631619+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:53:29.631619+0000","last_undegraded":"2024-06-13T06:53:29.631619+0000","last_fullsized":"2024-06-13T06:53:29.631619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640472+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:53:29.640472+0000","last_peered":"2024-06-13T06:53:29.640472+0000","last_clean":"2024-06-13T06:53:29.640472+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:53:29.640472+0000","last_undegraded":"2024-06-13T06:53:29.640472+0000","last_fullsized":"2024-06-13T06:53:29.640472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632523+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:53:29.632523+0000","last_peered":"2024-06-13T06:53:29.632523+0000","last_clean":"2024-06-13T06:53:29.632523+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:53:29.632523+0000","last_undegraded":"2024-06-13T06:53:29.632523+0000","last_fullsized":"2024-06-13T06:53:29.632523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628229+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:53:29.628229+0000","last_peered":"2024-06-13T06:53:29.628229+0000","last_clean":"2024-06-13T06:53:29.628229+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:53:29.628229+0000","last_undegraded":"2024-06-13T06:53:29.628229+0000","last_fullsized":"2024-06-13T06:53:29.628229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68448,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":718,"num_read_kb":434,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351035,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612672,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":1210149,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776196,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037836,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150152,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":613576,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":614926,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888162,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554540,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724638,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:53:44.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph pg dump --format=json 2024-06-13T06:53:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:45.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:44 smithi189 ceph-mon[36616]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:53:45.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:44 smithi175 ceph-mon[36886]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:53:45.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:44 smithi007 ceph-mon[32077]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-13T06:53:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 -- 172.21.15.7:0/1106471919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 msgr2=0x7f8bc410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 --2- 172.21.15.7:0/1106471919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc410f320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b0a0 tx=0x7f8bbc00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 -- 172.21.15.7:0/1106471919 shutdown_connections 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 --2- 172.21.15.7:0/1106471919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc410f320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 --2- 172.21.15.7:0/1106471919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc4108500 0x7f8bc41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 --2- 172.21.15.7:0/1106471919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc406a170 0x7f8bc406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 -- 172.21.15.7:0/1106471919 >> 172.21.15.7:0/1106471919 conn(0x7f8bc4078340 msgr2=0x7f8bc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 -- 172.21.15.7:0/1106471919 shutdown_connections 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.058+0000 7f8bcb600700 1 -- 172.21.15.7:0/1106471919 wait complete. 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.059+0000 7f8bcb600700 1 Processor -- start 2024-06-13T06:53:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.059+0000 7f8bcb600700 1 -- start start 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 0x7f8bc419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4112760 con 0x7f8bc4108500 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41125e0 con 0x7f8bc406a170 2024-06-13T06:53:47.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bcb600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41128e0 con 0x7f8bc406ab30 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57260/0 (socket says 172.21.15.7:57260) 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41188/0 (socket says 172.21.15.7:41188) 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc3e00700 1 -- 172.21.15.7:0/118764032 learned_addr learned my addr 172.21.15.7:0/118764032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.060+0000 7f8bc8c00700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 0x7f8bc419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 -- 172.21.15.7:0/118764032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 msgr2=0x7f8bc419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 -- 172.21.15.7:0/118764032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 msgr2=0x7f8bc419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 0x7f8bc419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 -- 172.21.15.7:0/118764032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc41990e0 con 0x7f8bc4108500 2024-06-13T06:53:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc8c00700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 0x7f8bc419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc3e00700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc9600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b0a0 tx=0x7f8bbc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc021070 con 0x7f8bc4108500 2024-06-13T06:53:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc4199370 con 0x7f8bc4108500 2024-06-13T06:53:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc007b80 con 0x7f8bc4108500 2024-06-13T06:53:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.061+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc005040 con 0x7f8bc4108500 2024-06-13T06:53:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.062+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41a5690 con 0x7f8bc4108500 2024-06-13T06:53:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.063+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc4103bc0 con 0x7f8bc4108500 2024-06-13T06:53:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.067+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8bbc024070 con 0x7f8bc4108500 2024-06-13T06:53:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.068+0000 7f8bc1600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 0x7f8bac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.068+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8bbc0d0240 con 0x7f8bc4108500 2024-06-13T06:53:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.068+0000 7f8bc8c00700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 0x7f8bac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.069+0000 7f8bc8c00700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 0x7f8bac0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009680 tx=0x7f8bb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.071+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bbc007d30 con 0x7f8bc4108500 2024-06-13T06:53:47.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:46 smithi175 ceph-mon[36886]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:47.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:46 smithi007 ceph-mon[32077]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:47.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.301+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8bc40628d0 con 0x7f8bac0a2c20 2024-06-13T06:53:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.330+0000 7f8bc1600700 1 -- 172.21.15.7:0/118764032 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340761 (secure 0 0 0) 0x7f8bc40628d0 con 0x7f8bac0a2c20 2024-06-13T06:53:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:46 smithi189 ceph-mon[36616]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:47.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 msgr2=0x7f8bac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 0x7f8bac0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009680 tx=0x7f8bb8009260 comp rx=0 tx=0).stop 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 msgr2=0x7f8bc4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b0a0 tx=0x7f8bbc004d00 comp rx=0 tx=0).stop 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 shutdown_connections 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bc4108500 0x7f8bc4198980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8bac0a2c20 0x7f8bac0a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bc406a170 0x7f8bc419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 --2- 172.21.15.7:0/118764032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc406ab30 0x7f8bc419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 >> 172.21.15.7:0/118764032 conn(0x7f8bc4078340 msgr2=0x7f8bc40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.337+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 shutdown_connections 2024-06-13T06:53:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:47.338+0000 7f8bcb600700 1 -- 172.21.15.7:0/118764032 wait complete. 2024-06-13T06:53:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:53:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:47 smithi175 ceph-mon[36886]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:48.156 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:47 smithi007 ceph-mon[32077]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:48.228 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-06-13T06:53:45.701206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659050,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":869,"num_read_kb":553,"num_write":359,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378916,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562908,"statfs":{"total":1151940427776,"available":1151552417792,"internally_reserved":0,"allocated":13438976,"data_stored":8600444,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006303"},"pg_stats":[{"pgid":"3.3f","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632049+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:53:29.632049+0000","last_peered":"2024-06-13T06:53:29.632049+0000","last_clean":"2024-06-13T06:53:29.632049+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:53:29.632049+0000","last_undegraded":"2024-06-13T06:53:29.632049+0000","last_fullsized":"2024-06-13T06:53:29.632049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626164+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:53:29.626164+0000","last_peered":"2024-06-13T06:53:29.626164+0000","last_clean":"2024-06-13T06:53:29.626164+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:53:29.626164+0000","last_undegraded":"2024-06-13T06:53:29.626164+0000","last_fullsized":"2024-06-13T06:53:29.626164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622717+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:53:29.622717+0000","last_peered":"2024-06-13T06:53:29.622717+0000","last_clean":"2024-06-13T06:53:29.622717+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:53:29.622717+0000","last_undegraded":"2024-06-13T06:53:29.622717+0000","last_fullsized":"2024-06-13T06:53:29.622717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"98'8","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622494+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:53:29.622494+0000","last_peered":"2024-06-13T06:53:29.622494+0000","last_clean":"2024-06-13T06:53:29.622494+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:53:29.622494+0000","last_undegraded":"2024-06-13T06:53:29.622494+0000","last_fullsized":"2024-06-13T06:53:29.622494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633368+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:53:29.633368+0000","last_peered":"2024-06-13T06:53:29.633368+0000","last_clean":"2024-06-13T06:53:29.633368+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:53:29.633368+0000","last_undegraded":"2024-06-13T06:53:29.633368+0000","last_fullsized":"2024-06-13T06:53:29.633368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633444+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:53:29.633444+0000","last_peered":"2024-06-13T06:53:29.633444+0000","last_clean":"2024-06-13T06:53:29.633444+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:53:29.633444+0000","last_undegraded":"2024-06-13T06:53:29.633444+0000","last_fullsized":"2024-06-13T06:53:29.633444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"98'5","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628727+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:53:29.628727+0000","last_peered":"2024-06-13T06:53:29.628727+0000","last_clean":"2024-06-13T06:53:29.628727+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:53:29.628727+0000","last_undegraded":"2024-06-13T06:53:29.628727+0000","last_fullsized":"2024-06-13T06:53:29.628727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637897+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:53:29.637897+0000","last_peered":"2024-06-13T06:53:29.637897+0000","last_clean":"2024-06-13T06:53:29.637897+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:53:29.637897+0000","last_undegraded":"2024-06-13T06:53:29.637897+0000","last_fullsized":"2024-06-13T06:53:29.637897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631145+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:53:29.631145+0000","last_peered":"2024-06-13T06:53:29.631145+0000","last_clean":"2024-06-13T06:53:29.631145+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:53:29.631145+0000","last_undegraded":"2024-06-13T06:53:29.631145+0000","last_fullsized":"2024-06-13T06:53:29.631145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"98'16","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627110+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:53:29.627110+0000","last_peered":"2024-06-13T06:53:29.627110+0000","last_clean":"2024-06-13T06:53:29.627110+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:53:29.627110+0000","last_undegraded":"2024-06-13T06:53:29.627110+0000","last_fullsized":"2024-06-13T06:53:29.627110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632199+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:53:29.632199+0000","last_peered":"2024-06-13T06:53:29.632199+0000","last_clean":"2024-06-13T06:53:29.632199+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:53:29.632199+0000","last_undegraded":"2024-06-13T06:53:29.632199+0000","last_fullsized":"2024-06-13T06:53:29.632199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"98'7","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635014+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:53:29.635014+0000","last_peered":"2024-06-13T06:53:29.635014+0000","last_clean":"2024-06-13T06:53:29.635014+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:53:29.635014+0000","last_undegraded":"2024-06-13T06:53:29.635014+0000","last_fullsized":"2024-06-13T06:53:29.635014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631329+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:53:29.631329+0000","last_peered":"2024-06-13T06:53:29.631329+0000","last_clean":"2024-06-13T06:53:29.631329+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:53:29.631329+0000","last_undegraded":"2024-06-13T06:53:29.631329+0000","last_fullsized":"2024-06-13T06:53:29.631329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628371+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:53:29.628371+0000","last_peered":"2024-06-13T06:53:29.628371+0000","last_clean":"2024-06-13T06:53:29.628371+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:53:29.628371+0000","last_undegraded":"2024-06-13T06:53:29.628371+0000","last_fullsized":"2024-06-13T06:53:29.628371+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622830+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:53:29.622830+0000","last_peered":"2024-06-13T06:53:29.622830+0000","last_clean":"2024-06-13T06:53:29.622830+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:53:29.622830+0000","last_undegraded":"2024-06-13T06:53:29.622830+0000","last_fullsized":"2024-06-13T06:53:29.622830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631629+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:53:29.631629+0000","last_peered":"2024-06-13T06:53:29.631629+0000","last_clean":"2024-06-13T06:53:29.631629+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:53:29.631629+0000","last_undegraded":"2024-06-13T06:53:29.631629+0000","last_fullsized":"2024-06-13T06:53:29.631629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632097+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:53:29.632097+0000","last_peered":"2024-06-13T06:53:29.632097+0000","last_clean":"2024-06-13T06:53:29.632097+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:53:29.632097+0000","last_undegraded":"2024-06-13T06:53:29.632097+0000","last_fullsized":"2024-06-13T06:53:29.632097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634479+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:53:29.634479+0000","last_peered":"2024-06-13T06:53:29.634479+0000","last_clean":"2024-06-13T06:53:29.634479+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:53:29.634479+0000","last_undegraded":"2024-06-13T06:53:29.634479+0000","last_fullsized":"2024-06-13T06:53:29.634479+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629955+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:53:29.629955+0000","last_peered":"2024-06-13T06:53:29.629955+0000","last_clean":"2024-06-13T06:53:29.629955+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:53:29.629955+0000","last_undegraded":"2024-06-13T06:53:29.629955+0000","last_fullsized":"2024-06-13T06:53:29.629955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881746+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:51:00.881746+0000","last_peered":"2024-06-13T06:51:00.881746+0000","last_clean":"2024-06-13T06:51:00.881746+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:51:00.881746+0000","last_undegraded":"2024-06-13T06:51:00.881746+0000","last_fullsized":"2024-06-13T06:51:00.881746+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835279+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:51:45.835279+0000","last_peered":"2024-06-13T06:51:45.835279+0000","last_clean":"2024-06-13T06:51:45.835279+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:51:45.835279+0000","last_undegraded":"2024-06-13T06:51:45.835279+0000","last_fullsized":"2024-06-13T06:51:45.835279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635321+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:53:29.635321+0000","last_peered":"2024-06-13T06:53:29.635321+0000","last_clean":"2024-06-13T06:53:29.635321+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:53:29.635321+0000","last_undegraded":"2024-06-13T06:53:29.635321+0000","last_fullsized":"2024-06-13T06:53:29.635321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891212+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:51:00.891212+0000","last_peered":"2024-06-13T06:51:00.891212+0000","last_clean":"2024-06-13T06:51:00.891212+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:51:00.891212+0000","last_undegraded":"2024-06-13T06:51:00.891212+0000","last_fullsized":"2024-06-13T06:51:00.891212+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.803500+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:51:41.803500+0000","last_peered":"2024-06-13T06:51:41.803500+0000","last_clean":"2024-06-13T06:51:41.803500+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:51:41.803500+0000","last_undegraded":"2024-06-13T06:51:41.803500+0000","last_fullsized":"2024-06-13T06:51:41.803500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635642+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:53:29.635642+0000","last_peered":"2024-06-13T06:53:29.635642+0000","last_clean":"2024-06-13T06:53:29.635642+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:53:29.635642+0000","last_undegraded":"2024-06-13T06:53:29.635642+0000","last_fullsized":"2024-06-13T06:53:29.635642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891648+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891648+0000","last_peered":"2024-06-13T06:51:00.891648+0000","last_clean":"2024-06-13T06:51:00.891648+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891648+0000","last_undegraded":"2024-06-13T06:51:00.891648+0000","last_fullsized":"2024-06-13T06:51:00.891648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799045+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:51:41.799045+0000","last_peered":"2024-06-13T06:51:41.799045+0000","last_clean":"2024-06-13T06:51:41.799045+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:51:41.799045+0000","last_undegraded":"2024-06-13T06:51:41.799045+0000","last_fullsized":"2024-06-13T06:51:41.799045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634597+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:53:29.634597+0000","last_peered":"2024-06-13T06:53:29.634597+0000","last_clean":"2024-06-13T06:53:29.634597+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:53:29.634597+0000","last_undegraded":"2024-06-13T06:53:29.634597+0000","last_fullsized":"2024-06-13T06:53:29.634597+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891222+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891222+0000","last_peered":"2024-06-13T06:51:00.891222+0000","last_clean":"2024-06-13T06:51:00.891222+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891222+0000","last_undegraded":"2024-06-13T06:51:00.891222+0000","last_fullsized":"2024-06-13T06:51:00.891222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254033+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:51:01.254033+0000","last_peered":"2024-06-13T06:51:01.254033+0000","last_clean":"2024-06-13T06:51:01.254033+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:51:01.254033+0000","last_undegraded":"2024-06-13T06:51:01.254033+0000","last_fullsized":"2024-06-13T06:51:01.254033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631918+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:53:29.631918+0000","last_peered":"2024-06-13T06:53:29.631918+0000","last_clean":"2024-06-13T06:53:29.631918+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:53:29.631918+0000","last_undegraded":"2024-06-13T06:53:29.631918+0000","last_fullsized":"2024-06-13T06:53:29.631918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881987+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881987+0000","last_peered":"2024-06-13T06:51:00.881987+0000","last_clean":"2024-06-13T06:51:00.881987+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881987+0000","last_undegraded":"2024-06-13T06:51:00.881987+0000","last_fullsized":"2024-06-13T06:51:00.881987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:44.009018+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:53:44.009018+0000","last_peered":"2024-06-13T06:53:44.009018+0000","last_clean":"2024-06-13T06:53:44.009018+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:53:44.009018+0000","last_undegraded":"2024-06-13T06:53:44.009018+0000","last_fullsized":"2024-06-13T06:53:44.009018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617073+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:53:29.617073+0000","last_peered":"2024-06-13T06:53:29.617073+0000","last_clean":"2024-06-13T06:53:29.617073+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:53:29.617073+0000","last_undegraded":"2024-06-13T06:53:29.617073+0000","last_fullsized":"2024-06-13T06:53:29.617073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882139+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882139+0000","last_peered":"2024-06-13T06:51:00.882139+0000","last_clean":"2024-06-13T06:51:00.882139+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882139+0000","last_undegraded":"2024-06-13T06:51:00.882139+0000","last_fullsized":"2024-06-13T06:51:00.882139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.801075+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:51:41.801075+0000","last_peered":"2024-06-13T06:51:41.801075+0000","last_clean":"2024-06-13T06:51:41.801075+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:51:41.801075+0000","last_undegraded":"2024-06-13T06:51:41.801075+0000","last_fullsized":"2024-06-13T06:51:41.801075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640050+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:53:29.640050+0000","last_peered":"2024-06-13T06:53:29.640050+0000","last_clean":"2024-06-13T06:53:29.640050+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:53:29.640050+0000","last_undegraded":"2024-06-13T06:53:29.640050+0000","last_fullsized":"2024-06-13T06:53:29.640050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.884182+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:51:00.884182+0000","last_peered":"2024-06-13T06:51:00.884182+0000","last_clean":"2024-06-13T06:51:00.884182+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:51:00.884182+0000","last_undegraded":"2024-06-13T06:51:00.884182+0000","last_fullsized":"2024-06-13T06:51:00.884182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.805782+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:51:41.805782+0000","last_peered":"2024-06-13T06:51:41.805782+0000","last_clean":"2024-06-13T06:51:41.805782+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:51:41.805782+0000","last_undegraded":"2024-06-13T06:51:41.805782+0000","last_fullsized":"2024-06-13T06:51:41.805782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637137+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:53:29.637137+0000","last_peered":"2024-06-13T06:53:29.637137+0000","last_clean":"2024-06-13T06:53:29.637137+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:53:29.637137+0000","last_undegraded":"2024-06-13T06:53:29.637137+0000","last_fullsized":"2024-06-13T06:53:29.637137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888882+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:51:00.888882+0000","last_peered":"2024-06-13T06:51:00.888882+0000","last_clean":"2024-06-13T06:51:00.888882+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:51:00.888882+0000","last_undegraded":"2024-06-13T06:51:00.888882+0000","last_fullsized":"2024-06-13T06:51:00.888882+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780108+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:51:40.780108+0000","last_peered":"2024-06-13T06:51:40.780108+0000","last_clean":"2024-06-13T06:51:40.780108+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:51:40.780108+0000","last_undegraded":"2024-06-13T06:51:40.780108+0000","last_fullsized":"2024-06-13T06:51:40.780108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639788+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:53:29.639788+0000","last_peered":"2024-06-13T06:53:29.639788+0000","last_clean":"2024-06-13T06:53:29.639788+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:53:29.639788+0000","last_undegraded":"2024-06-13T06:53:29.639788+0000","last_fullsized":"2024-06-13T06:53:29.639788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230296+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:51:01.230296+0000","last_peered":"2024-06-13T06:51:01.230296+0000","last_clean":"2024-06-13T06:51:01.230296+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:51:01.230296+0000","last_undegraded":"2024-06-13T06:51:01.230296+0000","last_fullsized":"2024-06-13T06:51:01.230296+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":164,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:44.008373+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:53:44.008373+0000","last_peered":"2024-06-13T06:53:44.008373+0000","last_clean":"2024-06-13T06:53:44.008373+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:53:44.008373+0000","last_undegraded":"2024-06-13T06:53:44.008373+0000","last_fullsized":"2024-06-13T06:53:44.008373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634415+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:53:29.634415+0000","last_peered":"2024-06-13T06:53:29.634415+0000","last_clean":"2024-06-13T06:53:29.634415+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:53:29.634415+0000","last_undegraded":"2024-06-13T06:53:29.634415+0000","last_fullsized":"2024-06-13T06:53:29.634415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.888254+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:51:00.888254+0000","last_peered":"2024-06-13T06:51:00.888254+0000","last_clean":"2024-06-13T06:51:00.888254+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:51:00.888254+0000","last_undegraded":"2024-06-13T06:51:00.888254+0000","last_fullsized":"2024-06-13T06:51:00.888254+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.779852+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:51:40.779852+0000","last_peered":"2024-06-13T06:51:40.779852+0000","last_clean":"2024-06-13T06:51:40.779852+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:51:40.779852+0000","last_undegraded":"2024-06-13T06:51:40.779852+0000","last_fullsized":"2024-06-13T06:51:40.779852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628538+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:53:29.628538+0000","last_peered":"2024-06-13T06:53:29.628538+0000","last_clean":"2024-06-13T06:53:29.628538+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:53:29.628538+0000","last_undegraded":"2024-06-13T06:53:29.628538+0000","last_fullsized":"2024-06-13T06:53:29.628538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891857+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:51:00.891857+0000","last_peered":"2024-06-13T06:51:00.891857+0000","last_clean":"2024-06-13T06:51:00.891857+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:51:00.891857+0000","last_undegraded":"2024-06-13T06:51:00.891857+0000","last_fullsized":"2024-06-13T06:51:00.891857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.833439+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:51:45.833439+0000","last_peered":"2024-06-13T06:51:45.833439+0000","last_clean":"2024-06-13T06:51:45.833439+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:51:45.833439+0000","last_undegraded":"2024-06-13T06:51:45.833439+0000","last_fullsized":"2024-06-13T06:51:45.833439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632252+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:53:29.632252+0000","last_peered":"2024-06-13T06:53:29.632252+0000","last_clean":"2024-06-13T06:53:29.632252+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:53:29.632252+0000","last_undegraded":"2024-06-13T06:53:29.632252+0000","last_fullsized":"2024-06-13T06:53:29.632252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891224+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:51:00.891224+0000","last_peered":"2024-06-13T06:51:00.891224+0000","last_clean":"2024-06-13T06:51:00.891224+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:51:00.891224+0000","last_undegraded":"2024-06-13T06:51:00.891224+0000","last_fullsized":"2024-06-13T06:51:00.891224+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'37","reported_seq":328,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:45.019768+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:53:45.019768+0000","last_peered":"2024-06-13T06:53:45.019768+0000","last_clean":"2024-06-13T06:53:45.019768+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:53:45.019768+0000","last_undegraded":"2024-06-13T06:53:45.019768+0000","last_fullsized":"2024-06-13T06:53:45.019768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64307,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":188,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.611823+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:53:29.611823+0000","last_peered":"2024-06-13T06:53:29.611823+0000","last_clean":"2024-06-13T06:53:29.611823+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:53:29.611823+0000","last_undegraded":"2024-06-13T06:53:29.611823+0000","last_fullsized":"2024-06-13T06:53:29.611823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253857+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:51:01.253857+0000","last_peered":"2024-06-13T06:51:01.253857+0000","last_clean":"2024-06-13T06:51:01.253857+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:51:01.253857+0000","last_undegraded":"2024-06-13T06:51:01.253857+0000","last_fullsized":"2024-06-13T06:51:01.253857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.830980+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:51:45.830980+0000","last_peered":"2024-06-13T06:51:45.830980+0000","last_clean":"2024-06-13T06:51:45.830980+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:51:45.830980+0000","last_undegraded":"2024-06-13T06:51:45.830980+0000","last_fullsized":"2024-06-13T06:51:45.830980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630741+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:53:29.630741+0000","last_peered":"2024-06-13T06:53:29.630741+0000","last_clean":"2024-06-13T06:53:29.630741+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:53:29.630741+0000","last_undegraded":"2024-06-13T06:53:29.630741+0000","last_fullsized":"2024-06-13T06:53:29.630741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.883955+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:51:00.883955+0000","last_peered":"2024-06-13T06:51:00.883955+0000","last_clean":"2024-06-13T06:51:00.883955+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:51:00.883955+0000","last_undegraded":"2024-06-13T06:51:00.883955+0000","last_fullsized":"2024-06-13T06:51:00.883955+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781368+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:51:40.781368+0000","last_peered":"2024-06-13T06:51:40.781368+0000","last_clean":"2024-06-13T06:51:40.781368+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:51:40.781368+0000","last_undegraded":"2024-06-13T06:51:40.781368+0000","last_fullsized":"2024-06-13T06:51:40.781368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"98'15","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632168+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:53:29.632168+0000","last_peered":"2024-06-13T06:53:29.632168+0000","last_clean":"2024-06-13T06:53:29.632168+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:53:29.632168+0000","last_undegraded":"2024-06-13T06:53:29.632168+0000","last_fullsized":"2024-06-13T06:53:29.632168+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892716+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:51:00.892716+0000","last_peered":"2024-06-13T06:51:00.892716+0000","last_clean":"2024-06-13T06:51:00.892716+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:51:00.892716+0000","last_undegraded":"2024-06-13T06:51:00.892716+0000","last_fullsized":"2024-06-13T06:51:00.892716+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":167,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:41.085487+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:53:41.085487+0000","last_peered":"2024-06-13T06:53:41.085487+0000","last_clean":"2024-06-13T06:53:41.085487+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:53:41.085487+0000","last_undegraded":"2024-06-13T06:53:41.085487+0000","last_fullsized":"2024-06-13T06:53:41.085487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633544+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:53:29.633544+0000","last_peered":"2024-06-13T06:53:29.633544+0000","last_clean":"2024-06-13T06:53:29.633544+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:53:29.633544+0000","last_undegraded":"2024-06-13T06:53:29.633544+0000","last_fullsized":"2024-06-13T06:53:29.633544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961104+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:51:00.961104+0000","last_peered":"2024-06-13T06:51:00.961104+0000","last_clean":"2024-06-13T06:51:00.961104+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:51:00.961104+0000","last_undegraded":"2024-06-13T06:51:00.961104+0000","last_fullsized":"2024-06-13T06:51:00.961104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":57,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.830961+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:51:45.830961+0000","last_peered":"2024-06-13T06:51:45.830961+0000","last_clean":"2024-06-13T06:51:45.830961+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:51:45.830961+0000","last_undegraded":"2024-06-13T06:51:45.830961+0000","last_fullsized":"2024-06-13T06:51:45.830961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622846+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:53:29.622846+0000","last_peered":"2024-06-13T06:53:29.622846+0000","last_clean":"2024-06-13T06:53:29.622846+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:53:29.622846+0000","last_undegraded":"2024-06-13T06:53:29.622846+0000","last_fullsized":"2024-06-13T06:53:29.622846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.253488+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:51:01.253488+0000","last_peered":"2024-06-13T06:51:01.253488+0000","last_clean":"2024-06-13T06:51:01.253488+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:51:01.253488+0000","last_undegraded":"2024-06-13T06:51:01.253488+0000","last_fullsized":"2024-06-13T06:51:01.253488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831263+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:51:45.831263+0000","last_peered":"2024-06-13T06:51:45.831263+0000","last_clean":"2024-06-13T06:51:45.831263+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:51:45.831263+0000","last_undegraded":"2024-06-13T06:51:45.831263+0000","last_fullsized":"2024-06-13T06:51:45.831263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627776+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:53:29.627776+0000","last_peered":"2024-06-13T06:53:29.627776+0000","last_clean":"2024-06-13T06:53:29.627776+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:53:29.627776+0000","last_undegraded":"2024-06-13T06:53:29.627776+0000","last_fullsized":"2024-06-13T06:53:29.627776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254101+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:51:01.254101+0000","last_peered":"2024-06-13T06:51:01.254101+0000","last_clean":"2024-06-13T06:51:01.254101+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:51:01.254101+0000","last_undegraded":"2024-06-13T06:51:01.254101+0000","last_fullsized":"2024-06-13T06:51:01.254101+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.896197+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:51:00.896197+0000","last_peered":"2024-06-13T06:51:00.896197+0000","last_clean":"2024-06-13T06:51:00.896197+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:51:00.896197+0000","last_undegraded":"2024-06-13T06:51:00.896197+0000","last_fullsized":"2024-06-13T06:51:00.896197+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636656+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:53:29.636656+0000","last_peered":"2024-06-13T06:53:29.636656+0000","last_clean":"2024-06-13T06:53:29.636656+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:53:29.636656+0000","last_undegraded":"2024-06-13T06:53:29.636656+0000","last_fullsized":"2024-06-13T06:53:29.636656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960360+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:51:00.960360+0000","last_peered":"2024-06-13T06:51:00.960360+0000","last_clean":"2024-06-13T06:51:00.960360+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:51:00.960360+0000","last_undegraded":"2024-06-13T06:51:00.960360+0000","last_fullsized":"2024-06-13T06:51:00.960360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.799920+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:51:41.799920+0000","last_peered":"2024-06-13T06:51:41.799920+0000","last_clean":"2024-06-13T06:51:41.799920+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:51:41.799920+0000","last_undegraded":"2024-06-13T06:51:41.799920+0000","last_fullsized":"2024-06-13T06:51:41.799920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623582+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:53:29.623582+0000","last_peered":"2024-06-13T06:53:29.623582+0000","last_clean":"2024-06-13T06:53:29.623582+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:53:29.623582+0000","last_undegraded":"2024-06-13T06:53:29.623582+0000","last_fullsized":"2024-06-13T06:53:29.623582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.961435+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:51:00.961435+0000","last_peered":"2024-06-13T06:51:00.961435+0000","last_clean":"2024-06-13T06:51:00.961435+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:51:00.961435+0000","last_undegraded":"2024-06-13T06:51:00.961435+0000","last_fullsized":"2024-06-13T06:51:00.961435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.621714+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:53:29.621714+0000","last_peered":"2024-06-13T06:53:29.621714+0000","last_clean":"2024-06-13T06:53:29.621714+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:53:29.621714+0000","last_undegraded":"2024-06-13T06:53:29.621714+0000","last_fullsized":"2024-06-13T06:53:29.621714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":162,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:40.081839+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:53:40.081839+0000","last_peered":"2024-06-13T06:53:40.081839+0000","last_clean":"2024-06-13T06:53:40.081839+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:53:40.081839+0000","last_undegraded":"2024-06-13T06:53:40.081839+0000","last_fullsized":"2024-06-13T06:53:40.081839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:58.774101+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:51:58.774101+0000","last_peered":"2024-06-13T06:51:58.774101+0000","last_clean":"2024-06-13T06:51:58.774101+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:51:58.774101+0000","last_undegraded":"2024-06-13T06:51:58.774101+0000","last_fullsized":"2024-06-13T06:51:58.774101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636617+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:53:29.636617+0000","last_peered":"2024-06-13T06:53:29.636617+0000","last_clean":"2024-06-13T06:53:29.636617+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:53:29.636617+0000","last_undegraded":"2024-06-13T06:53:29.636617+0000","last_fullsized":"2024-06-13T06:53:29.636617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831145+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:51:45.831145+0000","last_peered":"2024-06-13T06:51:45.831145+0000","last_clean":"2024-06-13T06:51:45.831145+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:51:45.831145+0000","last_undegraded":"2024-06-13T06:51:45.831145+0000","last_fullsized":"2024-06-13T06:51:45.831145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:11.024426+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:51:11.024426+0000","last_peered":"2024-06-13T06:51:11.024426+0000","last_clean":"2024-06-13T06:51:11.024426+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:51:11.024426+0000","last_undegraded":"2024-06-13T06:51:11.024426+0000","last_fullsized":"2024-06-13T06:51:11.024426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.254476+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:51:01.254476+0000","last_peered":"2024-06-13T06:51:01.254476+0000","last_clean":"2024-06-13T06:51:01.254476+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:51:01.254476+0000","last_undegraded":"2024-06-13T06:51:01.254476+0000","last_fullsized":"2024-06-13T06:51:01.254476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623385+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:53:29.623385+0000","last_peered":"2024-06-13T06:53:29.623385+0000","last_clean":"2024-06-13T06:53:29.623385+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:53:29.623385+0000","last_undegraded":"2024-06-13T06:53:29.623385+0000","last_fullsized":"2024-06-13T06:53:29.623385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:41.798470+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:51:41.798470+0000","last_peered":"2024-06-13T06:51:41.798470+0000","last_clean":"2024-06-13T06:51:41.798470+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:51:41.798470+0000","last_undegraded":"2024-06-13T06:51:41.798470+0000","last_fullsized":"2024-06-13T06:51:41.798470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.258410+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:51:01.258410+0000","last_peered":"2024-06-13T06:51:01.258410+0000","last_clean":"2024-06-13T06:51:01.258410+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:51:01.258410+0000","last_undegraded":"2024-06-13T06:51:01.258410+0000","last_fullsized":"2024-06-13T06:51:01.258410+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831109+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:51:45.831109+0000","last_peered":"2024-06-13T06:51:45.831109+0000","last_clean":"2024-06-13T06:51:45.831109+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:51:45.831109+0000","last_undegraded":"2024-06-13T06:51:45.831109+0000","last_fullsized":"2024-06-13T06:51:45.831109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622400+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:53:29.622400+0000","last_peered":"2024-06-13T06:53:29.622400+0000","last_clean":"2024-06-13T06:53:29.622400+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:53:29.622400+0000","last_undegraded":"2024-06-13T06:53:29.622400+0000","last_fullsized":"2024-06-13T06:53:29.622400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.230194+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:51:01.230194+0000","last_peered":"2024-06-13T06:51:01.230194+0000","last_clean":"2024-06-13T06:51:01.230194+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:51:01.230194+0000","last_undegraded":"2024-06-13T06:51:01.230194+0000","last_fullsized":"2024-06-13T06:51:01.230194+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'12","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:23.465258+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:52:23.465258+0000","last_peered":"2024-06-13T06:52:23.465258+0000","last_clean":"2024-06-13T06:52:23.465258+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:52:23.465258+0000","last_undegraded":"2024-06-13T06:52:23.465258+0000","last_fullsized":"2024-06-13T06:52:23.465258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631623+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:53:29.631623+0000","last_peered":"2024-06-13T06:53:29.631623+0000","last_clean":"2024-06-13T06:53:29.631623+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:53:29.631623+0000","last_undegraded":"2024-06-13T06:53:29.631623+0000","last_fullsized":"2024-06-13T06:53:29.631623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881608+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:51:00.881608+0000","last_peered":"2024-06-13T06:51:00.881608+0000","last_clean":"2024-06-13T06:51:00.881608+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:51:00.881608+0000","last_undegraded":"2024-06-13T06:51:00.881608+0000","last_fullsized":"2024-06-13T06:51:00.881608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781280+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:51:40.781280+0000","last_peered":"2024-06-13T06:51:40.781280+0000","last_clean":"2024-06-13T06:51:40.781280+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:51:40.781280+0000","last_undegraded":"2024-06-13T06:51:40.781280+0000","last_fullsized":"2024-06-13T06:51:40.781280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635239+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:53:29.635239+0000","last_peered":"2024-06-13T06:53:29.635239+0000","last_clean":"2024-06-13T06:53:29.635239+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:53:29.635239+0000","last_undegraded":"2024-06-13T06:53:29.635239+0000","last_fullsized":"2024-06-13T06:53:29.635239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.256834+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:51:01.256834+0000","last_peered":"2024-06-13T06:51:01.256834+0000","last_clean":"2024-06-13T06:51:01.256834+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:51:01.256834+0000","last_undegraded":"2024-06-13T06:51:01.256834+0000","last_fullsized":"2024-06-13T06:51:01.256834+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.780226+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:51:40.780226+0000","last_peered":"2024-06-13T06:51:40.780226+0000","last_clean":"2024-06-13T06:51:40.780226+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:51:40.780226+0000","last_undegraded":"2024-06-13T06:51:40.780226+0000","last_fullsized":"2024-06-13T06:51:40.780226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"98'2","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.587970+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:53:29.587970+0000","last_peered":"2024-06-13T06:53:29.587970+0000","last_clean":"2024-06-13T06:53:29.587970+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:53:29.587970+0000","last_undegraded":"2024-06-13T06:53:29.587970+0000","last_fullsized":"2024-06-13T06:53:29.587970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.892510+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:51:00.892510+0000","last_peered":"2024-06-13T06:51:00.892510+0000","last_clean":"2024-06-13T06:51:00.892510+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:51:00.892510+0000","last_undegraded":"2024-06-13T06:51:00.892510+0000","last_fullsized":"2024-06-13T06:51:00.892510+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:40.781263+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:51:40.781263+0000","last_peered":"2024-06-13T06:51:40.781263+0000","last_clean":"2024-06-13T06:51:40.781263+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:51:40.781263+0000","last_undegraded":"2024-06-13T06:51:40.781263+0000","last_fullsized":"2024-06-13T06:51:40.781263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.624953+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:53:29.624953+0000","last_peered":"2024-06-13T06:53:29.624953+0000","last_clean":"2024-06-13T06:53:29.624953+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:53:29.624953+0000","last_undegraded":"2024-06-13T06:53:29.624953+0000","last_fullsized":"2024-06-13T06:53:29.624953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:06.674426+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:52:06.674426+0000","last_peered":"2024-06-13T06:52:06.674426+0000","last_clean":"2024-06-13T06:52:06.674426+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:52:06.674426+0000","last_undegraded":"2024-06-13T06:52:06.674426+0000","last_fullsized":"2024-06-13T06:52:06.674426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:42.777256+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:51:42.777256+0000","last_peered":"2024-06-13T06:51:42.777256+0000","last_clean":"2024-06-13T06:51:42.777256+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:51:42.777256+0000","last_undegraded":"2024-06-13T06:51:42.777256+0000","last_fullsized":"2024-06-13T06:51:42.777256+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639123+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:53:29.639123+0000","last_peered":"2024-06-13T06:53:29.639123+0000","last_clean":"2024-06-13T06:53:29.639123+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:53:29.639123+0000","last_undegraded":"2024-06-13T06:53:29.639123+0000","last_fullsized":"2024-06-13T06:53:29.639123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891484+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:51:00.891484+0000","last_peered":"2024-06-13T06:51:00.891484+0000","last_clean":"2024-06-13T06:51:00.891484+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:51:00.891484+0000","last_undegraded":"2024-06-13T06:51:00.891484+0000","last_fullsized":"2024-06-13T06:51:00.891484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831122+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:51:45.831122+0000","last_peered":"2024-06-13T06:51:45.831122+0000","last_clean":"2024-06-13T06:51:45.831122+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:51:45.831122+0000","last_undegraded":"2024-06-13T06:51:45.831122+0000","last_fullsized":"2024-06-13T06:51:45.831122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"98'5","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631856+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:53:29.631856+0000","last_peered":"2024-06-13T06:53:29.631856+0000","last_clean":"2024-06-13T06:53:29.631856+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:53:29.631856+0000","last_undegraded":"2024-06-13T06:53:29.631856+0000","last_fullsized":"2024-06-13T06:53:29.631856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.943373+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:51:00.943373+0000","last_peered":"2024-06-13T06:51:00.943373+0000","last_clean":"2024-06-13T06:51:00.943373+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:51:00.943373+0000","last_undegraded":"2024-06-13T06:51:00.943373+0000","last_fullsized":"2024-06-13T06:51:00.943373+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.835140+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:51:45.835140+0000","last_peered":"2024-06-13T06:51:45.835140+0000","last_clean":"2024-06-13T06:51:45.835140+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:51:45.835140+0000","last_undegraded":"2024-06-13T06:51:45.835140+0000","last_fullsized":"2024-06-13T06:51:45.835140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"98'7","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632579+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:53:29.632579+0000","last_peered":"2024-06-13T06:53:29.632579+0000","last_clean":"2024-06-13T06:53:29.632579+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:53:29.632579+0000","last_undegraded":"2024-06-13T06:53:29.632579+0000","last_fullsized":"2024-06-13T06:53:29.632579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.258645+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:51:01.258645+0000","last_peered":"2024-06-13T06:51:01.258645+0000","last_clean":"2024-06-13T06:51:01.258645+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:51:01.258645+0000","last_undegraded":"2024-06-13T06:51:01.258645+0000","last_fullsized":"2024-06-13T06:51:01.258645+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:45.831717+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:51:45.831717+0000","last_peered":"2024-06-13T06:51:45.831717+0000","last_clean":"2024-06-13T06:51:45.831717+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:51:45.831717+0000","last_undegraded":"2024-06-13T06:51:45.831717+0000","last_fullsized":"2024-06-13T06:51:45.831717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"98'6","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629662+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:53:29.629662+0000","last_peered":"2024-06-13T06:53:29.629662+0000","last_clean":"2024-06-13T06:53:29.629662+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:53:29.629662+0000","last_undegraded":"2024-06-13T06:53:29.629662+0000","last_fullsized":"2024-06-13T06:53:29.629662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:01.252489+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:51:01.252489+0000","last_peered":"2024-06-13T06:51:01.252489+0000","last_clean":"2024-06-13T06:51:01.252489+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:51:01.252489+0000","last_undegraded":"2024-06-13T06:51:01.252489+0000","last_fullsized":"2024-06-13T06:51:01.252489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.960980+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:51:00.960980+0000","last_peered":"2024-06-13T06:51:00.960980+0000","last_clean":"2024-06-13T06:51:00.960980+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:51:00.960980+0000","last_undegraded":"2024-06-13T06:51:00.960980+0000","last_fullsized":"2024-06-13T06:51:00.960980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"98'9","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.620947+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:53:29.620947+0000","last_peered":"2024-06-13T06:53:29.620947+0000","last_clean":"2024-06-13T06:53:29.620947+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:53:29.620947+0000","last_undegraded":"2024-06-13T06:53:29.620947+0000","last_fullsized":"2024-06-13T06:53:29.620947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623417+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:53:29.623417+0000","last_peered":"2024-06-13T06:53:29.623417+0000","last_clean":"2024-06-13T06:53:29.623417+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:53:29.623417+0000","last_undegraded":"2024-06-13T06:53:29.623417+0000","last_fullsized":"2024-06-13T06:53:29.623417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"98'12","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623598+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:53:29.623598+0000","last_peered":"2024-06-13T06:53:29.623598+0000","last_clean":"2024-06-13T06:53:29.623598+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:53:29.623598+0000","last_undegraded":"2024-06-13T06:53:29.623598+0000","last_fullsized":"2024-06-13T06:53:29.623598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629587+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:53:29.629587+0000","last_peered":"2024-06-13T06:53:29.629587+0000","last_clean":"2024-06-13T06:53:29.629587+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:53:29.629587+0000","last_undegraded":"2024-06-13T06:53:29.629587+0000","last_fullsized":"2024-06-13T06:53:29.629587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638021+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:53:29.638021+0000","last_peered":"2024-06-13T06:53:29.638021+0000","last_clean":"2024-06-13T06:53:29.638021+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:53:29.638021+0000","last_undegraded":"2024-06-13T06:53:29.638021+0000","last_fullsized":"2024-06-13T06:53:29.638021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"98'4","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631033+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:53:29.631033+0000","last_peered":"2024-06-13T06:53:29.631033+0000","last_clean":"2024-06-13T06:53:29.631033+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:53:29.631033+0000","last_undegraded":"2024-06-13T06:53:29.631033+0000","last_fullsized":"2024-06-13T06:53:29.631033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617246+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:53:29.617246+0000","last_peered":"2024-06-13T06:53:29.617246+0000","last_clean":"2024-06-13T06:53:29.617246+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:53:29.617246+0000","last_undegraded":"2024-06-13T06:53:29.617246+0000","last_fullsized":"2024-06-13T06:53:29.617246+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629337+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:53:29.629337+0000","last_peered":"2024-06-13T06:53:29.629337+0000","last_clean":"2024-06-13T06:53:29.629337+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:53:29.629337+0000","last_undegraded":"2024-06-13T06:53:29.629337+0000","last_fullsized":"2024-06-13T06:53:29.629337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"98'2","reported_seq":37,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629570+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:53:29.629570+0000","last_peered":"2024-06-13T06:53:29.629570+0000","last_clean":"2024-06-13T06:53:29.629570+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:53:29.629570+0000","last_undegraded":"2024-06-13T06:53:29.629570+0000","last_fullsized":"2024-06-13T06:53:29.629570+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639896+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:53:29.639896+0000","last_peered":"2024-06-13T06:53:29.639896+0000","last_clean":"2024-06-13T06:53:29.639896+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:53:29.639896+0000","last_undegraded":"2024-06-13T06:53:29.639896+0000","last_fullsized":"2024-06-13T06:53:29.639896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631619+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:53:29.631619+0000","last_peered":"2024-06-13T06:53:29.631619+0000","last_clean":"2024-06-13T06:53:29.631619+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:53:29.631619+0000","last_undegraded":"2024-06-13T06:53:29.631619+0000","last_fullsized":"2024-06-13T06:53:29.631619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"98'10","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640472+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:53:29.640472+0000","last_peered":"2024-06-13T06:53:29.640472+0000","last_clean":"2024-06-13T06:53:29.640472+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:53:29.640472+0000","last_undegraded":"2024-06-13T06:53:29.640472+0000","last_fullsized":"2024-06-13T06:53:29.640472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"98'11","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632523+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:53:29.632523+0000","last_peered":"2024-06-13T06:53:29.632523+0000","last_clean":"2024-06-13T06:53:29.632523+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:53:29.632523+0000","last_undegraded":"2024-06-13T06:53:29.632523+0000","last_fullsized":"2024-06-13T06:53:29.632523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"98'8","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628229+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:53:29.628229+0000","last_peered":"2024-06-13T06:53:29.628229+0000","last_clean":"2024-06-13T06:53:29.628229+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:53:29.628229+0000","last_undegraded":"2024-06-13T06:53:29.628229+0000","last_fullsized":"2024-06-13T06:53:29.628229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68448,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":743,"num_read_kb":444,"num_write":176,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351035,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32376,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712776,"statfs":{"total":95995035648,"available":95961882624,"internally_reserved":0,"allocated":1228800,"data_stored":1210149,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776197,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037837,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150152,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":613576,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":614926,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554540,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724639,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:53:48.233 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-13T06:53:48.233 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-13T06:53:48.233 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-13T06:53:48.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph health --format=json 2024-06-13T06:53:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:47 smithi189 ceph-mon[36616]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:53:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:49.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:48 smithi175 ceph-mon[36886]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:49.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:48 smithi007 ceph-mon[32077]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:49.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:48 smithi189 ceph-mon[36616]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:50 smithi175 ceph-mon[36886]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 -- 172.21.15.7:0/514445962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac113350 msgr2=0x7f71ac115740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 --2- 172.21.15.7:0/514445962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac113350 0x7f71ac115740 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b0a0 tx=0x7f71a400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 -- 172.21.15.7:0/514445962 shutdown_connections 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 --2- 172.21.15.7:0/514445962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac113350 0x7f71ac115740 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 --2- 172.21.15.7:0/514445962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac0769c0 0x7f71ac076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 --2- 172.21.15.7:0/514445962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0772e0 0x7f71ac112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 -- 172.21.15.7:0/514445962 >> 172.21.15.7:0/514445962 conn(0x7f71ac1001a0 msgr2=0x7f71ac1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 -- 172.21.15.7:0/514445962 shutdown_connections 2024-06-13T06:53:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.127+0000 7f71b2e00700 1 -- 172.21.15.7:0/514445962 wait complete. 2024-06-13T06:53:51.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:50 smithi007 ceph-mon[32077]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.129+0000 7f71b2e00700 1 Processor -- start 2024-06-13T06:53:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.129+0000 7f71b2e00700 1 -- start start 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.129+0000 7f71b2e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 0x7f71ac19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.129+0000 7f71b2e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 0x7f71ac19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.129+0000 7f71b2e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 0x7f71ac1a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b2e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac118de0 con 0x7f71ac0772e0 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b2e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac118c60 con 0x7f71ac113350 2024-06-13T06:53:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b2e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac118f60 con 0x7f71ac0769c0 2024-06-13T06:53:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71abe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 0x7f71ac19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71ab400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 0x7f71ac19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71abe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 0x7f71ac19f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58164/0 (socket says 172.21.15.7:58164) 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71ab400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 0x7f71ac19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36522/0 (socket says 172.21.15.7:36522) 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71abe00700 1 -- 172.21.15.7:0/671927448 learned_addr learned my addr 172.21.15.7:0/671927448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 0x7f71ac1a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 -- 172.21.15.7:0/671927448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 msgr2=0x7f71ac19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 0x7f71ac19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 -- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 msgr2=0x7f71ac19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 0x7f71ac19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.130+0000 7f71b0e00700 1 -- 172.21.15.7:0/671927448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac1a4260 con 0x7f71ac113350 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71b0e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 0x7f71ac1a3b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71ac06bcd0 tx=0x7f71a4004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4021070 con 0x7f71ac113350 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1a44f0 con 0x7f71ac113350 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4007b80 con 0x7f71ac113350 2024-06-13T06:53:51.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4005040 con 0x7f71ac113350 2024-06-13T06:53:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.131+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1abdd0 con 0x7f71ac113350 2024-06-13T06:53:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.133+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac069180 con 0x7f71ac113350 2024-06-13T06:53:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.134+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f71a4024070 con 0x7f71ac113350 2024-06-13T06:53:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.136+0000 7f71a8c00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 0x7f71900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.136+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f71a400f020 con 0x7f71ac113350 2024-06-13T06:53:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.136+0000 7f71abe00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 0x7f71900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.137+0000 7f71abe00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 0x7f71900a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d20 tx=0x7f719c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.138+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71a402b020 con 0x7f71ac113350 2024-06-13T06:53:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:50 smithi189 ceph-mon[36616]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:53:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.425+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f71ac02dca0 con 0x7f71ac113350 2024-06-13T06:53:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.425+0000 7f71a8c00700 1 -- 172.21.15.7:0/671927448 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f71a4007d30 con 0x7f71ac113350 2024-06-13T06:53:51.425 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:53:51.425 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-13T06:53:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.428+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 msgr2=0x7f71900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.428+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 0x7f71900a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d20 tx=0x7f719c009260 comp rx=0 tx=0).stop 2024-06-13T06:53:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.428+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 msgr2=0x7f71ac1a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.428+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 0x7f71ac1a3b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71ac06bcd0 tx=0x7f71a4004d00 comp rx=0 tx=0).stop 2024-06-13T06:53:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 shutdown_connections 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71900a2ca0 0x7f71900a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71ac113350 0x7f71ac1a3b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ac0769c0 0x7f71ac19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 --2- 172.21.15.7:0/671927448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac0772e0 0x7f71ac19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 >> 172.21.15.7:0/671927448 conn(0x7f71ac1001a0 msgr2=0x7f71ac102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 shutdown_connections 2024-06-13T06:53:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:51.429+0000 7f71b2e00700 1 -- 172.21.15.7:0/671927448 wait complete. 2024-06-13T06:53:52.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:51 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/671927448' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:53:52.250 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-13T06:53:52.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:53:52.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:51 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/671927448' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:53:52.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:51 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/671927448' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-13T06:53:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:53.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:52 smithi007 ceph-mon[32077]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-13T06:53:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:52 smithi189 ceph-mon[36616]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-13T06:53:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:52 smithi175 ceph-mon[36886]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-06-13T06:53:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 -- 172.21.15.7:0/1668590825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 msgr2=0x7f54f8111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 --2- 172.21.15.7:0/1668590825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f8111280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b0a0 tx=0x7f54f000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:54 smithi007 ceph-mon[32077]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 -- 172.21.15.7:0/1668590825 shutdown_connections 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 --2- 172.21.15.7:0/1668590825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f8111280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 --2- 172.21.15.7:0/1668590825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f80746c0 0x7f54f8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 --2- 172.21.15.7:0/1668590825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f8074fe0 0x7f54f810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 -- 172.21.15.7:0/1668590825 >> 172.21.15.7:0/1668590825 conn(0x7f54f80795f0 msgr2=0x7f54f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 -- 172.21.15.7:0/1668590825 shutdown_connections 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.071+0000 7f54fd000700 1 -- 172.21.15.7:0/1668590825 wait complete. 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 Processor -- start 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 -- start start 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 0x7f54f819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f81149e0 con 0x7f54f8074fe0 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f8114860 con 0x7f54f810ee90 2024-06-13T06:53:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.073+0000 7f54fd000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54f8114b60 con 0x7f54f80746c0 2024-06-13T06:53:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 0x7f54f819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58182/0 (socket says 172.21.15.7:58182) 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f5e00700 1 -- 172.21.15.7:0/3011135800 learned_addr learned my addr 172.21.15.7:0/3011135800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f6800700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 -- 172.21.15.7:0/3011135800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 msgr2=0x7f54f819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 -- 172.21.15.7:0/3011135800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 msgr2=0x7f54f819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54ed400700 1 -- 172.21.15.7:0/3011135800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f819fe00 con 0x7f54f8074fe0 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f5e00700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.074+0000 7f54f6800700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54ed400700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 0x7f54f819b330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e9c0 tx=0x7f54e000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0004230 con 0x7f54f8074fe0 2024-06-13T06:53:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e00043d0 con 0x7f54f8074fe0 2024-06-13T06:53:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f81a00f0 con 0x7f54f8074fe0 2024-06-13T06:53:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0010430 con 0x7f54f8074fe0 2024-06-13T06:53:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.075+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f81a7970 con 0x7f54f8074fe0 2024-06-13T06:53:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.076+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f8069180 con 0x7f54f8074fe0 2024-06-13T06:53:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.078+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f54e000c7c0 con 0x7f54f8074fe0 2024-06-13T06:53:55.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.080+0000 7f54ef400700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 0x7f54dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:55.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.080+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54e00cf720 con 0x7f54f8074fe0 2024-06-13T06:53:55.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.080+0000 7f54f5e00700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 0x7f54dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:55.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.081+0000 7f54f5e00700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 0x7f54dc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54e8011550 tx=0x7f54e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:55.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.085+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f54e009c2e0 con 0x7f54f8074fe0 2024-06-13T06:53:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:54 smithi189 ceph-mon[36616]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.366+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f81a7b40 con 0x7f54f8074fe0 2024-06-13T06:53:55.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.368+0000 7f54ef400700 1 -- 172.21.15.7:0/3011135800 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7f54e009bbe0 con 0x7f54f8074fe0 2024-06-13T06:53:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:54 smithi175 ceph-mon[36886]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:53:55.368 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:53:55.368 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.371+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 msgr2=0x7f54dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.371+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 0x7f54dc0a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f54e8011550 tx=0x7f54e8009260 comp rx=0 tx=0).stop 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.371+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 msgr2=0x7f54f819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.371+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 0x7f54f819b330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e9c0 tx=0x7f54e000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 shutdown_connections 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54dc0a2c20 0x7f54dc0a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f810ee90 0x7f54f819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54f80746c0 0x7f54f819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 --2- 172.21.15.7:0/3011135800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f8074fe0 0x7f54f819b330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 >> 172.21.15.7:0/3011135800 conn(0x7f54f80795f0 msgr2=0x7f54f80ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 shutdown_connections 2024-06-13T06:53:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:55.372+0000 7f54fd000700 1 -- 172.21.15.7:0/3011135800 wait complete. 2024-06-13T06:53:55.373 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:53:56.139 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:55 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3011135800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:56.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:53:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:55 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3011135800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:56.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:55 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3011135800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:53:56.698 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:53:57.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:56 smithi007 ceph-mon[32077]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:57.268 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:56 smithi175 ceph-mon[36886]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:56 smithi189 ceph-mon[36616]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:53:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.025+0000 7f89c5a00700 1 -- 172.21.15.7:0/1791084333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c006b0a0 msgr2=0x7f89c006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.025+0000 7f89c5a00700 1 --2- 172.21.15.7:0/1791084333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c006b0a0 0x7f89c006b500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009990 tx=0x7f89a8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 -- 172.21.15.7:0/1791084333 shutdown_connections 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 --2- 172.21.15.7:0/1791084333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c010ede0 0x7f89c01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 --2- 172.21.15.7:0/1791084333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c01033a0 0x7f89c006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 --2- 172.21.15.7:0/1791084333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c006b0a0 0x7f89c006b500 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 -- 172.21.15.7:0/1791084333 >> 172.21.15.7:0/1791084333 conn(0x7f89c0078340 msgr2=0x7f89c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 -- 172.21.15.7:0/1791084333 shutdown_connections 2024-06-13T06:53:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.026+0000 7f89c5a00700 1 -- 172.21.15.7:0/1791084333 wait complete. 2024-06-13T06:53:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.027+0000 7f89c5a00700 1 Processor -- start 2024-06-13T06:53:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.027+0000 7f89c5a00700 1 -- start start 2024-06-13T06:53:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 0x7f89c019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0114a10 con 0x7f89c01033a0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0114890 con 0x7f89c010ede0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89c5a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0114b90 con 0x7f89c006b0a0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89b7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 0x7f89c019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89bea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89bf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89bea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58200/0 (socket says 172.21.15.7:58200) 2024-06-13T06:53:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89b7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 0x7f89c019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36544/0 (socket says 172.21.15.7:36544) 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89bea00700 1 -- 172.21.15.7:0/484695500 learned_addr learned my addr 172.21.15.7:0/484695500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.028+0000 7f89bf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42720/0 (socket says 172.21.15.7:42720) 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 -- 172.21.15.7:0/484695500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 msgr2=0x7f89c019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 -- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 msgr2=0x7f89c019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 0x7f89c019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 -- 172.21.15.7:0/484695500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c019feb0 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bf400700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89bea00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9c0 tx=0x7f89b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000cbd0 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b000ee60 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0005630 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.029+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c01a01a0 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.030+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01a7a20 con 0x7f89c006b0a0 2024-06-13T06:53:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.031+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c01019d0 con 0x7f89c006b0a0 2024-06-13T06:53:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.033+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f89b0004750 con 0x7f89c006b0a0 2024-06-13T06:53:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.034+0000 7f89b7400700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 0x7f89a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:53:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.034+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f89b00cf600 con 0x7f89c006b0a0 2024-06-13T06:53:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.035+0000 7f89b7e00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 0x7f89a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:53:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.035+0000 7f89b7e00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 0x7f89a40a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89c0078bd0 tx=0x7f89a8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:53:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.038+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89b009c1c0 con 0x7f89c006b0a0 2024-06-13T06:53:59.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:58 smithi007 ceph-mon[32077]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.324+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89c0069180 con 0x7f89c006b0a0 2024-06-13T06:53:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.325+0000 7f89b7400700 1 -- 172.21.15.7:0/484695500 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7f89b009bac0 con 0x7f89c006b0a0 2024-06-13T06:53:59.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:53:59.325 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.328+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 msgr2=0x7f89a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 0x7f89a40a5160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89c0078bd0 tx=0x7f89a8005a90 comp rx=0 tx=0).stop 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 msgr2=0x7f89c019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9c0 tx=0x7f89b000ecd0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 shutdown_connections 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89a40a2ca0 0x7f89a40a5160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89c010ede0 0x7f89c019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c006b0a0 0x7f89c019aea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 --2- 172.21.15.7:0/484695500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c01033a0 0x7f89c019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:53:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 >> 172.21.15.7:0/484695500 conn(0x7f89c0078340 msgr2=0x7f89c0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:53:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 shutdown_connections 2024-06-13T06:53:59.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:53:59.329+0000 7f89c5a00700 1 -- 172.21.15.7:0/484695500 wait complete. 2024-06-13T06:53:59.330 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:53:59.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:58 smithi189 ceph-mon[36616]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:58 smithi175 ceph-mon[36886]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:53:59.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:53:59 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/484695500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:00.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:54:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:53:59 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/484695500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:53:59 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/484695500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:01.107 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:00 smithi007 ceph-mon[32077]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:00 smithi189 ceph-mon[36616]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:00 smithi175 ceph-mon[36886]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 -- 172.21.15.7:0/3892593086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406aa00 msgr2=0x7f130406ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 --2- 172.21.15.7:0/3892593086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406aa00 0x7f130406ae80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f12f00098f0 tx=0x7f12f0009c00 comp rx=0 tx=0).stop 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 -- 172.21.15.7:0/3892593086 shutdown_connections 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 --2- 172.21.15.7:0/3892593086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f130406b3c0 0x7f130410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 --2- 172.21.15.7:0/3892593086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13041084f0 0x7f13041088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 --2- 172.21.15.7:0/3892593086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406aa00 0x7f130406ae80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 -- 172.21.15.7:0/3892593086 >> 172.21.15.7:0/3892593086 conn(0x7f1304078300 msgr2=0x7f1304078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 -- 172.21.15.7:0/3892593086 shutdown_connections 2024-06-13T06:54:02.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.972+0000 7f130a800700 1 -- 172.21.15.7:0/3892593086 wait complete. 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.973+0000 7f130a800700 1 Processor -- start 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 -- start start 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 0x7f130419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 0x7f130419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13041127c0 con 0x7f130406aa00 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1304112640 con 0x7f13041084f0 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f130a800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1304112940 con 0x7f130406b3c0 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f1303400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f1303400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50444/0 (socket says 172.21.15.7:50444) 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.974+0000 7f1303400700 1 -- 172.21.15.7:0/428699440 learned_addr learned my addr 172.21.15.7:0/428699440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1302a00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 0x7f130419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303e00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 0x7f130419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 -- 172.21.15.7:0/428699440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 msgr2=0x7f130419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 0x7f130419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 -- 172.21.15.7:0/428699440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 msgr2=0x7f130419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 0x7f130419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 -- 172.21.15.7:0/428699440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130419fec0 con 0x7f130406aa00 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1302a00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 0x7f130419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303e00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 0x7f130419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.975+0000 7f1303400700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ec10 tx=0x7f12f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.976+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f400cc90 con 0x7f130406aa00 2024-06-13T06:54:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.976+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f400ce30 con 0x7f130406aa00 2024-06-13T06:54:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.976+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13041a01b0 con 0x7f130406aa00 2024-06-13T06:54:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.976+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4010690 con 0x7f130406aa00 2024-06-13T06:54:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.976+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041a7a30 con 0x7f130406aa00 2024-06-13T06:54:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.977+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1304069180 con 0x7f130406aa00 2024-06-13T06:54:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.979+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f12f4003680 con 0x7f130406aa00 2024-06-13T06:54:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.981+0000 7f12fbe00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 0x7f12e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.981+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f12f40cf6c0 con 0x7f130406aa00 2024-06-13T06:54:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.981+0000 7f1302a00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 0x7f12e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.982+0000 7f1302a00700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 0x7f12e80a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009810 tx=0x7f12f0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:02.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:02 smithi007 ceph-mon[32077]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:02.986+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12f409c280 con 0x7f130406aa00 2024-06-13T06:54:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.217+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f130419c210 con 0x7f130406aa00 2024-06-13T06:54:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.220+0000 7f12fbe00700 1 -- 172.21.15.7:0/428699440 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f12f409bb80 con 0x7f130406aa00 2024-06-13T06:54:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:03.220 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:54:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.223+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 msgr2=0x7f12e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.223+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 0x7f12e80a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009810 tx=0x7f12f0019040 comp rx=0 tx=0).stop 2024-06-13T06:54:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.223+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 msgr2=0x7f130419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.223+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ec10 tx=0x7f12f400ef20 comp rx=0 tx=0).stop 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 shutdown_connections 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f12e80a2c20 0x7f12e80a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13041084f0 0x7f130419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f130406aa00 0x7f130419aeb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 --2- 172.21.15.7:0/428699440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f130406b3c0 0x7f130419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 >> 172.21.15.7:0/428699440 conn(0x7f1304078300 msgr2=0x7f130410d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 shutdown_connections 2024-06-13T06:54:03.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:03.224+0000 7f130a800700 1 -- 172.21.15.7:0/428699440 wait complete. 2024-06-13T06:54:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:02 smithi189 ceph-mon[36616]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:03.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:02 smithi175 ceph-mon[36886]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:04.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:03 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/428699440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:04.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:04.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:03 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/428699440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:04.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:03 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/428699440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:05.091 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:04 smithi007 ceph-mon[32077]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:04 smithi189 ceph-mon[36616]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:04 smithi175 ceph-mon[36886]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 -- 172.21.15.7:0/3857100071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 msgr2=0x7f2000074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 --2- 172.21.15.7:0/3857100071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f2000074aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009990 tx=0x7f1ff0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 -- 172.21.15.7:0/3857100071 shutdown_connections 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 --2- 172.21.15.7:0/3857100071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 0x7f20001112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 --2- 172.21.15.7:0/3857100071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f2000074aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 --2- 172.21.15.7:0/3857100071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.025+0000 7f2006a00700 1 -- 172.21.15.7:0/3857100071 >> 172.21.15.7:0/3857100071 conn(0x7f20000795f0 msgr2=0x7f2000079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.026+0000 7f2006a00700 1 -- 172.21.15.7:0/3857100071 shutdown_connections 2024-06-13T06:54:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.026+0000 7f2006a00700 1 -- 172.21.15.7:0/3857100071 wait complete. 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 Processor -- start 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 -- start start 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f200019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 0x7f200019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2000114a40 con 0x7f20000746c0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20001148c0 con 0x7f200010eef0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.027+0000 7f2006a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2000114bc0 con 0x7f2000074fe0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1ff7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f200019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1ff7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49420/0 (socket says 172.21.15.7:49420) 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1ff7e00700 1 -- 172.21.15.7:0/3906652156 learned_addr learned my addr 172.21.15.7:0/3906652156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f2004a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 0x7f200019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 -- 172.21.15.7:0/3906652156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 msgr2=0x7f200019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 -- 172.21.15.7:0/3906652156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 msgr2=0x7f200019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 0x7f200019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.028+0000 7f1fffe00700 1 -- 172.21.15.7:0/3906652156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200019fe00 con 0x7f20000746c0 2024-06-13T06:54:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f1fffe00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f200019adf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098e0 tx=0x7f1ff000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0027070 con 0x7f20000746c0 2024-06-13T06:54:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001a0090 con 0x7f20000746c0 2024-06-13T06:54:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff0003b60 con 0x7f20000746c0 2024-06-13T06:54:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0022d50 con 0x7f20000746c0 2024-06-13T06:54:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.029+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a7970 con 0x7f20000746c0 2024-06-13T06:54:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.030+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2000075d80 con 0x7f20000746c0 2024-06-13T06:54:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.032+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1ff002a070 con 0x7f20000746c0 2024-06-13T06:54:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.034+0000 7f1ffd600700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 0x7f1fe40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.034+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1ff00d02c0 con 0x7f20000746c0 2024-06-13T06:54:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.034+0000 7f1ff7e00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 0x7f1fe40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.035+0000 7f1ff7e00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 0x7f1fe40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20001109d0 tx=0x7f1fe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.038+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ff005e850 con 0x7f20000746c0 2024-06-13T06:54:07.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:06 smithi007 ceph-mon[32077]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.318+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20001a7b40 con 0x7f20000746c0 2024-06-13T06:54:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.319+0000 7f1ffd600700 1 -- 172.21.15.7:0/3906652156 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7f1ff0013070 con 0x7f20000746c0 2024-06-13T06:54:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:07.319 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 msgr2=0x7f1fe40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 0x7f1fe40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20001109d0 tx=0x7f1fe8009260 comp rx=0 tx=0).stop 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 msgr2=0x7f200019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f200019adf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098e0 tx=0x7f1ff000b8a0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 shutdown_connections 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1fe40a2c20 0x7f1fe40a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200010eef0 0x7f200019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20000746c0 0x7f200019adf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 --2- 172.21.15.7:0/3906652156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2000074fe0 0x7f200019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.322+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 >> 172.21.15.7:0/3906652156 conn(0x7f20000795f0 msgr2=0x7f20000ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.323+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 shutdown_connections 2024-06-13T06:54:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:07.323+0000 7f2006a00700 1 -- 172.21.15.7:0/3906652156 wait complete. 2024-06-13T06:54:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:54:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:06 smithi189 ceph-mon[36616]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:06 smithi175 ceph-mon[36886]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:08.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-13T06:50:19.413292+0000', 'modified': '2024-06-13T06:51:49.856213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24539, 'mds_1': 24527, 'mds_2': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/4202454996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 4202454996}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 4202454996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/2460722721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2460722721}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2460722721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6839/3291148051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 3291148051}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 3291148051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.7:6839/827561506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6838', 'nonce': 827561506}, {'type': 'v1', 'addr': '172.21.15.7:6839', 'nonce': 827561506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.7:6841/1761725822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 1761725822}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 1761725822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6839/4167274347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 4167274347}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 4167274347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24527, 24539, 24521]} 2024-06-13T06:54:08.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-13T06:54:08.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.g status 2024-06-13T06:54:08.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:07 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3906652156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:07 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3906652156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:07 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3906652156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:09.009 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:08 smithi007 ceph-mon[32077]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:08 smithi189 ceph-mon[36616]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:08 smithi175 ceph-mon[36886]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 -- 172.21.15.7:0/1664313609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc108ac0 msgr2=0x7f10dc108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 --2- 172.21.15.7:0/1664313609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc108ac0 0x7f10dc108f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009990 tx=0x7f10c8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 -- 172.21.15.7:0/1664313609 shutdown_connections 2024-06-13T06:54:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 --2- 172.21.15.7:0/1664313609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 --2- 172.21.15.7:0/1664313609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc10eac0 0x7f10dc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 --2- 172.21.15.7:0/1664313609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc108ac0 0x7f10dc108f40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 -- 172.21.15.7:0/1664313609 >> 172.21.15.7:0/1664313609 conn(0x7f10dc075c60 msgr2=0x7f10dc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 -- 172.21.15.7:0/1664313609 shutdown_connections 2024-06-13T06:54:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.920+0000 7f10e1c00700 1 -- 172.21.15.7:0/1664313609 wait complete. 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 Processor -- start 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 -- start start 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc108ac0 0x7f10dc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc118e40 con 0x7f10dc10eac0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc118cc0 con 0x7f10dc108ac0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.922+0000 7f10e1c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc118fc0 con 0x7f10dc109480 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10da000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10daa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc108ac0 0x7f10dc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41252/0 (socket says 172.21.15.7:41252) 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 -- 172.21.15.7:0/2686100268 learned_addr learned my addr 172.21.15.7:0/2686100268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 -- 172.21.15.7:0/2686100268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 msgr2=0x7f10dc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 -- 172.21.15.7:0/2686100268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc108ac0 msgr2=0x7f10dc19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc108ac0 0x7f10dc19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10db400700 1 -- 172.21.15.7:0/2686100268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a4300 con 0x7f10dc10eac0 2024-06-13T06:54:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.923+0000 7f10da000700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10db400700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10d400ef30 tx=0x7f10d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10d3400700 1 -- 172.21.15.7:0/2686100268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d40092e0 con 0x7f10dc10eac0 2024-06-13T06:54:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10d3400700 1 -- 172.21.15.7:0/2686100268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4004bb0 con 0x7f10dc10eac0 2024-06-13T06:54:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10d3400700 1 -- 172.21.15.7:0/2686100268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004280 con 0x7f10dc10eac0 2024-06-13T06:54:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a4590 con 0x7f10dc10eac0 2024-06-13T06:54:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.924+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1abe70 con 0x7f10dc10eac0 2024-06-13T06:54:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.928+0000 7f10d3400700 1 -- 172.21.15.7:0/2686100268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f10d401b030 con 0x7f10dc10eac0 2024-06-13T06:54:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10ec4d1b80 1 Processor -- start 2024-06-13T06:54:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10d3400700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 0x7f10c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10ec4d1b80 1 -- start start 2024-06-13T06:54:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10d3400700 1 -- 172.21.15.7:0/2686100268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f10d40cee30 con 0x7f10dc10eac0 2024-06-13T06:54:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10da000700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 0x7f10c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8df2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.929+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588f8d019c0 con 0x5588f8df9460 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588f8d44ab0 con 0x5588f8df31f0 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588f8cff8a0 con 0x5588f8df8b40 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10daa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10daa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38790/0 (socket says 172.21.15.7:38790) 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10daa00700 1 -- 172.21.15.7:0/500172539 learned_addr learned my addr 172.21.15.7:0/500172539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10da000700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8df2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.930+0000 7f10db400700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 0x7f10c00a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10dc077b90 tx=0x7f10c8009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 -- 172.21.15.7:0/500172539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 msgr2=0x5588f8df8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 -- 172.21.15.7:0/500172539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 msgr2=0x5588f8df3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 -- 172.21.15.7:0/500172539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f8cecf40 con 0x5588f8df9460 2024-06-13T06:54:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10daa00700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.931+0000 7f10da000700 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8df2cb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c8028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f353d07ab9ed0049 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.932+0000 7f10c7e00700 1 -- 172.21.15.7:0/500172539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8005050 con 0x5588f8df9460 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.932+0000 7f10c7e00700 1 -- 172.21.15.7:0/500172539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c802b040 con 0x5588f8df9460 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.932+0000 7f10c7e00700 1 -- 172.21.15.7:0/500172539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8003960 con 0x5588f8df9460 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/500172539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 msgr2=0x5588f8df2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8df2cb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c8028000 comp rx=0 tx=0).stop 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/500172539 shutdown_connections 2024-06-13T06:54:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8df8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/500172539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8df2cb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/500172539 >> 172.21.15.7:0/500172539 conn(0x5588f8d1f120 msgr2=0x5588f8dedfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/500172539 shutdown_connections 2024-06-13T06:54:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/500172539 wait complete. 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.933+0000 7f10ec4d1b80 1 -- start start 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 Processor -- start 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dfc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df9460 0x5588f8dfc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588f8ce0f10 con 0x5588f8df31f0 2024-06-13T06:54:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588f8d40a70 con 0x5588f8df9460 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588f8dffd80 con 0x5588f8df8b40 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.934+0000 7f10ec4d1b80 10 client.0 _ll_register_callbacks cb 0x5588f8cda330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10da000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dfc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41266/0 (socket says 172.21.15.7:41266) 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 -- 172.21.15.7:0/2614318375 learned_addr learned my addr 172.21.15.7:0/2614318375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10db400700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df9460 0x5588f8dfc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 -- 172.21.15.7:0/2614318375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 msgr2=0x5588f8dfc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dfc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 -- 172.21.15.7:0/2614318375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df9460 msgr2=0x5588f8dfc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df9460 0x5588f8dfc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.935+0000 7f10daa00700 1 -- 172.21.15.7:0/2614318375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f8c93880 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10daa00700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10cc011830 tx=0x7f10cc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab22a652b3230814 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00f710 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10c802b040 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f10cc00f8b0 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10cc027050 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10ec4d1b80 10 client.14856 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588f8e0a570 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.936+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c802b040 con 0x5588f8df31f0 2024-06-13T06:54:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10cc022030 con 0x5588f8df31f0 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 10 client.14856 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 10 client.14856 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 20 client.14856 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e0cba0 0x5588f8e0e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 4 client.14856 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f8d51fb0 con 0x5588f8e0cba0 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.937+0000 7f10da000700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e0cba0 0x5588f8e0e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.938+0000 7f10da000700 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e0cba0 0x5588f8e0e010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f52aed45d5da8b5 server_cookie=19c7db55b902c4f5 in_seq=0 out_seq=0 2024-06-13T06:54:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.938+0000 7f10c4c00700 10 client.14856 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:54:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10c4c00700 1 -- 172.21.15.7:0/2614318375 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588f8d51fb0 con 0x5588f8e0cba0 2024-06-13T06:54:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10c4c00700 10 client.14856 handle_command_reply: tid=0 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 client.14856 shutdown 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 msgr2=0x5588f8dff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10cc011830 tx=0x7f10cc011b40 comp rx=0 tx=0).stop 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 shutdown_connections 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e0cba0 0x5588f8e0e010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df9460 0x5588f8dfc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10c4c00700 0 client.14856 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df31f0 0x5588f8dff1b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2614318375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dfc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.939+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 >> 172.21.15.7:0/2614318375 conn(0x5588f8d1f120 msgr2=0x5588f8de7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.940+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 shutdown_connections 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.940+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2614318375 wait complete. 2024-06-13T06:54:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.940+0000 7f10ec4d1b80 20 client.14856 trim_cache size 0 max 16384 2024-06-13T06:54:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.965+0000 7f10ec4d1b80 1 Processor -- start 2024-06-13T06:54:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.965+0000 7f10ec4d1b80 1 -- start start 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.965+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.965+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0c350 con 0x5588f8df9460 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0ca20 con 0x5588f8df31f0 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0d0f0 con 0x5588f8df8b40 2024-06-13T06:54:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10db400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10db400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41274/0 (socket says 172.21.15.7:41274) 2024-06-13T06:54:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10db400700 1 -- 172.21.15.7:0/2226831173 learned_addr learned my addr 172.21.15.7:0/2226831173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10da000700 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.966+0000 7f10daa00700 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 -- 172.21.15.7:0/2226831173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 msgr2=0x5588f8dee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 -- 172.21.15.7:0/2226831173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 msgr2=0x5588f8df2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 -- 172.21.15.7:0/2226831173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f8df8640 con 0x5588f8df9460 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10db400700 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c6f0 tx=0x7f10d40976a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14477a547042f061 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10c7400700 1 -- 172.21.15.7:0/2226831173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d409b380 con 0x5588f8df9460 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10c7400700 1 -- 172.21.15.7:0/2226831173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d409f730 con 0x5588f8df9460 2024-06-13T06:54:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.967+0000 7f10c7400700 1 -- 172.21.15.7:0/2226831173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d40a3ad0 con 0x5588f8df9460 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2226831173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 msgr2=0x5588f8de80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c6f0 tx=0x7f10d40976a0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2226831173 shutdown_connections 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df9460 0x5588f8de80e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8df2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/2226831173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df8b40 0x5588f8dee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2226831173 >> 172.21.15.7:0/2226831173 conn(0x5588f8d1f120 msgr2=0x5588f8deda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.968+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2226831173 shutdown_connections 2024-06-13T06:54:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.969+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/2226831173 wait complete. 2024-06-13T06:54:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.969+0000 7f10ec4d1b80 1 -- start start 2024-06-13T06:54:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.969+0000 7f10ec4d1b80 1 Processor -- start 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df8b40 0x5588f8dfe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 0x5588f8dfea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0de90 con 0x5588f8df8b40 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0d7c0 con 0x5588f8df31f0 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588f8e0e560 con 0x5588f8df9460 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10ec4d1b80 10 client.0 _ll_register_callbacks cb 0x5588f8cda330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10da000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df8b40 0x5588f8dfe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50642/0 (socket says 172.21.15.7:50642) 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 -- 172.21.15.7:0/3563473213 learned_addr learned my addr 172.21.15.7:0/3563473213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10db400700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 0x5588f8dfea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 -- 172.21.15.7:0/3563473213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 msgr2=0x5588f8dfea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 0x5588f8dfea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.970+0000 7f10daa00700 1 -- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df8b40 msgr2=0x5588f8dfe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10daa00700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df8b40 0x5588f8dfe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10daa00700 1 -- 172.21.15.7:0/3563473213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f8e15830 con 0x5588f8df31f0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10db400700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 0x5588f8dfea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10daa00700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f10cc01ad10 tx=0x7f10cc01a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df754dc19ec722a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01aaa0 con 0x5588f8df31f0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d409f730 con 0x5588f8df31f0 2024-06-13T06:54:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.971+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f10cc0050a0 con 0x5588f8df31f0 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10cc005300 con 0x5588f8df31f0 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 10 client.24989 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5588f8e78610 con 0x5588f8df31f0 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d409f730 con 0x5588f8df31f0 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f10cc022040 con 0x5588f8df31f0 2024-06-13T06:54:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 10 client.24989 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 10 client.24989 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 20 client.24989 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e7d130 0x5588f8e7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 4 client.24989 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.972+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "status"}) -- 0x5588f8d4c530 con 0x5588f8e7d130 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.973+0000 7f10da000700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e7d130 0x5588f8e7be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.973+0000 7f10da000700 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e7d130 0x5588f8e7be40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3832ecbd4dde2439 server_cookie=70176d5be3bd6bd1 in_seq=0 out_seq=0 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.973+0000 7f10c4c00700 10 client.24989 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10c4c00700 1 -- 172.21.15.7:0/3563473213 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5588f8d4c530 con 0x5588f8e7d130 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10c4c00700 10 client.24989 handle_command_reply: tid=0 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 client.24989 shutdown 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 msgr2=0x5588f8e17d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f10cc01ad10 tx=0x7f10cc01a6a0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 shutdown_connections 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588f8df9460 0x5588f8dfea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5588f8e7d130 0x5588f8e7be40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5588f8df31f0 0x5588f8e17d10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 --2- 172.21.15.7:0/3563473213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588f8df8b40 0x5588f8dfe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 >> 172.21.15.7:0/3563473213 conn(0x5588f8d1f120 msgr2=0x5588f8df9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10c4c00700 0 client.24989 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 shutdown_connections 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 1 -- 172.21.15.7:0/3563473213 wait complete. 2024-06-13T06:54:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.974+0000 7f10ec4d1b80 20 client.24989 trim_cache size 0 max 16384 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24521, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 149.18667501100001, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 260.93304985100002 2024-06-13T06:54:10.976 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 msgr2=0x7f10c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 0x7f10c00a5160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10dc077b90 tx=0x7f10c8009510 comp rx=0 tx=0).stop 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 msgr2=0x7f10dc1a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10d400ef30 tx=0x7f10d400c3a0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 shutdown_connections 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10dc10eac0 0x7f10dc1a3bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10c00a2ca0 0x7f10c00a5160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc108ac0 0x7f10dc19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 --2- 172.21.15.7:0/2686100268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10dc109480 0x7f10dc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 >> 172.21.15.7:0/2686100268 conn(0x7f10dc075c60 msgr2=0x7f10dc076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 shutdown_connections 2024-06-13T06:54:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:10.975+0000 7f10e1c00700 1 -- 172.21.15.7:0/2686100268 wait complete. 2024-06-13T06:54:11.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:10 smithi007 ceph-mon[32077]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:10 smithi189 ceph-mon[36616]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:10 smithi175 ceph-mon[36886]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:11.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.a status 2024-06-13T06:54:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:12 smithi007 ceph-mon[32077]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:12 smithi189 ceph-mon[36616]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:12 smithi175 ceph-mon[36886]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.412+0000 7f8021c00700 1 -- 172.21.15.7:0/1399570539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c109480 msgr2=0x7f801c115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.412+0000 7f8021c00700 1 --2- 172.21.15.7:0/1399570539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c109480 0x7f801c115990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8014009990 tx=0x7f8014009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 -- 172.21.15.7:0/1399570539 shutdown_connections 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 --2- 172.21.15.7:0/1399570539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c109480 0x7f801c115990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 --2- 172.21.15.7:0/1399570539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c10eac0 0x7f801c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 --2- 172.21.15.7:0/1399570539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 0x7f801c108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 -- 172.21.15.7:0/1399570539 >> 172.21.15.7:0/1399570539 conn(0x7f801c075c60 msgr2=0x7f801c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 -- 172.21.15.7:0/1399570539 shutdown_connections 2024-06-13T06:54:14.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.413+0000 7f8021c00700 1 -- 172.21.15.7:0/1399570539 wait complete. 2024-06-13T06:54:14.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.414+0000 7f8021c00700 1 Processor -- start 2024-06-13T06:54:14.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.414+0000 7f8021c00700 1 -- start start 2024-06-13T06:54:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.414+0000 7f8021c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 0x7f801c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.414+0000 7f8021c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8021c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 0x7f801c1a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8021c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f801c118e00 con 0x7f801c10eac0 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8021c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f801c118c80 con 0x7f801c109480 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8021c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f801c118f80 con 0x7f801c108ac0 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f801aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 0x7f801c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f801b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 0x7f801c1a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50654/0 (socket says 172.21.15.7:50654) 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 -- 172.21.15.7:0/1024987088 learned_addr learned my addr 172.21.15.7:0/1024987088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 -- 172.21.15.7:0/1024987088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 msgr2=0x7f801c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 0x7f801c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 -- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 msgr2=0x7f801c1a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 0x7f801c1a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f8013e00700 1 -- 172.21.15.7:0/1024987088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1a4260 con 0x7f801c109480 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.415+0000 7f801b400700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 0x7f801c1a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8013e00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f800400e9a0 tx=0x7f800400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8013400700 1 -- 172.21.15.7:0/1024987088 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400ca10 con 0x7f801c109480 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1a4550 con 0x7f801c109480 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8013400700 1 -- 172.21.15.7:0/1024987088 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800400cbb0 con 0x7f801c109480 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8013400700 1 -- 172.21.15.7:0/1024987088 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004018680 con 0x7f801c109480 2024-06-13T06:54:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.416+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1abdd0 con 0x7f801c109480 2024-06-13T06:54:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.421+0000 7f8013400700 1 -- 172.21.15.7:0/1024987088 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8004010430 con 0x7f801c109480 2024-06-13T06:54:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.422+0000 7f802c64fb80 1 Processor -- start 2024-06-13T06:54:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.422+0000 7f802c64fb80 1 -- start start 2024-06-13T06:54:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.422+0000 7f8013400700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 0x7f80000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f8013400700 1 -- 172.21.15.7:0/1024987088 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80040cf620 con 0x7f801c109480 2024-06-13T06:54:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c8198bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c8189a9c0 con 0x562c8198c1f0 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c818ddab0 con 0x562c81992460 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f802c64fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562c818988a0 con 0x562c81991b40 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f801b400700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 0x7f80000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f801aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f801aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38852/0 (socket says 172.21.15.7:38852) 2024-06-13T06:54:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f801aa00700 1 -- 172.21.15.7:0/2604860481 learned_addr learned my addr 172.21.15.7:0/2604860481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f8013e00700 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c8198bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.423+0000 7f801b400700 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 -- 172.21.15.7:0/2604860481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 msgr2=0x562c8198bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c8198bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 -- 172.21.15.7:0/2604860481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 msgr2=0x562c8198c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 -- 172.21.15.7:0/2604860481 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c81885f40 con 0x562c81991b40 2024-06-13T06:54:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f801aa00700 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f800c011650 tx=0x7f800c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf7088edecafadb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.424+0000 7f800be00700 1 -- 172.21.15.7:0/2604860481 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c019020 con 0x562c81991b40 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.425+0000 7f801b400700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 0x7f80000a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f801400b2e0 tx=0x7f801401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.425+0000 7f800be00700 1 -- 172.21.15.7:0/2604860481 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800c0079e0 con 0x562c81991b40 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.425+0000 7f800be00700 1 -- 172.21.15.7:0/2604860481 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c015ad0 con 0x562c81991b40 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 -- 172.21.15.7:0/2604860481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 msgr2=0x562c81991f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f800c011650 tx=0x7f800c009920 comp rx=0 tx=0).stop 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 -- 172.21.15.7:0/2604860481 shutdown_connections 2024-06-13T06:54:14.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81991f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2604860481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c8198bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 -- 172.21.15.7:0/2604860481 >> 172.21.15.7:0/2604860481 conn(0x562c818b8120 msgr2=0x562c81986fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 -- 172.21.15.7:0/2604860481 shutdown_connections 2024-06-13T06:54:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.426+0000 7f802c64fb80 1 -- 172.21.15.7:0/2604860481 wait complete. 2024-06-13T06:54:14.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 -- start start 2024-06-13T06:54:14.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 Processor -- start 2024-06-13T06:54:14.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81995460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c819959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c81879f10 con 0x562c8198c1f0 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c818d9a70 con 0x562c81992460 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562c81998e40 con 0x562c81991b40 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.427+0000 7f802c64fb80 10 client.0 _ll_register_callbacks cb 0x562c81873330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f801aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f801aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41318/0 (socket says 172.21.15.7:41318) 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f801aa00700 1 -- 172.21.15.7:0/994415951 learned_addr learned my addr 172.21.15.7:0/994415951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f801b400700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c819959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81995460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 -- 172.21.15.7:0/994415951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 msgr2=0x562c819959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c819959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 -- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 msgr2=0x562c81998160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f8013e00700 1 -- 172.21.15.7:0/994415951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8182c880 con 0x562c81991b40 2024-06-13T06:54:14.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.428+0000 7f801aa00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:14.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8013e00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81995460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8004000c00 tx=0x7f800409c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60babcce0499a903 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8008c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:54:14.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800409fe90 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8004099de0 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800c0079e0 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80040a4750 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f802c64fb80 10 client.24994 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c819a3610 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.429+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800c0079e0 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f80040b4ce0 con 0x562c81991b40 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 10 client.24994 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 10 client.24994 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 20 client.24994 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c819a5ba0 0x562c819a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 4 client.24994 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:54:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c818eafb0 con 0x562c819a5ba0 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.430+0000 7f801aa00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c819a5ba0 0x562c819a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.431+0000 7f801aa00700 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c819a5ba0 0x562c819a7010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2054d8a3c362ee39 server_cookie=d33d1a4ed4237101 in_seq=0 out_seq=0 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.431+0000 7f8008c00700 10 client.24994 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f8008c00700 1 -- 172.21.15.7:0/994415951 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c818eafb0 con 0x562c819a5ba0 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f8008c00700 10 client.24994 handle_command_reply: tid=0 2024-06-13T06:54:14.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 client.24994 shutdown 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 msgr2=0x562c81995460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81995460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8004000c00 tx=0x7f800409c2d0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 shutdown_connections 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81992460 0x562c819959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c81998160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c819a5ba0 0x562c819a7010 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 --2- 172.21.15.7:0/994415951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c81995460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f8008c00700 0 client.24994 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 >> 172.21.15.7:0/994415951 conn(0x562c818b8120 msgr2=0x562c81980740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 shutdown_connections 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.432+0000 7f802c64fb80 1 -- 172.21.15.7:0/994415951 wait complete. 2024-06-13T06:54:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.433+0000 7f802c64fb80 20 client.24994 trim_cache size 0 max 16384 2024-06-13T06:54:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 Processor -- start 2024-06-13T06:54:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 -- start start 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81991b40 0x562c819810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c819a5350 con 0x562c8198c1f0 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c819a5a20 con 0x562c81991b40 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.456+0000 7f802c64fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562c819a60f0 con 0x562c81992460 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f8013e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81991b40 0x562c819810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38866/0 (socket says 172.21.15.7:38866) 2024-06-13T06:54:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41326/0 (socket says 172.21.15.7:41326) 2024-06-13T06:54:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 -- 172.21.15.7:0/2419979893 learned_addr learned my addr 172.21.15.7:0/2419979893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 -- 172.21.15.7:0/2419979893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81991b40 msgr2=0x562c819810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81991b40 0x562c819810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 -- 172.21.15.7:0/2419979893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 msgr2=0x562c8198bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.457+0000 7f801b400700 1 -- 172.21.15.7:0/2419979893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c819886c0 con 0x562c81992460 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.458+0000 7f801b400700 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f8014005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0f491818d3be94 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.458+0000 7f800b400700 1 -- 172.21.15.7:0/2419979893 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400bdc0 con 0x562c81992460 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.458+0000 7f800b400700 1 -- 172.21.15.7:0/2419979893 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014037070 con 0x562c81992460 2024-06-13T06:54:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.458+0000 7f800b400700 1 -- 172.21.15.7:0/2419979893 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014004990 con 0x562c81992460 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 -- 172.21.15.7:0/2419979893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 msgr2=0x562c81997ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80140098e0 tx=0x7f8014005960 comp rx=0 tx=0).stop 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 -- 172.21.15.7:0/2419979893 shutdown_connections 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81992460 0x562c81997ca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c8198c1f0 0x562c8198bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2419979893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c81991b40 0x562c819810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 -- 172.21.15.7:0/2419979893 >> 172.21.15.7:0/2419979893 conn(0x562c818b8120 msgr2=0x562c81993d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 -- 172.21.15.7:0/2419979893 shutdown_connections 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.459+0000 7f802c64fb80 1 -- 172.21.15.7:0/2419979893 wait complete. 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.460+0000 7f802c64fb80 1 -- start start 2024-06-13T06:54:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.460+0000 7f802c64fb80 1 Processor -- start 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.460+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c8198c1f0 0x562c819b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.460+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c819b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f802c64fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f802c64fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562c819a6e90 con 0x562c81992460 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f802c64fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562c819a67c0 con 0x562c8198c1f0 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f802c64fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562c819a7560 con 0x562c81991b40 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f802c64fb80 10 client.0 _ll_register_callbacks cb 0x562c81873330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f801b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f801b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41336/0 (socket says 172.21.15.7:41336) 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f801b400700 1 -- 172.21.15.7:0/2576365808 learned_addr learned my addr 172.21.15.7:0/2576365808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c819b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f801aa00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c8198c1f0 0x562c819b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 -- 172.21.15.7:0/2576365808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c8198c1f0 msgr2=0x562c819b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c8198c1f0 0x562c819b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 -- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 msgr2=0x562c819ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f8013e00700 1 -- 172.21.15.7:0/2576365808 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c819ab980 con 0x562c81991b40 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.461+0000 7f801b400700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f8013e00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c819b09d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8004099d90 tx=0x7f800409b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48565141148ba12 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f8008c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800409fe90 con 0x562c81991b40 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80040a42c0 con 0x562c81991b40 2024-06-13T06:54:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8014037070 con 0x562c81991b40 2024-06-13T06:54:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80040a44c0 con 0x562c81991b40 2024-06-13T06:54:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f802c64fb80 10 client.25000 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.462+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c81a11b70 con 0x562c81991b40 2024-06-13T06:54:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8014037070 con 0x562c81991b40 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f80040b4ce0 con 0x562c81991b40 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 10 client.25000 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 10 client.25000 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 20 client.25000 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c81a16750 0x562c81a15460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 4 client.25000 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "status"}) -- 0x562c818e5530 con 0x562c81a16750 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.463+0000 7f801aa00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c81a16750 0x562c81a15460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f801aa00700 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c81a16750 0x562c81a15460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa0b7db3a8a944a7 server_cookie=4e24b38d5e7ea628 in_seq=0 out_seq=0 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f8008c00700 10 client.25000 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f8008c00700 1 -- 172.21.15.7:0/2576365808 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562c818e5530 con 0x562c81a16750 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f8008c00700 10 client.25000 handle_command_reply: tid=0 2024-06-13T06:54:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f802c64fb80 1 client.25000 shutdown 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 msgr2=0x562c819b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.464+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c819b09d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8004099d90 tx=0x7f800409b370 comp rx=0 tx=0).stop 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 shutdown_connections 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c81992460 0x562c819ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562c8198c1f0 0x562c819b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x562c81a16750 0x562c81a15460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 --2- 172.21.15.7:0/2576365808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c81991b40 0x562c819b09d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 >> 172.21.15.7:0/2576365808 conn(0x562c818b8120 msgr2=0x562c81986a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f8008c00700 0 client.25000 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 shutdown_connections 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 1 -- 172.21.15.7:0/2576365808 wait complete. 2024-06-13T06:54:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.465+0000 7f802c64fb80 20 client.25000 trim_cache size 0 max 16384 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24527, 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 153.697304175, 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:54:14.467 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 259.85639274599998 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 msgr2=0x7f80000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 0x7f80000a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f801400b2e0 tx=0x7f801401a040 comp rx=0 tx=0).stop 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 msgr2=0x7f801c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f800400e9a0 tx=0x7f800400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 shutdown_connections 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f801c10eac0 0x7f801c1a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f801c108ac0 0x7f801c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80000a2c20 0x7f80000a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 --2- 172.21.15.7:0/1024987088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f801c109480 0x7f801c19f790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 >> 172.21.15.7:0/1024987088 conn(0x7f801c075c60 msgr2=0x7f801c076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 shutdown_connections 2024-06-13T06:54:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:14.466+0000 7f8021c00700 1 -- 172.21.15.7:0/1024987088 wait complete. 2024-06-13T06:54:15.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:14 smithi007 ceph-mon[32077]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:14 smithi189 ceph-mon[36616]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:15.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.b status 2024-06-13T06:54:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:14 smithi175 ceph-mon[36886]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:15.809 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:17.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:16 smithi007 ceph-mon[32077]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:16 smithi189 ceph-mon[36616]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:16 smithi175 ceph-mon[36886]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 -- 172.21.15.7:0/506459136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9848103ef0 msgr2=0x7f984810c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 --2- 172.21.15.7:0/506459136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9848103ef0 0x7f984810c3e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9834009990 tx=0x7f9834009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 -- 172.21.15.7:0/506459136 shutdown_connections 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 --2- 172.21.15.7:0/506459136 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 0x7f984810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 --2- 172.21.15.7:0/506459136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98481035d0 0x7f98481039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 --2- 172.21.15.7:0/506459136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9848103ef0 0x7f984810c3e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 -- 172.21.15.7:0/506459136 >> 172.21.15.7:0/506459136 conn(0x7f98480fce20 msgr2=0x7f98480ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 -- 172.21.15.7:0/506459136 shutdown_connections 2024-06-13T06:54:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.147+0000 7f984d000700 1 -- 172.21.15.7:0/506459136 wait complete. 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.148+0000 7f984d000700 1 Processor -- start 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 -- start start 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98481035d0 0x7f984819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 0x7f984819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9848112770 con 0x7f98481035d0 2024-06-13T06:54:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98481125f0 con 0x7f984810ca00 2024-06-13T06:54:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f984d000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98481128f0 con 0x7f9848103ef0 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f9847400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98481035d0 0x7f984819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38884/0 (socket says 172.21.15.7:38884) 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.149+0000 7f9846a00700 1 -- 172.21.15.7:0/696299257 learned_addr learned my addr 172.21.15.7:0/696299257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9847e00700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 0x7f984819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 -- 172.21.15.7:0/696299257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 msgr2=0x7f984819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 0x7f984819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 -- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98481035d0 msgr2=0x7f984819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98481035d0 0x7f984819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 -- 172.21.15.7:0/696299257 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819fe70 con 0x7f9848103ef0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.150+0000 7f9846a00700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f9834005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.151+0000 7f983fe00700 1 -- 172.21.15.7:0/696299257 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834027070 con 0x7f9848103ef0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.151+0000 7f983fe00700 1 -- 172.21.15.7:0/696299257 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9834003b60 con 0x7f9848103ef0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.151+0000 7f983fe00700 1 -- 172.21.15.7:0/696299257 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834022d40 con 0x7f9848103ef0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.151+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481a0100 con 0x7f9848103ef0 2024-06-13T06:54:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.151+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a79e0 con 0x7f9848103ef0 2024-06-13T06:54:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.154+0000 7f983fe00700 1 -- 172.21.15.7:0/696299257 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f983402a070 con 0x7f9848103ef0 2024-06-13T06:54:18.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.155+0000 7f983fe00700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 0x7f982c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f983fe00700 1 -- 172.21.15.7:0/696299257 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f98340d0220 con 0x7f9848103ef0 2024-06-13T06:54:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 Processor -- start 2024-06-13T06:54:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9847400700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 0x7f982c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 -- start start 2024-06-13T06:54:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bbd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571f6acc9c0 con 0x5571f6bc84e0 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571f6b0fab0 con 0x5571f6bbcd10 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.156+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571f6aca8a0 con 0x5571f6bbd630 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9847400700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 0x7f982c0a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9838009c60 tx=0x7f9838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38896/0 (socket says 172.21.15.7:38896) 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 -- 172.21.15.7:0/604674978 learned_addr learned my addr 172.21.15.7:0/604674978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9847e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9847400700 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bbd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 -- 172.21.15.7:0/604674978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 msgr2=0x5571f6bbd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bbd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 -- 172.21.15.7:0/604674978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 msgr2=0x5571f6bca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.157+0000 7f9846a00700 1 -- 172.21.15.7:0/604674978 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f6ab7f40 con 0x5571f6bbd630 2024-06-13T06:54:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.158+0000 7f9846a00700 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9834005cb0 tx=0x7f98340227a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bedd54666e36a3d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.158+0000 7f983cc00700 1 -- 172.21.15.7:0/604674978 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98340a3070 con 0x5571f6bbd630 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.158+0000 7f983cc00700 1 -- 172.21.15.7:0/604674978 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983409ce70 con 0x5571f6bbd630 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.158+0000 7f983cc00700 1 -- 172.21.15.7:0/604674978 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834087170 con 0x5571f6bbd630 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 -- 172.21.15.7:0/604674978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 msgr2=0x5571f6bc7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9834005cb0 tx=0x7f98340227a0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 -- 172.21.15.7:0/604674978 shutdown_connections 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bbd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/604674978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc7fa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 -- 172.21.15.7:0/604674978 >> 172.21.15.7:0/604674978 conn(0x5571f6aea120 msgr2=0x5571f6bb9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 -- 172.21.15.7:0/604674978 shutdown_connections 2024-06-13T06:54:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.159+0000 7f9858fa6b80 1 -- 172.21.15.7:0/604674978 wait complete. 2024-06-13T06:54:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.160+0000 7f9858fa6b80 1 -- start start 2024-06-13T06:54:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.160+0000 7f9858fa6b80 1 Processor -- start 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.160+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bc7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bc55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571f6aabf10 con 0x5571f6bc84e0 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571f6b0ba70 con 0x5571f6bbcd10 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571f6aca8a0 con 0x5571f6bbd630 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9858fa6b80 10 client.0 _ll_register_callbacks cb 0x5571f6aa5330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bc55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9846a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38908/0 (socket says 172.21.15.7:38908) 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9846a00700 1 -- 172.21.15.7:0/2284394289 learned_addr learned my addr 172.21.15.7:0/2284394289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bc7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 -- 172.21.15.7:0/2284394289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 msgr2=0x5571f6bc50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 -- 172.21.15.7:0/2284394289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 msgr2=0x5571f6bc55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bc55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9847400700 1 -- 172.21.15.7:0/2284394289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f6a5e880 con 0x5571f6bbcd10 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.161+0000 7f9846a00700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9847400700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bc7da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9838009b20 tx=0x7f98380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f86c1a5520c6ecd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9831600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983801d620 con 0x5571f6bbcd10 2024-06-13T06:54:18.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983409ce70 con 0x5571f6bbcd10 2024-06-13T06:54:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f983801d7c0 con 0x5571f6bbcd10 2024-06-13T06:54:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f983801d9c0 con 0x5571f6bbcd10 2024-06-13T06:54:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9858fa6b80 10 client.25019 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.162+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571f6bd5540 con 0x5571f6bbcd10 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983409ce70 con 0x5571f6bbcd10 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9838005140 con 0x5571f6bbcd10 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 10 client.25019 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 10 client.25019 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 20 client.25019 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6bd7b70 0x5571f6bd8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 4 client.25019 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.163+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571f6b1cfb0 con 0x5571f6bd7b70 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.164+0000 7f9846a00700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6bd7b70 0x5571f6bd8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.164+0000 7f9846a00700 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6bd7b70 0x5571f6bd8fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4106a805354bd7 server_cookie=c263fac15a3a96ac in_seq=0 out_seq=0 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.164+0000 7f9831600700 10 client.25019 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9831600700 1 -- 172.21.15.7:0/2284394289 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5571f6b1cfb0 con 0x5571f6bd7b70 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9831600700 10 client.25019 handle_command_reply: tid=0 2024-06-13T06:54:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 client.25019 shutdown 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 msgr2=0x5571f6bc7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bc7da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9838009b20 tx=0x7f98380047c0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 shutdown_connections 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6bd7b70 0x5571f6bd8fe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bc84e0 0x5571f6bc55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbcd10 0x5571f6bc7da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9831600700 0 client.25019 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/2284394289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbd630 0x5571f6bc50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 >> 172.21.15.7:0/2284394289 conn(0x5571f6aea120 msgr2=0x5571f6bb8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.165+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 shutdown_connections 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.166+0000 7f9858fa6b80 1 -- 172.21.15.7:0/2284394289 wait complete. 2024-06-13T06:54:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.166+0000 7f9858fa6b80 20 client.25019 trim_cache size 0 max 16384 2024-06-13T06:54:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 Processor -- start 2024-06-13T06:54:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 -- start start 2024-06-13T06:54:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbcd10 0x5571f6bb9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 0x5571f6bc78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd7320 con 0x5571f6bbcd10 2024-06-13T06:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd79f0 con 0x5571f6bbd630 2024-06-13T06:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.190+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd80c0 con 0x5571f6bc84e0 2024-06-13T06:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9847e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 0x5571f6bc78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50740/0 (socket says 172.21.15.7:50740) 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 -- 172.21.15.7:0/3732149550 learned_addr learned my addr 172.21.15.7:0/3732149550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9847400700 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbcd10 0x5571f6bb9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 -- 172.21.15.7:0/3732149550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 msgr2=0x5571f6bc78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 0x5571f6bc78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 -- 172.21.15.7:0/3732149550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbcd10 msgr2=0x5571f6bb9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbcd10 0x5571f6bb9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9846a00700 1 -- 172.21.15.7:0/3732149550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f6bbeb60 con 0x5571f6bbd630 2024-06-13T06:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.191+0000 7f9847e00700 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 0x5571f6bc78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.192+0000 7f9846a00700 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9834022750 tx=0x7f983401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae3cfa3f94ad1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.192+0000 7f9833e00700 1 -- 172.21.15.7:0/3732149550 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98340a5350 con 0x5571f6bbd630 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.192+0000 7f9833e00700 1 -- 172.21.15.7:0/3732149550 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983409aa70 con 0x5571f6bbd630 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.192+0000 7f9833e00700 1 -- 172.21.15.7:0/3732149550 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983409cc30 con 0x5571f6bbd630 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- 172.21.15.7:0/3732149550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 msgr2=0x5571f6bc0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9834022750 tx=0x7f983401a040 comp rx=0 tx=0).stop 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- 172.21.15.7:0/3732149550 shutdown_connections 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bc84e0 0x5571f6bc78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbcd10 0x5571f6bb9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/3732149550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bbd630 0x5571f6bc0600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- 172.21.15.7:0/3732149550 >> 172.21.15.7:0/3732149550 conn(0x5571f6aea120 msgr2=0x5571f6bbfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- 172.21.15.7:0/3732149550 shutdown_connections 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- 172.21.15.7:0/3732149550 wait complete. 2024-06-13T06:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.193+0000 7f9858fa6b80 1 -- start start 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 Processor -- start 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbd630 0x5571f6be2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd8e60 con 0x5571f6bbd630 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd8790 con 0x5571f6bc84e0 2024-06-13T06:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571f6bd9530 con 0x5571f6bbcd10 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.194+0000 7f9858fa6b80 10 client.0 _ll_register_callbacks cb 0x5571f6aa5330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9846a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbd630 0x5571f6be2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50744/0 (socket says 172.21.15.7:50744) 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38926/0 (socket says 172.21.15.7:38926) 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 -- 172.21.15.7:0/1078317919 learned_addr learned my addr 172.21.15.7:0/1078317919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 -- 172.21.15.7:0/1078317919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 msgr2=0x5571f6be26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 -- 172.21.15.7:0/1078317919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbd630 msgr2=0x5571f6be2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbd630 0x5571f6be2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 -- 172.21.15.7:0/1078317919 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f6bc21a0 con 0x5571f6bc84e0 2024-06-13T06:54:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847400700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9847e00700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9840011df0 tx=0x7f98400077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d71ac9fae32261b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.195+0000 7f9831600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840025060 con 0x5571f6bc84e0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983409aa70 con 0x5571f6bc84e0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9840020410 con 0x5571f6bc84e0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f984002c050 con 0x5571f6bc84e0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9858fa6b80 10 client.25025 fetch_fsmap learned FSMap version 25 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571f6be2060 con 0x5571f6bc84e0 2024-06-13T06:54:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983409aa70 con 0x5571f6bc84e0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.196+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9840015390 con 0x5571f6bc84e0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 10 client.25025 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 10 client.25025 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 20 client.25025 populate_metadata read hostname 'smithi007' 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6c47be0 0x5571f6c468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 4 client.25025 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "status"}) -- 0x5571f6b17530 con 0x5571f6c47be0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.197+0000 7f9847400700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6c47be0 0x5571f6c468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9847400700 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6c47be0 0x5571f6c468f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffef454d6bad41a server_cookie=9a8875b87fb3e82 in_seq=0 out_seq=0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9831600700 10 client.25025 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9831600700 1 -- 172.21.15.7:0/1078317919 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5571f6b17530 con 0x5571f6c47be0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9831600700 10 client.25025 handle_command_reply: tid=0 2024-06-13T06:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 client.25025 shutdown 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 msgr2=0x5571f6bc1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9840011df0 tx=0x7f98400077e0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 shutdown_connections 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5571f6c47be0 0x5571f6c468f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571f6bc84e0 0x5571f6bc1930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571f6bbcd10 0x5571f6be26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 --2- 172.21.15.7:0/1078317919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571f6bbd630 0x5571f6be2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 >> 172.21.15.7:0/1078317919 conn(0x5571f6aea120 msgr2=0x5571f6bc0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9831600700 0 client.25025 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 shutdown_connections 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.198+0000 7f9858fa6b80 1 -- 172.21.15.7:0/1078317919 wait complete. 2024-06-13T06:54:18.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f9858fa6b80 20 client.25025 trim_cache size 0 max 16384 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24539, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 238.69712017200001, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 260.57419171700002 2024-06-13T06:54:18.201 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 msgr2=0x7f982c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 0x7f982c0a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9838009c60 tx=0x7f9838009260 comp rx=0 tx=0).stop 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 msgr2=0x7f984819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f9834005810 comp rx=0 tx=0).stop 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 shutdown_connections 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f982c0a2c20 0x7f982c0a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f984810ca00 0x7f984819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98481035d0 0x7f984819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 --2- 172.21.15.7:0/696299257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9848103ef0 0x7f984819b3a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.199+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 >> 172.21.15.7:0/696299257 conn(0x7f98480fce20 msgr2=0x7f98480ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.200+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 shutdown_connections 2024-06-13T06:54:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:18.200+0000 7f984d000700 1 -- 172.21.15.7:0/696299257 wait complete. 2024-06-13T06:54:18.988 INFO:tasks.cephadm:Setup complete, yielding 2024-06-13T06:54:18.989 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-13T06:54:19.002 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:54:19.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch status' 2024-06-13T06:54:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:18 smithi189 ceph-mon[36616]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:18 smithi175 ceph-mon[36886]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:19.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:18 smithi007 ceph-mon[32077]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:20 smithi189 ceph-mon[36616]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:20 smithi175 ceph-mon[36886]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:21.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:20 smithi007 ceph-mon[32077]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 -- 172.21.15.7:0/3978608285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1041e0 msgr2=0x7f06dc10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 --2- 172.21.15.7:0/3978608285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1041e0 0x7f06dc10aa90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b0d0 tx=0x7f06d400b3e0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 -- 172.21.15.7:0/3978608285 shutdown_connections 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 --2- 172.21.15.7:0/3978608285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1041e0 0x7f06dc10aa90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 --2- 172.21.15.7:0/3978608285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc1007c0 0x7f06dc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 --2- 172.21.15.7:0/3978608285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc103840 0x7f06dc103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 -- 172.21.15.7:0/3978608285 >> 172.21.15.7:0/3978608285 conn(0x7f06dc0fc120 msgr2=0x7f06dc0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 -- 172.21.15.7:0/3978608285 shutdown_connections 2024-06-13T06:54:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.916+0000 7f06e1200700 1 -- 172.21.15.7:0/3978608285 wait complete. 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.917+0000 7f06e1200700 1 Processor -- start 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.917+0000 7f06e1200700 1 -- start start 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc103840 0x7f06dc108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 0x7f06dc104f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10e0b0 con 0x7f06dc1007c0 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10df30 con 0x7f06dc1041e0 2024-06-13T06:54:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06e1200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10e230 con 0x7f06dc103840 2024-06-13T06:54:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40484/0 (socket says 172.21.15.7:40484) 2024-06-13T06:54:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06d9e00700 1 -- 172.21.15.7:0/3835440862 learned_addr learned my addr 172.21.15.7:0/3835440862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06d9400700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc103840 0x7f06dc108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.918+0000 7f06da800700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 0x7f06dc104f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 -- 172.21.15.7:0/3835440862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc103840 msgr2=0x7f06dc108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc103840 0x7f06dc108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 -- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 msgr2=0x7f06dc1083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 -- 172.21.15.7:0/3835440862 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc105820 con 0x7f06dc1041e0 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06d9e00700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06da800700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 0x7f06dc104f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d010 tx=0x7f06d4007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4021020 con 0x7f06dc1041e0 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc1a6ce0 con 0x7f06dc1041e0 2024-06-13T06:54:21.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d4004c50 con 0x7f06dc1041e0 2024-06-13T06:54:21.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4005290 con 0x7f06dc1041e0 2024-06-13T06:54:21.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.919+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc1a7080 con 0x7f06dc1041e0 2024-06-13T06:54:21.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.920+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc069180 con 0x7f06dc1041e0 2024-06-13T06:54:21.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.923+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f06d4024020 con 0x7f06dc1041e0 2024-06-13T06:54:21.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.924+0000 7f06d2a00700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 0x7f06c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:21.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.924+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f06d400f020 con 0x7f06dc1041e0 2024-06-13T06:54:21.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.924+0000 7f06d9e00700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 0x7f06c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:21.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.925+0000 7f06d9e00700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 0x7f06c00a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:21.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:21.926+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f06d409dae0 con 0x7f06dc1041e0 2024-06-13T06:54:22.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.166+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f06dc102d50 con 0x7f06c00a2c20 2024-06-13T06:54:22.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.168+0000 7f06d2a00700 1 -- 172.21.15.7:0/3835440862 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f06dc102d50 con 0x7f06c00a2c20 2024-06-13T06:54:22.168 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-06-13T06:54:22.168 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-06-13T06:54:22.168 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-06-13T06:54:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.171+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 msgr2=0x7f06c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.171+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 0x7f06c00a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc009260 comp rx=0 tx=0).stop 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.171+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 msgr2=0x7f06dc104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.171+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 0x7f06dc104f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d010 tx=0x7f06d4007a20 comp rx=0 tx=0).stop 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 shutdown_connections 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06c00a2c20 0x7f06c00a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06dc1041e0 0x7f06dc104f50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06dc1007c0 0x7f06dc1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 --2- 172.21.15.7:0/3835440862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06dc103840 0x7f06dc108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 >> 172.21.15.7:0/3835440862 conn(0x7f06dc0fc120 msgr2=0x7f06dc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 shutdown_connections 2024-06-13T06:54:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:22.172+0000 7f06e1200700 1 -- 172.21.15.7:0/3835440862 wait complete. 2024-06-13T06:54:23.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-13T06:54:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:22 smithi189 ceph-mon[36616]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:22 smithi189 ceph-mon[36616]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:22 smithi175 ceph-mon[36886]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:22 smithi175 ceph-mon[36886]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:22 smithi007 ceph-mon[32077]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:23.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:22 smithi007 ceph-mon[32077]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:24.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:23 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:24.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:23 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:24.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:23 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:54:25.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:25 smithi007 ceph-mon[32077]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:25.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:25 smithi189 ceph-mon[36616]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:25.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:25 smithi175 ceph-mon[36886]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:25.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.822+0000 7f741e600700 1 -- 172.21.15.7:0/620122905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 msgr2=0x7f7418108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.822+0000 7f741e600700 1 --2- 172.21.15.7:0/620122905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f7418108f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74080099c0 tx=0x7f7408009cd0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 -- 172.21.15.7:0/620122905 shutdown_connections 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 --2- 172.21.15.7:0/620122905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f741810a6d0 0x7f7418110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 --2- 172.21.15.7:0/620122905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f7418108f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 --2- 172.21.15.7:0/620122905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7418109d30 0x7f741810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 -- 172.21.15.7:0/620122905 >> 172.21.15.7:0/620122905 conn(0x7f7418077b00 msgr2=0x7f7418079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 -- 172.21.15.7:0/620122905 shutdown_connections 2024-06-13T06:54:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.823+0000 7f741e600700 1 -- 172.21.15.7:0/620122905 wait complete. 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.824+0000 7f741e600700 1 Processor -- start 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.824+0000 7f741e600700 1 -- start start 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.824+0000 7f741e600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f741819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.824+0000 7f741e600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7418109d30 0x7f741819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.824+0000 7f741e600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 0x7f74181a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f741e600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7418114710 con 0x7f741810a6d0 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f741e600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7418114590 con 0x7f7418108b30 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f741e600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7418114890 con 0x7f7418109d30 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 0x7f74181a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 0x7f74181a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40492/0 (socket says 172.21.15.7:40492) 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f741819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417e00700 1 -- 172.21.15.7:0/1835176126 learned_addr learned my addr 172.21.15.7:0/1835176126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f740ea00700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7418109d30 0x7f741819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 -- 172.21.15.7:0/1835176126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7418109d30 msgr2=0x7f741819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7418109d30 0x7f741819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 -- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 msgr2=0x7f74181a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 0x7f74181a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.825+0000 7f7417400700 1 -- 172.21.15.7:0/1835176126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181a6960 con 0x7f7418108b30 2024-06-13T06:54:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f7417400700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f741819ed10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74080098e0 tx=0x7f740800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408009e60 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7408003e30 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181a6bf0 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740800fd90 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.826+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181a70d0 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.827+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7418069180 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.829+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f740800f4f0 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.831+0000 7f7414c00700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 0x7f73fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.831+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f74080d01a0 con 0x7f7418108b30 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.831+0000 7f740ea00700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 0x7f73fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.832+0000 7f740ea00700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 0x7f73fc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7400009c60 tx=0x7f7400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:25.833+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7408013070 con 0x7f7418108b30 2024-06-13T06:54:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.076+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f74180628d0 con 0x7f73fc0a2c20 2024-06-13T06:54:26.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:26.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:26.079 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:54:26.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.125+0000 7f7414c00700 1 -- 172.21.15.7:0/1835176126 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f74180628d0 con 0x7f73fc0a2c20 2024-06-13T06:54:26.126 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-13T06:54:26.126 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (4m) 2m ago 4m 16.2M - 19.0.0-2946-g3d89accf b045ac545f82 37bb32521bb1 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi175 running (4m) 2m ago 4m 25.9M - 19.0.0-2946-g3d89accf b045ac545f82 539679fdd791 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi189 running (4m) 2m ago 4m 15.3M - 19.0.0-2946-g3d89accf b045ac545f82 16209d14c56c 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (4m) 2m ago 4m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 78b13c73d997 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi175 running (4m) 2m ago 4m 17.0M - 19.0.0-2946-g3d89accf b045ac545f82 2a5dcef6be2b 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi189 running (4m) 2m ago 4m 15.9M - 19.0.0-2946-g3d89accf b045ac545f82 bc1e457d6d8e 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (4m) 2m ago 4m 16.1M - 19.0.0-2946-g3d89accf b045ac545f82 5cef0ce4949f 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi175 running (4m) 2m ago 4m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 7e885081543e 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi189 running (4m) 2m ago 4m 19.5M - 19.0.0-2946-g3d89accf b045ac545f82 cf002acecaac 2024-06-13T06:54:26.127 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (5m) 2m ago 5m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 33bb3558c20d 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi175 running (5m) 2m ago 5m 17.1M - 19.0.0-2946-g3d89accf b045ac545f82 ee57faf52901 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi189 running (5m) 2m ago 5m 16.9M - 19.0.0-2946-g3d89accf b045ac545f82 17d6ac0c5a2b 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (13m) 2m ago 13m 517M - 19.0.0-2946-g3d89accf b045ac545f82 0f285255b691 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi175 *:8443,8765 running (11m) 2m ago 11m 454M - 19.0.0-2946-g3d89accf b045ac545f82 fc9ef54016b9 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi189 *:8443,8765 running (11m) 2m ago 11m 453M - 19.0.0-2946-g3d89accf b045ac545f82 94ce2ec073c9 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (13m) 2m ago 13m 50.1M 2048M 19.0.0-2946-g3d89accf b045ac545f82 1f9fe77f8603 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi175 running (11m) 2m ago 11m 42.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 c3844d35f6f4 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi189 running (11m) 2m ago 11m 43.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 462d97e6aa14 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 2m ago 10m 57.0M 4273M 19.0.0-2946-g3d89accf b045ac545f82 733eaa5a3d07 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi175 running (10m) 2m ago 10m 58.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 72daf7692a12 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi189 running (9m) 2m ago 9m 58.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 51f7ec104a1c 2024-06-13T06:54:26.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (9m) 2m ago 9m 43.5M 4273M 19.0.0-2946-g3d89accf b045ac545f82 ecb354582b20 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi175 running (8m) 2m ago 8m 52.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 298f66c6cc65 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi189 running (8m) 2m ago 8m 59.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c823c0ed8510 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (7m) 2m ago 7m 41.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 cc31d7521250 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi175 running (7m) 2m ago 7m 57.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 14a1204d1ae9 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi189 running (7m) 2m ago 7m 43.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 18dcbe4c70eb 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (6m) 2m ago 6m 53.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 e06cad935ca1 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi175 running (5m) 2m ago 5m 53.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 30f3703a1fe7 2024-06-13T06:54:26.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi189 running (5m) 2m ago 5m 53.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a336930347a2 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 msgr2=0x7f73fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 0x7f73fc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7400009c60 tx=0x7f7400009260 comp rx=0 tx=0).stop 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 msgr2=0x7f741819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f741819ed10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74080098e0 tx=0x7f740800bf60 comp rx=0 tx=0).stop 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 shutdown_connections 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f741810a6d0 0x7f74181a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73fc0a2c20 0x7f73fc0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7418108b30 0x7f741819ed10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:26.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 --2- 172.21.15.7:0/1835176126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7418109d30 0x7f741819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:26.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.128+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 >> 172.21.15.7:0/1835176126 conn(0x7f7418077b00 msgr2=0x7f741810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:26.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.129+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 shutdown_connections 2024-06-13T06:54:26.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:26.129+0000 7f741e600700 1 -- 172.21.15.7:0/1835176126 wait complete. 2024-06-13T06:54:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:54:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:54:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:54:26.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:54:27.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-13T06:54:27.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:27 smithi189 ceph-mon[36616]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:27.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:27 smithi189 ceph-mon[36616]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:27.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:27 smithi175 ceph-mon[36886]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:27.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:27 smithi175 ceph-mon[36886]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:27.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:27 smithi007 ceph-mon[32077]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:27.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:27 smithi007 ceph-mon[32077]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:29.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:29 smithi007 ceph-mon[32077]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:29 smithi189 ceph-mon[36616]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:29.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:29 smithi175 ceph-mon[36886]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.800+0000 7fa0b0a00700 1 -- 172.21.15.7:0/3509751781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 msgr2=0x7fa0ac105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.800+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/3509751781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac105370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009990 tx=0x7fa09c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 -- 172.21.15.7:0/3509751781 shutdown_connections 2024-06-13T06:54:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/3509751781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1081e0 0x7fa0ac10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/3509751781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac105370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/3509751781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 -- 172.21.15.7:0/3509751781 >> 172.21.15.7:0/3509751781 conn(0x7fa0ac0fc9b0 msgr2=0x7fa0ac0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 -- 172.21.15.7:0/3509751781 shutdown_connections 2024-06-13T06:54:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.801+0000 7fa0b0a00700 1 -- 172.21.15.7:0/3509751781 wait complete. 2024-06-13T06:54:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 Processor -- start 2024-06-13T06:54:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 -- start start 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1081e0 0x7fa0ac101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac10e150 con 0x7fa0ac102f80 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac10dfd0 con 0x7fa0ac1081e0 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0b0a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac10e2d0 con 0x7fa0ac1058b0 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.802+0000 7fa0a9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40502/0 (socket says 172.21.15.7:40502) 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9e00700 1 -- 172.21.15.7:0/2614371391 learned_addr learned my addr 172.21.15.7:0/2614371391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0aa800700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1081e0 0x7fa0ac101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 -- 172.21.15.7:0/2614371391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1081e0 msgr2=0x7fa0ac101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1081e0 0x7fa0ac101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 -- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 msgr2=0x7fa0ac1028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9400700 1 -- 172.21.15.7:0/2614371391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac101d20 con 0x7fa0ac1058b0 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.803+0000 7fa0a9e00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0a9400700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac100f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa09800d950 tx=0x7fa09800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098016070 con 0x7fa0ac1058b0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09800fd70 con 0x7fa0ac1058b0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac1a28b0 con 0x7fa0ac1058b0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09800e3b0 con 0x7fa0ac1058b0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.804+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac1a2df0 con 0x7fa0ac1058b0 2024-06-13T06:54:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.805+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac069180 con 0x7fa0ac1058b0 2024-06-13T06:54:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.808+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa098019070 con 0x7fa0ac1058b0 2024-06-13T06:54:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.809+0000 7fa0a2a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 0x7fa0900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.810+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa0980cee80 con 0x7fa0ac1058b0 2024-06-13T06:54:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.810+0000 7fa0a9e00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 0x7fa0900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.810+0000 7fa0a9e00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 0x7fa0900a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b2e0 tx=0x7fa09c00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:29.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:29.811+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa098010460 con 0x7fa0ac1058b0 2024-06-13T06:54:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.050+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa0ac0628d0 con 0x7fa0900a2ca0 2024-06-13T06:54:30.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.067+0000 7fa0a2a00700 1 -- 172.21.15.7:0/2614371391 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa0ac0628d0 con 0x7fa0900a2ca0 2024-06-13T06:54:30.067 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-13T06:54:30.068 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 2m ago 5m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi175=b;smithi175=e;smithi175=h;smithi175=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-06-13T06:54:30.068 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 2m ago 11m smithi007=x;smithi175=y;smithi189=z;count:3 2024-06-13T06:54:30.068 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 2m ago 11m smithi007:172.21.15.7=a;smithi175:172.21.15.175=b;smithi189:172.21.15.189=c;count:3 2024-06-13T06:54:30.068 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 2m ago - 2024-06-13T06:54:30.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 msgr2=0x7fa0900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 0x7fa0900a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b2e0 tx=0x7fa09c00b880 comp rx=0 tx=0).stop 2024-06-13T06:54:30.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 msgr2=0x7fa0ac100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac100f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa09800d950 tx=0x7fa09800dc60 comp rx=0 tx=0).stop 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 shutdown_connections 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa0900a2ca0 0x7fa0900a5160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0ac1081e0 0x7fa0ac101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ac102f80 0x7fa0ac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 --2- 172.21.15.7:0/2614371391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0ac1058b0 0x7fa0ac100f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.070+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 >> 172.21.15.7:0/2614371391 conn(0x7fa0ac0fc9b0 msgr2=0x7fa0ac103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.071+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 shutdown_connections 2024-06-13T06:54:30.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:30.071+0000 7fa0b0a00700 1 -- 172.21.15.7:0/2614371391 wait complete. 2024-06-13T06:54:30.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-13T06:54:31.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:31 smithi007 ceph-mon[32077]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:31.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:31 smithi007 ceph-mon[32077]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:31 smithi189 ceph-mon[36616]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:31 smithi189 ceph-mon[36616]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:31 smithi175 ceph-mon[36886]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:31 smithi175 ceph-mon[36886]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:33.144 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:33 smithi007 ceph-mon[32077]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:33 smithi189 ceph-mon[36616]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:33 smithi175 ceph-mon[36886]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 -- 172.21.15.7:0/3929097856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c1061d0 msgr2=0x7fb72c10ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 --2- 172.21.15.7:0/3929097856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c1061d0 0x7fb72c10ca80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7240099c0 tx=0x7fb724009cd0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 -- 172.21.15.7:0/3929097856 shutdown_connections 2024-06-13T06:54:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 --2- 172.21.15.7:0/3929097856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c1061d0 0x7fb72c10ca80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 --2- 172.21.15.7:0/3929097856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 --2- 172.21.15.7:0/3929097856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c105830 0x7fb72c105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 -- 172.21.15.7:0/3929097856 >> 172.21.15.7:0/3929097856 conn(0x7fb72c0ffbd0 msgr2=0x7fb72c102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 -- 172.21.15.7:0/3929097856 shutdown_connections 2024-06-13T06:54:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.673+0000 7fb732800700 1 -- 172.21.15.7:0/3929097856 wait complete. 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.674+0000 7fb732800700 1 Processor -- start 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.674+0000 7fb732800700 1 -- start start 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c1061d0 0x7fb72c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c110250 con 0x7fb72c105830 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1100d0 con 0x7fb72c104630 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb732800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1103d0 con 0x7fb72c1061d0 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb730a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48488/0 (socket says 172.21.15.7:48488) 2024-06-13T06:54:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb730a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52682/0 (socket says 172.21.15.7:52682) 2024-06-13T06:54:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 -- 172.21.15.7:0/278006825 learned_addr learned my addr 172.21.15.7:0/278006825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731e00700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c1061d0 0x7fb72c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 -- 172.21.15.7:0/278006825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c1061d0 msgr2=0x7fb72c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c1061d0 0x7fb72c1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 -- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 msgr2=0x7fb72c19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.675+0000 7fb731400700 1 -- 172.21.15.7:0/278006825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a23d0 con 0x7fb72c104630 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb730a00700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb731400700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb72000efc0 tx=0x7fb72000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200092e0 con 0x7fb72c104630 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72000c9c0 con 0x7fb72c104630 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a2660 con 0x7fb72c104630 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720004610 con 0x7fb72c104630 2024-06-13T06:54:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.676+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a2bf0 con 0x7fb72c104630 2024-06-13T06:54:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.677+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c069180 con 0x7fb72c104630 2024-06-13T06:54:33.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.680+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb720044d10 con 0x7fb72c104630 2024-06-13T06:54:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.681+0000 7fb72a000700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 0x7fb7140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.681+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb7200cf690 con 0x7fb72c104630 2024-06-13T06:54:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.681+0000 7fb730a00700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 0x7fb7140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.682+0000 7fb730a00700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 0x7fb7140a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb718009680 tx=0x7fb718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.683+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb720022020 con 0x7fb72c104630 2024-06-13T06:54:33.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.925+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb72c0628d0 con 0x7fb7140a2ca0 2024-06-13T06:54:33.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.929+0000 7fb72a000700 1 -- 172.21.15.7:0/278006825 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb72c0628d0 con 0x7fb7140a2ca0 2024-06-13T06:54:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-06-13T06:54:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-06-13T06:54:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:smithi175 172.21.15.175 2024-06-13T06:54:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:smithi189 172.21.15.189 2024-06-13T06:54:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-06-13T06:54:33.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 msgr2=0x7fb7140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:33.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 0x7fb7140a5160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb718009680 tx=0x7fb718009260 comp rx=0 tx=0).stop 2024-06-13T06:54:33.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 msgr2=0x7fb72c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:33.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb72000efc0 tx=0x7fb72000c3a0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 shutdown_connections 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb72c1061d0 0x7fb72c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb7140a2ca0 0x7fb7140a5160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb72c104630 0x7fb72c19a770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 --2- 172.21.15.7:0/278006825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c105830 0x7fb72c19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 >> 172.21.15.7:0/278006825 conn(0x7fb72c0ffbd0 msgr2=0x7fb72c108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 shutdown_connections 2024-06-13T06:54:33.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:33.932+0000 7fb732800700 1 -- 172.21.15.7:0/278006825 wait complete. 2024-06-13T06:54:34.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-13T06:54:35.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:35 smithi007 ceph-mon[32077]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:35.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:35 smithi007 ceph-mon[32077]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:35 smithi189 ceph-mon[36616]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:35 smithi189 ceph-mon[36616]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:35 smithi175 ceph-mon[36886]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:35 smithi175 ceph-mon[36886]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:37 smithi189 ceph-mon[36616]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:37.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:37 smithi007 ceph-mon[32077]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:37 smithi175 ceph-mon[36886]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.677+0000 7f2340c00700 1 -- 172.21.15.7:0/1460182820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c102fe0 msgr2=0x7f233c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.677+0000 7f2340c00700 1 --2- 172.21.15.7:0/1460182820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c102fe0 0x7f233c1053d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f232c009990 tx=0x7f232c009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 -- 172.21.15.7:0/1460182820 shutdown_connections 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 --2- 172.21.15.7:0/1460182820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c108240 0x7f233c10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 --2- 172.21.15.7:0/1460182820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c102fe0 0x7f233c1053d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 --2- 172.21.15.7:0/1460182820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c105910 0x7f233c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 -- 172.21.15.7:0/1460182820 >> 172.21.15.7:0/1460182820 conn(0x7f233c0fc990 msgr2=0x7f233c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 -- 172.21.15.7:0/1460182820 shutdown_connections 2024-06-13T06:54:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.678+0000 7f2340c00700 1 -- 172.21.15.7:0/1460182820 wait complete. 2024-06-13T06:54:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 Processor -- start 2024-06-13T06:54:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 -- start start 2024-06-13T06:54:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c108240 0x7f233c1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f233c10e0c0 con 0x7f233c105910 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.679+0000 7f2340c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f233c10df40 con 0x7f233c102fe0 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2340c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f233c10e240 con 0x7f233c108240 2024-06-13T06:54:37.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52698/0 (socket says 172.21.15.7:52698) 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48506/0 (socket says 172.21.15.7:48506) 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339400700 1 -- 172.21.15.7:0/3425974126 learned_addr learned my addr 172.21.15.7:0/3425974126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f233a800700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c108240 0x7f233c1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 -- 172.21.15.7:0/3425974126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c108240 msgr2=0x7f233c1a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c108240 0x7f233c1a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 -- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 msgr2=0x7f233c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339e00700 1 -- 172.21.15.7:0/3425974126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c1a25f0 con 0x7f233c102fe0 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.680+0000 7f2339400700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2339e00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c00bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c026070 con 0x7f233c102fe0 2024-06-13T06:54:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c1a2880 con 0x7f233c102fe0 2024-06-13T06:54:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232c003d70 con 0x7f233c102fe0 2024-06-13T06:54:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c021410 con 0x7f233c102fe0 2024-06-13T06:54:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.681+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c1a2d30 con 0x7f233c102fe0 2024-06-13T06:54:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.683+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233c069180 con 0x7f233c102fe0 2024-06-13T06:54:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.684+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f232c029070 con 0x7f233c102fe0 2024-06-13T06:54:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.686+0000 7f2332a00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 0x7f23200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.686+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f232c0d01b0 con 0x7f233c102fe0 2024-06-13T06:54:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.686+0000 7f2339400700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 0x7f23200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.687+0000 7f2339400700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 0x7f23200a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2328011550 tx=0x7f2328009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:37.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.690+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f232c05d850 con 0x7f233c102fe0 2024-06-13T06:54:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.942+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f233c0628d0 con 0x7f23200a2c20 2024-06-13T06:54:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.944+0000 7f2332a00700 1 -- 172.21.15.7:0/3425974126 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f233c0628d0 con 0x7f23200a2c20 2024-06-13T06:54:37.945 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-13T06:54:37.945 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 6m ago Has a FileSystem, LVM detected 2024-06-13T06:54:37.945 INFO:teuthology.orchestra.run.smithi007.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-13T06:54:37.945 INFO:teuthology.orchestra.run.smithi007.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-13T06:54:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.947+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 msgr2=0x7f23200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.947+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 0x7f23200a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2328011550 tx=0x7f2328009260 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.947+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 msgr2=0x7f233c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.947+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c00bb90 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 shutdown_connections 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f233c108240 0x7f233c1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f23200a2c20 0x7f23200a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f233c102fe0 0x7f233c19a990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 --2- 172.21.15.7:0/3425974126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c105910 0x7f233c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 >> 172.21.15.7:0/3425974126 conn(0x7f233c0fc990 msgr2=0x7f233c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 shutdown_connections 2024-06-13T06:54:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:37.948+0000 7f2340c00700 1 -- 172.21.15.7:0/3425974126 wait complete. 2024-06-13T06:54:38.688 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-13T06:54:38.697 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:54:38.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-13T06:54:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:39.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:39 smithi007 ceph-mon[32077]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:39.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:39 smithi007 ceph-mon[32077]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:39.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:39 smithi189 ceph-mon[36616]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:39.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:39 smithi189 ceph-mon[36616]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:39 smithi175 ceph-mon[36886]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:39 smithi175 ceph-mon[36886]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-13T06:54:41.272 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:41 smithi007 ceph-mon[32077]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:41.497 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:41 smithi175 ceph-mon[36886]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:41.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:41 smithi189 ceph-mon[36616]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:41.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 -- 172.21.15.7:0/2998388498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0102420 msgr2=0x7fc9f0102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:41.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2998388498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0102420 0x7fc9f0102840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b0d0 tx=0x7fc9e000b3e0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 -- 172.21.15.7:0/2998388498 shutdown_connections 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2998388498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0103fc0 0x7fc9f010a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2998388498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0102420 0x7fc9f0102840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2998388498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103620 0x7fc9f0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 -- 172.21.15.7:0/2998388498 >> 172.21.15.7:0/2998388498 conn(0x7fc9f00fd9c0 msgr2=0x7fc9f00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 -- 172.21.15.7:0/2998388498 shutdown_connections 2024-06-13T06:54:41.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.549+0000 7fc9f6600700 1 -- 172.21.15.7:0/2998388498 wait complete. 2024-06-13T06:54:41.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.550+0000 7fc9f6600700 1 Processor -- start 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.550+0000 7fc9f6600700 1 -- start start 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0102420 0x7fc9f0076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 0x7fc9f0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010e040 con 0x7fc9f0102420 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010dec0 con 0x7fc9f0103620 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f6600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010e1c0 con 0x7fc9f0103fc0 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0102420 0x7fc9f0076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f5c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9e7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 0x7fc9f0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:41.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f5c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44776/0 (socket says 172.21.15.7:44776) 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.551+0000 7fc9f5c00700 1 -- 172.21.15.7:0/2163560982 learned_addr learned my addr 172.21.15.7:0/2163560982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 -- 172.21.15.7:0/2163560982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 msgr2=0x7fc9f0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 0x7fc9f0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 -- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0102420 msgr2=0x7fc9f0076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0102420 0x7fc9f0076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 -- 172.21.15.7:0/2163560982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f0075300 con 0x7fc9f0103fc0 2024-06-13T06:54:41.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9e7e00700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 0x7fc9f0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f5c00700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800e940 tx=0x7fc9e800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8004a30 con 0x7fc9f0103fc0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e8005500 con 0x7fc9f0103fc0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f00755f0 con 0x7fc9f0103fc0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.552+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800ce20 con 0x7fc9f0103fc0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.553+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a70c0 con 0x7fc9f0103fc0 2024-06-13T06:54:41.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.554+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f0069180 con 0x7fc9f0103fc0 2024-06-13T06:54:41.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.556+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc9e8044b30 con 0x7fc9f0103fc0 2024-06-13T06:54:41.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.557+0000 7fc9ee000700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 0x7fc9d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:41.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.557+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc9e80cf550 con 0x7fc9f0103fc0 2024-06-13T06:54:41.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.557+0000 7fc9f5200700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 0x7fc9d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:41.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.558+0000 7fc9f5200700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 0x7fc9d40a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b770 tx=0x7fc9e000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:41.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.559+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc9e809c110 con 0x7fc9f0103fc0 2024-06-13T06:54:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.837+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc9f002dd20 con 0x7fc9f0103fc0 2024-06-13T06:54:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.838+0000 7fc9ee000700 1 -- 172.21.15.7:0/2163560982 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3050 (secure 0 0 0) 0x7fc9e8044e20 con 0x7fc9f0103fc0 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:e25 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 25 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-13T06:54:41.839 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-06-13T06:50:19.413292+0000 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-06-13T06:51:49.856213+0000 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-06-13T06:54:41.840 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24539,1=24527,2=24521} 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3,4] 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24539 members: 24527,24539,24521 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24539} state up:active seq 7 addr [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{0:24511} state up:standby-replay seq 1 addr [v2:172.21.15.189:6838/3291148051,v1:172.21.15.189:6839/3291148051] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{1:24527} state up:active seq 28 addr [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{1:24509} state up:standby-replay seq 1 addr [v2:172.21.15.175:6836/2460722721,v1:172.21.15.175:6837/2460722721] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{2:24521} state up:active seq 29 addr [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{2:24496} state up:standby-replay seq 1 addr [v2:172.21.15.189:6836/4202454996,v1:172.21.15.189:6837/4202454996] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14361} state up:standby seq 1 addr [v2:172.21.15.7:6834/3083618837,v1:172.21.15.7:6835/3083618837] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.189:6832/1184948673,v1:172.21.15.189:6833/1184948673] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.175:6832/587777641,v1:172.21.15.175:6833/587777641] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:24475} state up:standby seq 1 addr [v2:172.21.15.7:6836/3062011119,v1:172.21.15.7:6837/3062011119] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{-1:24479} state up:standby seq 1 addr [v2:172.21.15.175:6834/1289787148,v1:172.21.15.175:6835/1289787148] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{-1:24490} state up:standby seq 1 addr [v2:172.21.15.189:6834/4153094559,v1:172.21.15.189:6835/4153094559] compat {c=[1],r=[1],i=[1fff]}] 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.842+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 msgr2=0x7fc9d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.842+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 0x7fc9d40a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b770 tx=0x7fc9e000b040 comp rx=0 tx=0).stop 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.842+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 msgr2=0x7fc9f0074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.842+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800e940 tx=0x7fc9e800ec50 comp rx=0 tx=0).stop 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 shutdown_connections 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc9f0103fc0 0x7fc9f0074dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9f0102420 0x7fc9f0076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc9d40a2c20 0x7fc9d40a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 --2- 172.21.15.7:0/2163560982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc9f0103620 0x7fc9f0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 >> 172.21.15.7:0/2163560982 conn(0x7fc9f00fd9c0 msgr2=0x7fc9f0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 shutdown_connections 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:41.843+0000 7fc9f6600700 1 -- 172.21.15.7:0/2163560982 wait complete. 2024-06-13T06:54:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:54:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2163560982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:54:42.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2163560982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:54:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2163560982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-13T06:54:42.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-13T06:54:43.318 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:43 smithi189 ceph-mon[36616]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:43 smithi175 ceph-mon[36886]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:43.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:43 smithi007 ceph-mon[32077]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:45.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:45 smithi007 ceph-mon[32077]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:45 smithi175 ceph-mon[36886]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.447+0000 7fc845000700 1 -- 172.21.15.7:0/3906215714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 msgr2=0x7fc84010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.447+0000 7fc845000700 1 --2- 172.21.15.7:0/3906215714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc84010cb50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc83800b0d0 tx=0x7fc83800b3e0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 -- 172.21.15.7:0/3906215714 shutdown_connections 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 --2- 172.21.15.7:0/3906215714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc84010cb50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 --2- 172.21.15.7:0/3906215714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840104700 0x7fc840104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 --2- 172.21.15.7:0/3906215714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840105900 0x7fc840105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 -- 172.21.15.7:0/3906215714 >> 172.21.15.7:0/3906215714 conn(0x7fc8400ffca0 msgr2=0x7fc8401020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 -- 172.21.15.7:0/3906215714 shutdown_connections 2024-06-13T06:54:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.448+0000 7fc845000700 1 -- 172.21.15.7:0/3906215714 wait complete. 2024-06-13T06:54:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 Processor -- start 2024-06-13T06:54:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 -- start start 2024-06-13T06:54:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 0x7fc84019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 0x7fc84019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc8401a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc840110320 con 0x7fc8401062a0 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401101a0 con 0x7fc840105900 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.449+0000 7fc845000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401104a0 con 0x7fc840104700 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc8401a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc8401a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49006/0 (socket says 172.21.15.7:49006) 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83e800700 1 -- 172.21.15.7:0/1689667173 learned_addr learned my addr 172.21.15.7:0/1689667173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 0x7fc84019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83d400700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 0x7fc84019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 -- 172.21.15.7:0/1689667173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 msgr2=0x7fc84019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 0x7fc84019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 -- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 msgr2=0x7fc8401a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc8401a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83de00700 1 -- 172.21.15.7:0/1689667173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8401a2570 con 0x7fc840104700 2024-06-13T06:54:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.450+0000 7fc83d400700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 0x7fc84019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc83de00700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 0x7fc84019a910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc83000e9f0 tx=0x7fc83000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83000cb70 con 0x7fc840104700 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8401a2860 con 0x7fc840104700 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83000ee90 con 0x7fc840104700 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830018680 con 0x7fc840104700 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.451+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401a2df0 con 0x7fc840104700 2024-06-13T06:54:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.452+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc840069180 con 0x7fc840104700 2024-06-13T06:54:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.454+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc83001f020 con 0x7fc840104700 2024-06-13T06:54:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.456+0000 7fc836a00700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 0x7fc8240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.456+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc8300cf7f0 con 0x7fc840104700 2024-06-13T06:54:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.456+0000 7fc83d400700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 0x7fc8240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.457+0000 7fc83d400700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 0x7fc8240a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc82c011490 tx=0x7fc82c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.458+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc83009c3b0 con 0x7fc840104700 2024-06-13T06:54:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:45 smithi189 ceph-mon[36616]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.688+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc8400630a0 con 0x7fc840104700 2024-06-13T06:54:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.690+0000 7fc836a00700 1 -- 172.21.15.7:0/1689667173 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v98) ==== 56+0+5550 (secure 0 0 0) 0x7fc83009bcb0 con 0x7fc840104700 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 98 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:fsid b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-06-13T06:40:52.731225+0000 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-06-13T06:51:00.875426+0000 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 27 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-06-13T06:54:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/625935948,v1:172.21.15.7:6803/625935948] [v2:172.21.15.7:6804/625935948,v1:172.21.15.7:6805/625935948] exists,up e144dfd9-116a-4342-8fe5-e40f94ed6071 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/2094167604,v1:172.21.15.175:6801/2094167604] [v2:172.21.15.175:6802/2094167604,v1:172.21.15.175:6803/2094167604] exists,up c21f9125-8329-4858-a611-5aeb5e3d6da0 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2765218208,v1:172.21.15.189:6801/2765218208] [v2:172.21.15.189:6802/2765218208,v1:172.21.15.189:6803/2765218208] exists,up 073fad02-1136-4f2e-bd65-8a2a7ceb5a69 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/2396372699,v1:172.21.15.7:6811/2396372699] [v2:172.21.15.7:6812/2396372699,v1:172.21.15.7:6813/2396372699] exists,up 60aad01f-267a-43ad-b0d3-acb77db40434 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 34 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/576672935,v1:172.21.15.175:6809/576672935] [v2:172.21.15.175:6810/576672935,v1:172.21.15.175:6811/576672935] exists,up 2d14e0c7-6370-4889-8f20-87458638b022 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/1526104805,v1:172.21.15.189:6809/1526104805] [v2:172.21.15.189:6810/1526104805,v1:172.21.15.189:6811/1526104805] exists,up 6f01cff3-7564-4832-a4f8-0b2920c1b201 2024-06-13T06:54:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/3095237375,v1:172.21.15.7:6819/3095237375] [v2:172.21.15.7:6820/3095237375,v1:172.21.15.7:6821/3095237375] exists,up 785c5d21-30ac-46c0-9f6c-543778283cb7 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 52 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/935680699,v1:172.21.15.175:6817/935680699] [v2:172.21.15.175:6818/935680699,v1:172.21.15.175:6819/935680699] exists,up 41e3e868-f9cf-461e-84ce-4321ccdd9fd8 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 60 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/1210852331,v1:172.21.15.189:6817/1210852331] [v2:172.21.15.189:6818/1210852331,v1:172.21.15.189:6819/1210852331] exists,up f9407bcc-7a87-425c-a20b-dd361583d662 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 68 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/1288548158,v1:172.21.15.7:6827/1288548158] [v2:172.21.15.7:6828/1288548158,v1:172.21.15.7:6829/1288548158] exists,up b78e7f03-fad5-401e-a013-def76183c002 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/2694135540,v1:172.21.15.175:6825/2694135540] [v2:172.21.15.175:6826/2694135540,v1:172.21.15.175:6827/2694135540] exists,up f61b6ecf-df12-4a81-93f4-b17aac68e6f4 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 81 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/1625559024,v1:172.21.15.189:6825/1625559024] [v2:172.21.15.189:6826/1625559024,v1:172.21.15.189:6827/1625559024] exists,up 2053f9b6-4b88-48c3-8665-7501ea3b4b3b 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.4 [2,11] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.a [2,11] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.25 [5,9] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.34 [4,9] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.3c [4,9] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.3 [5,8] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.5 [9,10] 2024-06-13T06:54:45.694 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.b [9,8] 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 4.e [9,11] 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/3124714037 expires 2024-06-14T06:41:59.455768+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3381938819 expires 2024-06-14T06:41:26.246814+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1735370722 expires 2024-06-14T06:41:59.455768+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1576245213 expires 2024-06-14T06:41:59.455768+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/239861889 expires 2024-06-14T06:41:26.246814+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/4091601586 expires 2024-06-14T06:41:26.246814+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/3124714037 expires 2024-06-14T06:41:59.455768+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1109324950 expires 2024-06-14T06:41:26.246814+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/4091601586 expires 2024-06-14T06:41:26.246814+0000 2024-06-13T06:54:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1476683041 expires 2024-06-14T06:41:59.455768+0000 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 msgr2=0x7fc8240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 0x7fc8240a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc82c011490 tx=0x7fc82c009260 comp rx=0 tx=0).stop 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 msgr2=0x7fc84019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 0x7fc84019a910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc83000e9f0 tx=0x7fc83000ed00 comp rx=0 tx=0).stop 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 shutdown_connections 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8401062a0 0x7fc8401a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc840104700 0x7fc84019a910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8240a2c20 0x7fc8240a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 --2- 172.21.15.7:0/1689667173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc840105900 0x7fc84019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:45.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 >> 172.21.15.7:0/1689667173 conn(0x7fc8400ffca0 msgr2=0x7fc840108ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:45.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 shutdown_connections 2024-06-13T06:54:45.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:45.693+0000 7fc845000700 1 -- 172.21.15.7:0/1689667173 wait complete. 2024-06-13T06:54:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:46 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1689667173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:54:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:46 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1689667173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:54:46.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:46 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1689667173' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-13T06:54:46.593 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-13T06:54:46.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:47 smithi175 ceph-mon[36886]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:47.380 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:47 smithi007 ceph-mon[32077]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:47 smithi189 ceph-mon[36616]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:49.111 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:49 smithi007 ceph-mon[32077]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:49.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:49 smithi175 ceph-mon[36886]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.405+0000 7f7bb8a00700 1 -- 172.21.15.7:0/21120055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 msgr2=0x7f7bb4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.405+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/21120055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb4104b10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009990 tx=0x7f7ba0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 -- 172.21.15.7:0/21120055 shutdown_connections 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/21120055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb4105050 0x7f7bb4111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/21120055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb410a690 0x7f7bb410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/21120055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb4104b10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 -- 172.21.15.7:0/21120055 >> 172.21.15.7:0/21120055 conn(0x7f7bb41001a0 msgr2=0x7f7bb41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:49.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 -- 172.21.15.7:0/21120055 shutdown_connections 2024-06-13T06:54:49.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 -- 172.21.15.7:0/21120055 wait complete. 2024-06-13T06:54:49.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 Processor -- start 2024-06-13T06:54:49.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.407+0000 7f7bb8a00700 1 -- start start 2024-06-13T06:54:49.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:49.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 0x7f7bb419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:49.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb410a690 0x7f7bb419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:49.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4114a20 con 0x7f7bb410a690 2024-06-13T06:54:49.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41148a0 con 0x7f7bb4105050 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb8a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4114ba0 con 0x7f7bb4104690 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb410a690 0x7f7bb419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44806/0 (socket says 172.21.15.7:44806) 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 -- 172.21.15.7:0/301039917 learned_addr learned my addr 172.21.15.7:0/301039917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:49.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1400700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 0x7f7bb419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 -- 172.21.15.7:0/301039917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 msgr2=0x7f7bb419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 0x7f7bb419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 -- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb410a690 msgr2=0x7f7bb419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb410a690 0x7f7bb419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 -- 172.21.15.7:0/301039917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb419fe00 con 0x7f7bb4104690 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1400700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 0x7f7bb419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7bb1e00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e9a0 tx=0x7f7ba400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.409+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400cbd0 con 0x7f7bb4104690 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.410+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba400cd70 con 0x7f7bb4104690 2024-06-13T06:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.410+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb41a00f0 con 0x7f7bb4104690 2024-06-13T06:54:49.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.410+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4018750 con 0x7f7bb4104690 2024-06-13T06:54:49.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.410+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41a7970 con 0x7f7bb4104690 2024-06-13T06:54:49.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.412+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb4069180 con 0x7f7bb4104690 2024-06-13T06:54:49.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.414+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7ba4010430 con 0x7f7bb4104690 2024-06-13T06:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.416+0000 7f7baaa00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.416+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7ba40cfa30 con 0x7f7bb4104690 2024-06-13T06:54:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.416+0000 7f7bb1400700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.417+0000 7f7bb1400700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 0x7f7b980a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0006010 tx=0x7f7ba001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.418+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ba409c5f0 con 0x7f7bb4104690 2024-06-13T06:54:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:49 smithi189 ceph-mon[36616]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.703+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bb419c150 con 0x7f7bb4104690 2024-06-13T06:54:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.704+0000 7f7baaa00700 1 -- 172.21.15.7:0/301039917 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7f7ba409bef0 con 0x7f7bb4104690 2024-06-13T06:54:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:49.703 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:54:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 msgr2=0x7f7b980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 0x7f7b980a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0006010 tx=0x7f7ba001a040 comp rx=0 tx=0).stop 2024-06-13T06:54:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 msgr2=0x7f7bb419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:49.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e9a0 tx=0x7f7ba400ecb0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 shutdown_connections 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb410a690 0x7f7bb419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bb4104690 0x7f7bb419adf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.707+0000 7f7bb8a00700 1 --2- 172.21.15.7:0/301039917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb4105050 0x7f7bb419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.708+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 >> 172.21.15.7:0/301039917 conn(0x7f7bb41001a0 msgr2=0x7f7bb4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.708+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 shutdown_connections 2024-06-13T06:54:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:49.708+0000 7f7bb8a00700 1 -- 172.21.15.7:0/301039917 wait complete. 2024-06-13T06:54:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:54:50.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:50 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/301039917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:50.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:54:50.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:50 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/301039917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:50.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:50 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/301039917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:51.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:51 smithi007 ceph-mon[32077]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:51 smithi189 ceph-mon[36616]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:51 smithi175 ceph-mon[36886]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.229+0000 7fda42000700 1 -- 172.21.15.7:0/1922616549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 msgr2=0x7fda3c111130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.229+0000 7fda42000700 1 --2- 172.21.15.7:0/1922616549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c111130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b0a0 tx=0x7fda3400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 -- 172.21.15.7:0/1922616549 shutdown_connections 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 --2- 172.21.15.7:0/1922616549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c111130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 --2- 172.21.15.7:0/1922616549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101190 0x7fda3c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 --2- 172.21.15.7:0/1922616549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101b40 0x7fda3c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 -- 172.21.15.7:0/1922616549 >> 172.21.15.7:0/1922616549 conn(0x7fda3c0fcdb0 msgr2=0x7fda3c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:53.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 -- 172.21.15.7:0/1922616549 shutdown_connections 2024-06-13T06:54:53.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.230+0000 7fda42000700 1 -- 172.21.15.7:0/1922616549 wait complete. 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 Processor -- start 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 -- start start 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 0x7fda3c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101b40 0x7fda3c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c114960 con 0x7fda3c10ed40 2024-06-13T06:54:53.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1147e0 con 0x7fda3c101b40 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.231+0000 7fda42000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c114ae0 con 0x7fda3c101190 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda41600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 0x7fda3c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda41600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41014/0 (socket says 172.21.15.7:41014) 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda41600700 1 -- 172.21.15.7:0/3073083373 learned_addr learned my addr 172.21.15.7:0/3073083373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda3be00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101b40 0x7fda3c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 -- 172.21.15.7:0/3073083373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101b40 msgr2=0x7fda3c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101b40 0x7fda3c19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 -- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 msgr2=0x7fda3c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda40c00700 1 -- 172.21.15.7:0/3073083373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c19fd60 con 0x7fda3c101190 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.232+0000 7fda41600700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda40c00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 0x7fda3c19ad50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9f0 tx=0x7fda3000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3000c9c0 con 0x7fda3c101190 2024-06-13T06:54:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3000cb60 con 0x7fda3c101190 2024-06-13T06:54:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a0050 con 0x7fda3c101190 2024-06-13T06:54:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30018680 con 0x7fda3c101190 2024-06-13T06:54:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.233+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a78d0 con 0x7fda3c101190 2024-06-13T06:54:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.234+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c069180 con 0x7fda3c101190 2024-06-13T06:54:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.237+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fda30010430 con 0x7fda3c101190 2024-06-13T06:54:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.238+0000 7fda39600700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 0x7fda240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.238+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fda300cf770 con 0x7fda3c101190 2024-06-13T06:54:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.239+0000 7fda3be00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 0x7fda240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.239+0000 7fda3be00700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 0x7fda240a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda3c102aa0 tx=0x7fda28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.240+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda3009c330 con 0x7fda3c101190 2024-06-13T06:54:53.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:53 smithi007 ceph-mon[32077]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.520+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3c19c0b0 con 0x7fda3c101190 2024-06-13T06:54:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.522+0000 7fda39600700 1 -- 172.21.15.7:0/3073083373 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11868 (secure 0 0 0) 0x7fda3c19c0b0 con 0x7fda3c101190 2024-06-13T06:54:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:53.521 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:51:49.856213+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:54:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 msgr2=0x7fda240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 0x7fda240a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda3c102aa0 tx=0x7fda28009260 comp rx=0 tx=0).stop 2024-06-13T06:54:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 msgr2=0x7fda3c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 0x7fda3c19ad50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9f0 tx=0x7fda3000ed00 comp rx=0 tx=0).stop 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 shutdown_connections 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda3c10ed40 0x7fda3c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda3c101190 0x7fda3c19ad50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fda240a2ca0 0x7fda240a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 --2- 172.21.15.7:0/3073083373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda3c101b40 0x7fda3c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 >> 172.21.15.7:0/3073083373 conn(0x7fda3c0fcdb0 msgr2=0x7fda3c110f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.525+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 shutdown_connections 2024-06-13T06:54:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:53.526+0000 7fda42000700 1 -- 172.21.15.7:0/3073083373 wait complete. 2024-06-13T06:54:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-06-13T06:54:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:53 smithi189 ceph-mon[36616]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:53 smithi175 ceph-mon[36886]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:54:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:54 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3073083373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:54.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:54:54.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:54 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3073083373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:54.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:54 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3073083373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:54:54.915 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:55.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:55 smithi007 ceph-mon[32077]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:55 smithi189 ceph-mon[36616]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:55 smithi175 ceph-mon[36886]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 -- 172.21.15.7:0/3098452192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5990107840 msgr2=0x7f599010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 --2- 172.21.15.7:0/3098452192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5990107840 0x7f599010f3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5988009fc0 tx=0x7f598800b330 comp rx=0 tx=0).stop 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 -- 172.21.15.7:0/3098452192 shutdown_connections 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 --2- 172.21.15.7:0/3098452192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5990107840 0x7f599010f3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 --2- 172.21.15.7:0/3098452192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 --2- 172.21.15.7:0/3098452192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59901030f0 0x7f5990107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 -- 172.21.15.7:0/3098452192 >> 172.21.15.7:0/3098452192 conn(0x7f59900795f0 msgr2=0x7f5990079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 -- 172.21.15.7:0/3098452192 shutdown_connections 2024-06-13T06:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.327+0000 7f5995a00700 1 -- 172.21.15.7:0/3098452192 wait complete. 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.328+0000 7f5995a00700 1 Processor -- start 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 -- start start 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5990107840 0x7f5990075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5990112760 con 0x7f5990107840 2024-06-13T06:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59901125e0 con 0x7f59901030f0 2024-06-13T06:54:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.329+0000 7f5995a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59901128e0 con 0x7f5990102740 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5990107840 0x7f5990075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47938/0 (socket says 172.21.15.7:47938) 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57488/0 (socket says 172.21.15.7:57488) 2024-06-13T06:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 -- 172.21.15.7:0/1895106532 learned_addr learned my addr 172.21.15.7:0/1895106532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 -- 172.21.15.7:0/1895106532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 msgr2=0x7f5990074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 -- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5990107840 msgr2=0x7f5990075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5990107840 0x7f5990075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 -- 172.21.15.7:0/1895106532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59901a7040 con 0x7f59901030f0 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.330+0000 7f598e000700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f597c00ec10 tx=0x7f597c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.331+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c00cc90 con 0x7f59901030f0 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.331+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597c00ce30 con 0x7f59901030f0 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.331+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59901a7330 con 0x7f59901030f0 2024-06-13T06:54:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.331+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c010690 con 0x7f59901030f0 2024-06-13T06:54:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.331+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59901a7870 con 0x7f59901030f0 2024-06-13T06:54:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.332+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5990069180 con 0x7f59901030f0 2024-06-13T06:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.335+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f597c003680 con 0x7f59901030f0 2024-06-13T06:54:57.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.336+0000 7f5987400700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 0x7f59740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:54:57.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.336+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f597c0cf6c0 con 0x7f59901030f0 2024-06-13T06:54:57.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.336+0000 7f598ea00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 0x7f59740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.337+0000 7f598ea00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 0x7f59740a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5980009c60 tx=0x7f5980009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:54:57.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.341+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f597c09c280 con 0x7f59901030f0 2024-06-13T06:54:57.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:57 smithi007 ceph-mon[32077]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:57.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:57 smithi175 ceph-mon[36886]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.568+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f599002dca0 con 0x7f59901030f0 2024-06-13T06:54:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.571+0000 7f5987400700 1 -- 172.21.15.7:0/1895106532 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f597c09bb80 con 0x7f59901030f0 2024-06-13T06:54:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:54:57.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 msgr2=0x7f59740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 0x7f59740a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5980009c60 tx=0x7f5980009260 comp rx=0 tx=0).stop 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 msgr2=0x7f5990074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f597c00ec10 tx=0x7f597c00ef20 comp rx=0 tx=0).stop 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 shutdown_connections 2024-06-13T06:54:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5990107840 0x7f5990075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990102740 0x7f5990074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59740a2c20 0x7f59740a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 --2- 172.21.15.7:0/1895106532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59901030f0 0x7f5990074cd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.574+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 >> 172.21.15.7:0/1895106532 conn(0x7f59900795f0 msgr2=0x7f59901004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.575+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 shutdown_connections 2024-06-13T06:54:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:54:57.575+0000 7f5995a00700 1 -- 172.21.15.7:0/1895106532 wait complete. 2024-06-13T06:54:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:57 smithi189 ceph-mon[36616]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:54:58.350 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:58 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1895106532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:58.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-13T06:50:19.413292+0000', 'modified': '2024-06-13T06:51:49.856213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24539, 'mds_1': 24527, 'mds_2': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/4202454996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 4202454996}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 4202454996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6837/2460722721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 2460722721}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2460722721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6839/3291148051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 3291148051}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 3291148051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'g', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.7:6839/827561506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6838', 'nonce': 827561506}, {'type': 'v1', 'addr': '172.21.15.7:6839', 'nonce': 827561506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.7:6841/1761725822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6840', 'nonce': 1761725822}, {'type': 'v1', 'addr': '172.21.15.7:6841', 'nonce': 1761725822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6839/4167274347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 4167274347}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 4167274347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24527, 24539, 24521]} 2024-06-13T06:54:58.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-13T06:54:58.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.g status 2024-06-13T06:54:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:58 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1895106532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:58.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:58 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1895106532' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:54:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:54:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:54:59 smithi007 ceph-mon[32077]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:54:59 smithi189 ceph-mon[36616]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:54:59.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:54:59 smithi175 ceph-mon[36886]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:55:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 -- 172.21.15.7:0/3521227558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 msgr2=0x7f36b8115710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 --2- 172.21.15.7:0/3521227558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b8115710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009f90 tx=0x7f36b000b330 comp rx=0 tx=0).stop 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 -- 172.21.15.7:0/3521227558 shutdown_connections 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 --2- 172.21.15.7:0/3521227558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b8115710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 --2- 172.21.15.7:0/3521227558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80769c0 0x7f36b8076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 --2- 172.21.15.7:0/3521227558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80772e0 0x7f36b8112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 -- 172.21.15.7:0/3521227558 >> 172.21.15.7:0/3521227558 conn(0x7f36b8100170 msgr2=0x7f36b8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.218 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:01 smithi007 ceph-mon[32077]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 -- 172.21.15.7:0/3521227558 shutdown_connections 2024-06-13T06:55:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.218+0000 7f36bce00700 1 -- 172.21.15.7:0/3521227558 wait complete. 2024-06-13T06:55:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.219+0000 7f36bce00700 1 Processor -- start 2024-06-13T06:55:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 -- start start 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80772e0 0x7f36b819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b81a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8118e00 con 0x7f36b80772e0 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8118c80 con 0x7f36b8113320 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36bce00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8118f80 con 0x7f36b80769c0 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36b5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36b5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46428/0 (socket says 172.21.15.7:46428) 2024-06-13T06:55:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.220+0000 7f36b5e00700 1 -- 172.21.15.7:0/1287493817 learned_addr learned my addr 172.21.15.7:0/1287493817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5400700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80772e0 0x7f36b819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b6800700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b81a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 -- 172.21.15.7:0/1287493817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 msgr2=0x7f36b81a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b81a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 -- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80772e0 msgr2=0x7f36b819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80772e0 0x7f36b819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 -- 172.21.15.7:0/1287493817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b81a4220 con 0x7f36b80769c0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b6800700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b81a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.221+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36a800ec10 tx=0x7f36a800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.222+0000 7f36aea00700 1 -- 172.21.15.7:0/1287493817 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800cd40 con 0x7f36b80769c0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.222+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b81a4510 con 0x7f36b80769c0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.222+0000 7f36aea00700 1 -- 172.21.15.7:0/1287493817 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a8004500 con 0x7f36b80769c0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.222+0000 7f36aea00700 1 -- 172.21.15.7:0/1287493817 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8010690 con 0x7f36b80769c0 2024-06-13T06:55:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.222+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b81abd90 con 0x7f36b80769c0 2024-06-13T06:55:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.225+0000 7f36aea00700 1 -- 172.21.15.7:0/1287493817 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f36a8003f10 con 0x7f36b80769c0 2024-06-13T06:55:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.226+0000 7f36aea00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 0x7f369c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36aea00700 1 -- 172.21.15.7:0/1287493817 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36a80cf770 con 0x7f36b80769c0 2024-06-13T06:55:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 Processor -- start 2024-06-13T06:55:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36b5400700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 0x7f369c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 -- start start 2024-06-13T06:55:01.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09c460 0x55d31e095cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e0961f0 0x55d31e096630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31dfa49c0 con 0x55d31e0961f0 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31dfe7ab0 con 0x55d31e09bb40 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.227+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d31dfa28a0 con 0x55d31e09c460 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b6800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e0961f0 0x55d31e096630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55962/0 (socket says 172.21.15.7:55962) 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 -- 172.21.15.7:0/2230811362 learned_addr learned my addr 172.21.15.7:0/2230811362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5400700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 0x7f369c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f36b8103b70 tx=0x7f36a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5400700 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09c460 0x55d31e095cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 -- 172.21.15.7:0/2230811362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09c460 msgr2=0x55d31e095cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09c460 0x55d31e095cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 -- 172.21.15.7:0/2230811362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e0961f0 msgr2=0x55d31e096630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e0961f0 0x55d31e096630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 -- 172.21.15.7:0/2230811362 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31df8ff40 con 0x55d31e09bb40 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.228+0000 7f36b5e00700 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36a80a4370 tx=0x7f36a8097fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230c6d91fdaa0974 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.229+0000 7f36a3400700 1 -- 172.21.15.7:0/2230811362 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a809c330 con 0x55d31e09bb40 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.229+0000 7f36a3400700 1 -- 172.21.15.7:0/2230811362 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a809bc30 con 0x55d31e09bb40 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.229+0000 7f36a3400700 1 -- 172.21.15.7:0/2230811362 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a80982a0 con 0x55d31e09bb40 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.229+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2230811362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 msgr2=0x55d31e09bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.229+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36a80a4370 tx=0x7f36a8097fa0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2230811362 shutdown_connections 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e0961f0 0x55d31e096630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e09bf20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2230811362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09c460 0x55d31e095cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2230811362 >> 172.21.15.7:0/2230811362 conn(0x55d31dfc2120 msgr2=0x55d31e090fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2230811362 shutdown_connections 2024-06-13T06:55:01.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2230811362 wait complete. 2024-06-13T06:55:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.230+0000 7f36c77a2b80 1 -- start start 2024-06-13T06:55:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 Processor -- start 2024-06-13T06:55:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e0a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09c460 0x55d31e09fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31df83f10 con 0x55d31e09bb40 2024-06-13T06:55:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31dfe3a70 con 0x55d31e09c460 2024-06-13T06:55:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0a2e40 con 0x55d31e0961f0 2024-06-13T06:55:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.231+0000 7f36c77a2b80 10 client.0 _ll_register_callbacks cb 0x55d31df7d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e0a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47524/0 (socket says 172.21.15.7:47524) 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 -- 172.21.15.7:0/1301957219 learned_addr learned my addr 172.21.15.7:0/1301957219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b6800700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09c460 0x55d31e09fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 -- 172.21.15.7:0/1301957219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 msgr2=0x55d31e0a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e0a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 -- 172.21.15.7:0/1301957219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09c460 msgr2=0x55d31e09fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09c460 0x55d31e09fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5400700 1 -- 172.21.15.7:0/1301957219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31df36880 con 0x55d31e09bb40 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.232+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e0a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f36b5400700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36a401e710 tx=0x7f36a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db16dcdc07a3599 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f3697e00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4028430 con 0x55d31e09bb40 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36a401f430 con 0x55d31e09bb40 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a809bc30 con 0x55d31e09bb40 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36a4004830 con 0x55d31e09bb40 2024-06-13T06:55:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f36c77a2b80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.233+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d31e0ad6d0 con 0x55d31e09bb40 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a809bc30 con 0x55d31e09bb40 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36a401f670 con 0x55d31e09bb40 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 10 client.14970 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 20 client.14970 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e0afd50 0x55d31e0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 4 client.14970 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d31dff4fb0 con 0x55d31e0afd50 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.234+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e0afd50 0x55d31e0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.235+0000 7f36b5e00700 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e0afd50 0x55d31e0b11c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2423614c0d04ac56 server_cookie=efe2933b48d9c9e6 in_seq=0 out_seq=0 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.235+0000 7f3697e00700 10 client.14970 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f3697e00700 1 -- 172.21.15.7:0/1301957219 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d31dff4fb0 con 0x55d31e0afd50 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f3697e00700 10 client.14970 handle_command_reply: tid=0 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 client.14970 shutdown 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 msgr2=0x55d31e09f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36a401e710 tx=0x7f36a40047c0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 shutdown_connections 2024-06-13T06:55:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e0afd50 0x55d31e0b11c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09c460 0x55d31e09fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e0a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/1301957219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09bb40 0x55d31e09f580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 >> 172.21.15.7:0/1301957219 conn(0x55d31dfc2120 msgr2=0x55d31e08a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.236+0000 7f3697e00700 0 client.14970 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:55:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.237+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 shutdown_connections 2024-06-13T06:55:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.237+0000 7f36c77a2b80 1 -- 172.21.15.7:0/1301957219 wait complete. 2024-06-13T06:55:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.237+0000 7f36c77a2b80 20 client.14970 trim_cache size 0 max 16384 2024-06-13T06:55:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.260+0000 7f36c77a2b80 1 Processor -- start 2024-06-13T06:55:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.260+0000 7f36c77a2b80 1 -- start start 2024-06-13T06:55:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e095cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e099cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0af500 con 0x55d31e09c460 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0afbd0 con 0x55d31e09bb40 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0b02a0 con 0x55d31e0961f0 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36b6800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36b6800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47538/0 (socket says 172.21.15.7:47538) 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36b6800700 1 -- 172.21.15.7:0/2986271906 learned_addr learned my addr 172.21.15.7:0/2986271906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36b5400700 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e099cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.261+0000 7f36b5e00700 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e095cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 -- 172.21.15.7:0/2986271906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 msgr2=0x55d31e095cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e095cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 -- 172.21.15.7:0/2986271906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 msgr2=0x55d31e099cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e099cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 -- 172.21.15.7:0/2986271906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d31e09c170 con 0x55d31e09c460 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36b6800700 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009f90 tx=0x7f36b0004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc5fdca801a5e47 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36a2a00700 1 -- 172.21.15.7:0/2986271906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004e50 con 0x55d31e09c460 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36a2a00700 1 -- 172.21.15.7:0/2986271906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b0005040 con 0x55d31e09c460 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.262+0000 7f36a2a00700 1 -- 172.21.15.7:0/2986271906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b001d640 con 0x55d31e09c460 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2986271906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 msgr2=0x55d31e08b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009f90 tx=0x7f36b0004ae0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2986271906 shutdown_connections 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e08b0e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e0961f0 0x55d31e095cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/2986271906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e09bb40 0x55d31e099cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2986271906 >> 172.21.15.7:0/2986271906 conn(0x55d31dfc2120 msgr2=0x55d31e0995c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2986271906 shutdown_connections 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.263+0000 7f36c77a2b80 1 -- 172.21.15.7:0/2986271906 wait complete. 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.264+0000 7f36c77a2b80 1 -- start start 2024-06-13T06:55:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.264+0000 7f36c77a2b80 1 Processor -- start 2024-06-13T06:55:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e0961f0 0x55d31e0bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e0a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0b1040 con 0x55d31e09c460 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0b0970 con 0x55d31e0961f0 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d31e0b1710 con 0x55d31e09bb40 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36c77a2b80 10 client.0 _ll_register_callbacks cb 0x55d31df7d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46472/0 (socket says 172.21.15.7:46472) 2024-06-13T06:55:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 -- 172.21.15.7:0/250464185 learned_addr learned my addr 172.21.15.7:0/250464185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5e00700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e0961f0 0x55d31e0bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b6800700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e0a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 -- 172.21.15.7:0/250464185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e0961f0 msgr2=0x55d31e0bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e0961f0 0x55d31e0bab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.265+0000 7f36b5400700 1 -- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 msgr2=0x55d31e0a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f36b5400700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e0a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f36b5400700 1 -- 172.21.15.7:0/250464185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d31e0a1cd0 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f36b5400700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f36a400a010 tx=0x7f36a4004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206fc89413f45b5a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f3697e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a401ee80 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b0005040 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.266+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36a401f640 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36a4044050 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 10 client.25096 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d31e0b93b0 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b0005040 con 0x55d31e09bb40 2024-06-13T06:55:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36a4005250 con 0x55d31e09bb40 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 10 client.25096 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 10 client.25096 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 20 client.25096 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e11ffb0 0x55d31e11ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 4 client.25096 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:55:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.267+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "status"}) -- 0x55d31dfef530 con 0x55d31e11ffb0 2024-06-13T06:55:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.268+0000 7f36b5e00700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e11ffb0 0x55d31e11ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.268+0000 7f36b5e00700 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e11ffb0 0x55d31e11ecc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de416a1a3837afa1 server_cookie=eab7a661d33dbf94 in_seq=0 out_seq=0 2024-06-13T06:55:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.268+0000 7f3697e00700 10 client.25096 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:55:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f3697e00700 1 -- 172.21.15.7:0/250464185 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d31dfef530 con 0x55d31e11ffb0 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f3697e00700 10 client.25096 handle_command_reply: tid=0 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 client.25096 shutdown 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 msgr2=0x55d31e0a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f36a400a010 tx=0x7f36a4004c80 comp rx=0 tx=0).stop 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 shutdown_connections 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d31e09c460 0x55d31e0a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55d31e11ffb0 0x55d31e11ecc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d31e0961f0 0x55d31e0bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 --2- 172.21.15.7:0/250464185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d31e09bb40 0x55d31e0a11d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 >> 172.21.15.7:0/250464185 conn(0x55d31dfc2120 msgr2=0x55d31e09dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f3697e00700 0 client.25096 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:55:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 shutdown_connections 2024-06-13T06:55:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.269+0000 7f36c77a2b80 1 -- 172.21.15.7:0/250464185 wait complete. 2024-06-13T06:55:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.270+0000 7f36c77a2b80 20 client.25096 trim_cache size 0 max 16384 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24521, 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 199.481947789, 2024-06-13T06:55:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:55:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:55:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:55:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 311.22832381500001 2024-06-13T06:55:01.277 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.270+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 msgr2=0x7f369c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.270+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 0x7f369c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f36b8103b70 tx=0x7f36a4009260 comp rx=0 tx=0).stop 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.270+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 msgr2=0x7f36b819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.270+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36a800ec10 tx=0x7f36a800ef20 comp rx=0 tx=0).stop 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 shutdown_connections 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f369c0a2c20 0x7f369c0a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36b8113320 0x7f36b81a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36b80769c0 0x7f36b819f210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 --2- 172.21.15.7:0/1287493817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b80772e0 0x7f36b819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 >> 172.21.15.7:0/1287493817 conn(0x7f36b8100170 msgr2=0x7f36b8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 shutdown_connections 2024-06-13T06:55:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:01.271+0000 7f36bce00700 1 -- 172.21.15.7:0/1287493817 wait complete. 2024-06-13T06:55:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:01 smithi189 ceph-mon[36616]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:01 smithi175 ceph-mon[36886]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:02.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.a status 2024-06-13T06:55:02.586 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:03.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:03 smithi007 ceph-mon[32077]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:03 smithi189 ceph-mon[36616]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:03 smithi175 ceph-mon[36886]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 -- 172.21.15.7:0/4085123088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a0104f80 msgr2=0x7f07a0111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 --2- 172.21.15.7:0/4085123088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a0104f80 0x7f07a0111490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0798009990 tx=0x7f0798009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 -- 172.21.15.7:0/4085123088 shutdown_connections 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 --2- 172.21.15.7:0/4085123088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a0104f80 0x7f07a0111490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 --2- 172.21.15.7:0/4085123088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a010a650 0x7f07a010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 --2- 172.21.15.7:0/4085123088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a01045c0 0x7f07a0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 -- 172.21.15.7:0/4085123088 >> 172.21.15.7:0/4085123088 conn(0x7f07a01000b0 msgr2=0x7f07a01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 -- 172.21.15.7:0/4085123088 shutdown_connections 2024-06-13T06:55:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.937+0000 7f079f200700 1 -- 172.21.15.7:0/4085123088 wait complete. 2024-06-13T06:55:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 Processor -- start 2024-06-13T06:55:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 -- start start 2024-06-13T06:55:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a01045c0 0x7f07a01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a010a650 0x7f07a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.939+0000 7f079f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0114960 con 0x7f07a01045c0 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f079f200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07a01147e0 con 0x7f07a010a650 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f079f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07a0114ae0 con 0x7f07a0104f80 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f079de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a01045c0 0x7f07a01a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46486/0 (socket says 172.21.15.7:46486) 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 -- 172.21.15.7:0/3598576938 learned_addr learned my addr 172.21.15.7:0/3598576938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f079e800700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a010a650 0x7f07a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 -- 172.21.15.7:0/3598576938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a010a650 msgr2=0x7f07a019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a010a650 0x7f07a019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 -- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a01045c0 msgr2=0x7f07a01a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a01045c0 0x7f07a01a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.940+0000 7f0795400700 1 -- 172.21.15.7:0/3598576938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a019b280 con 0x7f07a0104f80 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f0795400700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f078800e9c0 tx=0x7f078800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f0797400700 1 -- 172.21.15.7:0/3598576938 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800cbd0 con 0x7f07a0104f80 2024-06-13T06:55:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a019b570 con 0x7f07a0104f80 2024-06-13T06:55:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01a77e0 con 0x7f07a0104f80 2024-06-13T06:55:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f0797400700 1 -- 172.21.15.7:0/3598576938 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078800ee60 con 0x7f07a0104f80 2024-06-13T06:55:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.941+0000 7f0797400700 1 -- 172.21.15.7:0/3598576938 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788005610 con 0x7f07a0104f80 2024-06-13T06:55:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.945+0000 7f0797400700 1 -- 172.21.15.7:0/3598576938 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0788004750 con 0x7f07a0104f80 2024-06-13T06:55:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.947+0000 7f07afb8bb80 1 Processor -- start 2024-06-13T06:55:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.947+0000 7f07afb8bb80 1 -- start start 2024-06-13T06:55:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.947+0000 7f0797400700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 0x7f07840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.947+0000 7f0797400700 1 -- 172.21.15.7:0/3598576938 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f07880cf490 con 0x7f07a0104f80 2024-06-13T06:55:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.947+0000 7f079e800700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 0x7f07840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae2d10 0x5640b5ae30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 0x5640b5aedfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5af08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b59f29c0 con 0x5640b5ae2d10 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640b5a35ab0 con 0x5640b5aee4e0 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640b59f08a0 con 0x5640b5ae3630 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f079e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5af08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f079de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae2d10 0x5640b5ae30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f0795400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 0x5640b5aedfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f079e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5af08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56018/0 (socket says 172.21.15.7:56018) 2024-06-13T06:55:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.948+0000 7f079e800700 1 -- 172.21.15.7:0/1633949059 learned_addr learned my addr 172.21.15.7:0/1633949059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 -- 172.21.15.7:0/1633949059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 msgr2=0x5640b5af08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5af08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 -- 172.21.15.7:0/1633949059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae2d10 msgr2=0x5640b5ae30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae2d10 0x5640b5ae30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f079e800700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 0x7f07840a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0798015000 tx=0x7f079800b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 -- 172.21.15.7:0/1633949059 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b59ddf40 con 0x5640b5ae3630 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f0795400700 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 0x5640b5aedfa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0788005260 tx=0x7f078809b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55397c230d63381 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f078f400700 1 -- 172.21.15.7:0/1633949059 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078809fd00 con 0x5640b5ae3630 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f078f400700 1 -- 172.21.15.7:0/1633949059 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07880a45c0 con 0x5640b5ae3630 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.949+0000 7f078f400700 1 -- 172.21.15.7:0/1633949059 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078809bbb0 con 0x5640b5ae3630 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 -- 172.21.15.7:0/1633949059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 msgr2=0x5640b5aedfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 0x5640b5aedfa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0788005260 tx=0x7f078809b330 comp rx=0 tx=0).stop 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 -- 172.21.15.7:0/1633949059 shutdown_connections 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5af08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae2d10 0x5640b5ae30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/1633949059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae3630 0x5640b5aedfa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 -- 172.21.15.7:0/1633949059 >> 172.21.15.7:0/1633949059 conn(0x5640b5a10120 msgr2=0x5640b5adf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.950+0000 7f07afb8bb80 1 -- 172.21.15.7:0/1633949059 shutdown_connections 2024-06-13T06:55:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.951+0000 7f07afb8bb80 1 -- 172.21.15.7:0/1633949059 wait complete. 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.951+0000 7f07afb8bb80 1 -- start start 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.951+0000 7f07afb8bb80 1 Processor -- start 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5aee4e0 0x5640b5aeb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b59d1f10 con 0x5640b5ae3630 2024-06-13T06:55:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640b5a31a70 con 0x5640b5ae2d10 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640b5af0f70 con 0x5640b5aee4e0 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f07afb8bb80 10 client.0 _ll_register_callbacks cb 0x5640b59cb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f0795400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f079de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f0795400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47580/0 (socket says 172.21.15.7:47580) 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f079de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56034/0 (socket says 172.21.15.7:56034) 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f0795400700 1 -- 172.21.15.7:0/4003570554 learned_addr learned my addr 172.21.15.7:0/4003570554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.952+0000 7f079e800700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5aee4e0 0x5640b5aeb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 -- 172.21.15.7:0/4003570554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5aee4e0 msgr2=0x5640b5aeb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5aee4e0 0x5640b5aeb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 -- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 msgr2=0x5640b5aeafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 -- 172.21.15.7:0/4003570554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b5984880 con 0x5640b5ae2d10 2024-06-13T06:55:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f0795400700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f079de00700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0790012d90 tx=0x7f0790014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491440747cd790bc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.953+0000 7f0783e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:55:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790004bb0 con 0x5640b5ae2d10 2024-06-13T06:55:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0790004d50 con 0x5640b5ae2d10 2024-06-13T06:55:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07880a45c0 con 0x5640b5ae2d10 2024-06-13T06:55:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0790014a80 con 0x5640b5ae2d10 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f07afb8bb80 10 client.25115 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5640b5afb740 con 0x5640b5ae2d10 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07880a45c0 con 0x5640b5ae2d10 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.954+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0790004400 con 0x5640b5ae2d10 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 10 client.25115 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 10 client.25115 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 20 client.25115 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5afde30 0x5640b5aff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 4 client.25115 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640b5a42fb0 con 0x5640b5afde30 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.955+0000 7f0795400700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5afde30 0x5640b5aff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.956+0000 7f0795400700 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5afde30 0x5640b5aff2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cf650af2a7de310 server_cookie=91181cd347baf003 in_seq=0 out_seq=0 2024-06-13T06:55:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.956+0000 7f0783e00700 10 client.25115 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:55:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.956+0000 7f0783e00700 1 -- 172.21.15.7:0/4003570554 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640b5a42fb0 con 0x5640b5afde30 2024-06-13T06:55:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.956+0000 7f0783e00700 10 client.25115 handle_command_reply: tid=0 2024-06-13T06:55:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 client.25115 shutdown 2024-06-13T06:55:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 msgr2=0x5640b5aedc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0790012d90 tx=0x7f0790014600 comp rx=0 tx=0).stop 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 shutdown_connections 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5afde30 0x5640b5aff2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5aee4e0 0x5640b5aeb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae2d10 0x5640b5aedc30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/4003570554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5aeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f0783e00700 0 client.25115 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 >> 172.21.15.7:0/4003570554 conn(0x5640b5a10120 msgr2=0x5640b5adee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 shutdown_connections 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 1 -- 172.21.15.7:0/4003570554 wait complete. 2024-06-13T06:55:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.957+0000 7f07afb8bb80 20 client.25115 trim_cache size 0 max 16384 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.981+0000 7f07afb8bb80 1 Processor -- start 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 -- start start 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 0x5640b5aed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 0x5640b5b08700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b5afd5e0 con 0x5640b5aee4e0 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640b5afdcb0 con 0x5640b5ae3630 2024-06-13T06:55:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640b5afe380 con 0x5640b5ae2d10 2024-06-13T06:55:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f079e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 0x5640b5b08700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f079de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f0795400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 0x5640b5aed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f079de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46518/0 (socket says 172.21.15.7:46518) 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.982+0000 7f079de00700 1 -- 172.21.15.7:0/110891427 learned_addr learned my addr 172.21.15.7:0/110891427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 -- 172.21.15.7:0/110891427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 msgr2=0x5640b5aed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 0x5640b5aed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 -- 172.21.15.7:0/110891427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 msgr2=0x5640b5b08700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 0x5640b5b08700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 -- 172.21.15.7:0/110891427 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b5ae06e0 con 0x5640b5ae2d10 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f0795400700 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 0x5640b5aed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079e800700 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 0x5640b5b08700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.983+0000 7f079de00700 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f07900094f0 tx=0x7f0790009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d1b5f936bdbb60 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f078ea00700 1 -- 172.21.15.7:0/110891427 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079002f030 con 0x5640b5ae2d10 2024-06-13T06:55:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f078ea00700 1 -- 172.21.15.7:0/110891427 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0790014c20 con 0x5640b5ae2d10 2024-06-13T06:55:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f078ea00700 1 -- 172.21.15.7:0/110891427 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790007da0 con 0x5640b5ae2d10 2024-06-13T06:55:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 -- 172.21.15.7:0/110891427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 msgr2=0x5640b5aed3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f07900094f0 tx=0x7f0790009700 comp rx=0 tx=0).stop 2024-06-13T06:55:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 -- 172.21.15.7:0/110891427 shutdown_connections 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5aee4e0 0x5640b5b08700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5aed3a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/110891427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5ae3630 0x5640b5aed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.984+0000 7f07afb8bb80 1 -- 172.21.15.7:0/110891427 >> 172.21.15.7:0/110891427 conn(0x5640b5a10120 msgr2=0x5640b5ae9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.985+0000 7f07afb8bb80 1 -- 172.21.15.7:0/110891427 shutdown_connections 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.985+0000 7f07afb8bb80 1 -- 172.21.15.7:0/110891427 wait complete. 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.985+0000 7f07afb8bb80 1 -- start start 2024-06-13T06:55:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.985+0000 7f07afb8bb80 1 Processor -- start 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5b02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5b02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b5aff120 con 0x5640b5ae3630 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5640b5afea50 con 0x5640b5aee4e0 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640b5aff7f0 con 0x5640b5ae2d10 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f07afb8bb80 10 client.0 _ll_register_callbacks cb 0x5640b59cb330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f0795400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5b02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f079de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f079de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46530/0 (socket says 172.21.15.7:46530) 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f079de00700 1 -- 172.21.15.7:0/3492998348 learned_addr learned my addr 172.21.15.7:0/3492998348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.986+0000 7f079e800700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5b02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 -- 172.21.15.7:0/3492998348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 msgr2=0x5640b5b02f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5b02f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 -- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 msgr2=0x5640b5b02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5b02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 -- 172.21.15.7:0/3492998348 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b5ae4b60 con 0x5640b5ae2d10 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f0795400700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5b02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f079de00700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0790004870 tx=0x7f0790005f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67919a16f1f20221 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f0783e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790039060 con 0x5640b5ae2d10 2024-06-13T06:55:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.987+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0790014c20 con 0x5640b5ae2d10 2024-06-13T06:55:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0790007c20 con 0x5640b5ae2d10 2024-06-13T06:55:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0790040050 con 0x5640b5ae2d10 2024-06-13T06:55:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f07afb8bb80 10 client.25114 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5640b5b720e0 con 0x5640b5ae2d10 2024-06-13T06:55:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0790014c20 con 0x5640b5ae2d10 2024-06-13T06:55:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0790034cb0 con 0x5640b5ae2d10 2024-06-13T06:55:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.988+0000 7f07afb8bb80 10 client.25114 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f07afb8bb80 10 client.25114 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f07afb8bb80 20 client.25114 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5b76ae0 0x5640b5b757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f07afb8bb80 4 client.25114 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "status"}) -- 0x5640b5a3d530 con 0x5640b5b76ae0 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f0795400700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5b76ae0 0x5640b5b757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f0795400700 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5b76ae0 0x5640b5b757f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8134c2ae58e1a05f server_cookie=dc243041a72b2b61 in_seq=0 out_seq=0 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.989+0000 7f0783e00700 10 client.25114 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f0783e00700 1 -- 172.21.15.7:0/3492998348 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5640b5a3d530 con 0x5640b5b76ae0 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f0783e00700 10 client.25114 handle_command_reply: tid=0 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 client.25114 shutdown 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 msgr2=0x5640b5b024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0790004870 tx=0x7f0790005f40 comp rx=0 tx=0).stop 2024-06-13T06:55:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 shutdown_connections 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5640b5aee4e0 0x5640b5b02f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5640b5b76ae0 0x5640b5b757f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640b5ae2d10 0x5640b5b024e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 --2- 172.21.15.7:0/3492998348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b5ae3630 0x5640b5b02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f0783e00700 0 client.25114 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 >> 172.21.15.7:0/3492998348 conn(0x5640b5a10120 msgr2=0x5640b5af91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 shutdown_connections 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 1 -- 172.21.15.7:0/3492998348 wait complete. 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.990+0000 7f07afb8bb80 20 client.25114 trim_cache size 0 max 16384 2024-06-13T06:55:04.992 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24527, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 204.222785423, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 310.38187048399999 2024-06-13T06:55:04.993 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 msgr2=0x7f07840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 0x7f07840a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0798015000 tx=0x7f079800b880 comp rx=0 tx=0).stop 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 msgr2=0x7f07a01a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f078800e9c0 tx=0x7f078800ecd0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 shutdown_connections 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f07840a2ca0 0x7f07840a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a010a650 0x7f07a019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07a01045c0 0x7f07a01a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 --2- 172.21.15.7:0/3598576938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a0104f80 0x7f07a01a0e50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 >> 172.21.15.7:0/3598576938 conn(0x7f07a01000b0 msgr2=0x7f07a0101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 shutdown_connections 2024-06-13T06:55:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:04.991+0000 7f079f200700 1 -- 172.21.15.7:0/3598576938 wait complete. 2024-06-13T06:55:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:05 smithi007 ceph-mon[32077]: pgmap v480: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:05 smithi189 ceph-mon[36616]: pgmap v480: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:05 smithi175 ceph-mon[36886]: pgmap v480: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:05.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.b status 2024-06-13T06:55:06.307 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:07.495 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:07 smithi007 ceph-mon[32077]: pgmap v481: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:07.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:07 smithi189 ceph-mon[36616]: pgmap v481: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:07 smithi175 ceph-mon[36886]: pgmap v481: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 -- 172.21.15.7:0/3559612089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388113660 msgr2=0x7f5388115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 --2- 172.21.15.7:0/3559612089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388113660 0x7f5388115a50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5380009f90 tx=0x7f538000b330 comp rx=0 tx=0).stop 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 -- 172.21.15.7:0/3559612089 shutdown_connections 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 --2- 172.21.15.7:0/3559612089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388113660 0x7f5388115a50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 --2- 172.21.15.7:0/3559612089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53880746c0 0x7f5388074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 --2- 172.21.15.7:0/3559612089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5388074fe0 0x7f5388113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.614+0000 7f538d000700 1 -- 172.21.15.7:0/3559612089 >> 172.21.15.7:0/3559612089 conn(0x7f5388079640 msgr2=0x7f5388079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.615+0000 7f538d000700 1 -- 172.21.15.7:0/3559612089 shutdown_connections 2024-06-13T06:55:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.615+0000 7f538d000700 1 -- 172.21.15.7:0/3559612089 wait complete. 2024-06-13T06:55:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 Processor -- start 2024-06-13T06:55:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 -- start start 2024-06-13T06:55:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 0x7f538819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 0x7f538819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5388113660 0x7f53881a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118d90 con 0x7f53880746c0 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118c10 con 0x7f5388074fe0 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.616+0000 7f538d000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5388118f10 con 0x7f5388113660 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 0x7f538819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 0x7f538819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385e00700 1 -- 172.21.15.7:0/2987611114 learned_addr learned my addr 172.21.15.7:0/2987611114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5386800700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5388113660 0x7f53881a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 0x7f538819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 -- 172.21.15.7:0/2987611114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5388113660 msgr2=0x7f53881a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5388113660 0x7f53881a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 -- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 msgr2=0x7f538819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 0x7f538819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.617+0000 7f5385400700 1 -- 172.21.15.7:0/2987611114 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53881a4270 con 0x7f5388074fe0 2024-06-13T06:55:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f5385400700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 0x7f538819f7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f537ea00700 1 -- 172.21.15.7:0/2987611114 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537400cb70 con 0x7f5388074fe0 2024-06-13T06:55:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f537ea00700 1 -- 172.21.15.7:0/2987611114 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537400ee90 con 0x7f5388074fe0 2024-06-13T06:55:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881a4560 con 0x7f5388074fe0 2024-06-13T06:55:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f537ea00700 1 -- 172.21.15.7:0/2987611114 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374003dc0 con 0x7f5388074fe0 2024-06-13T06:55:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.618+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881abde0 con 0x7f5388074fe0 2024-06-13T06:55:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.622+0000 7f537ea00700 1 -- 172.21.15.7:0/2987611114 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5374004000 con 0x7f5388074fe0 2024-06-13T06:55:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.623+0000 7f53979bab80 1 Processor -- start 2024-06-13T06:55:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 -- start start 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f74825cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74826630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f747349c0 con 0x561f7482c460 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f74777ab0 con 0x561f748261f0 2024-06-13T06:55:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.624+0000 7f53979bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561f747328a0 con 0x561f7482bb40 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5386800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5386800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46550/0 (socket says 172.21.15.7:46550) 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5386800700 1 -- 172.21.15.7:0/1832239741 learned_addr learned my addr 172.21.15.7:0/1832239741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f537ea00700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 0x7f536c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f74825cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385400700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74826630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 -- 172.21.15.7:0/1832239741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 msgr2=0x561f7482bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 -- 172.21.15.7:0/1832239741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 msgr2=0x561f74826630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f537ea00700 1 -- 172.21.15.7:0/2987611114 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f53740cf3d0 con 0x7f5388074fe0 2024-06-13T06:55:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74826630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5385e00700 1 -- 172.21.15.7:0/1832239741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7471ff40 con 0x561f7482c460 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5386800700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.625+0000 7f5386800700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 0x7f536c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.626+0000 7f5385400700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74826630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.626+0000 7f5385e00700 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f74825cb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5378009f10 tx=0x7f5378011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de96bbc179c00eb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.626+0000 7f5373400700 1 -- 172.21.15.7:0/1832239741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537801e070 con 0x561f7482c460 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.626+0000 7f5373400700 1 -- 172.21.15.7:0/1832239741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537800eb50 con 0x561f7482c460 2024-06-13T06:55:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.626+0000 7f5373400700 1 -- 172.21.15.7:0/1832239741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378019ca0 con 0x561f7482c460 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 -- 172.21.15.7:0/1832239741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 msgr2=0x561f74825cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f74825cb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5378009f10 tx=0x7f5378011720 comp rx=0 tx=0).stop 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f5386800700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 0x7f536c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5380009da0 tx=0x7f5380009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 -- 172.21.15.7:0/1832239741 shutdown_connections 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74826630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f7482bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 --2- 172.21.15.7:0/1832239741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f74825cb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.627+0000 7f53979bab80 1 -- 172.21.15.7:0/1832239741 >> 172.21.15.7:0/1832239741 conn(0x561f74752120 msgr2=0x561f74820fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.628+0000 7f53979bab80 1 -- 172.21.15.7:0/1832239741 shutdown_connections 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.628+0000 7f53979bab80 1 -- 172.21.15.7:0/1832239741 wait complete. 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.628+0000 7f53979bab80 1 -- start start 2024-06-13T06:55:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.628+0000 7f53979bab80 1 Processor -- start 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482bb40 0x561f7482f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482c460 0x561f7482f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f74713f10 con 0x561f7482bb40 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f74773a70 con 0x561f748261f0 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561f74832e40 con 0x561f7482c460 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f53979bab80 10 client.0 _ll_register_callbacks cb 0x561f7470d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f5385e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f5385e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56076/0 (socket says 172.21.15.7:56076) 2024-06-13T06:55:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f5385e00700 1 -- 172.21.15.7:0/2750488501 learned_addr learned my addr 172.21.15.7:0/2750488501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.629+0000 7f5385400700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482bb40 0x561f7482f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5386800700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482c460 0x561f7482f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 -- 172.21.15.7:0/2750488501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482c460 msgr2=0x561f7482f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482c460 0x561f7482f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 -- 172.21.15.7:0/2750488501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482bb40 msgr2=0x561f7482f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482bb40 0x561f7482f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 -- 172.21.15.7:0/2750488501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f746c6880 con 0x561f748261f0 2024-06-13T06:55:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5385e00700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f537800a010 tx=0x7f537800e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428bf8d4dccbf5c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5367e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:55:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.630+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378027020 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5378019880 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537800eb50 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5378032050 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f53979bab80 10 client.25133 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f7483d5d0 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.631+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537800eb50 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f53780047e0 con 0x561f748261f0 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 10 client.25133 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 10 client.25133 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 20 client.25133 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f7483f930 0x561f74840da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 4 client.25133 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f74784fb0 con 0x561f7483f930 2024-06-13T06:55:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.632+0000 7f5385400700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f7483f930 0x561f74840da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.633+0000 7f5385400700 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f7483f930 0x561f74840da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7d7866eee54a91 server_cookie=f4dcce75c57ddd6a in_seq=0 out_seq=0 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.633+0000 7f5367e00700 10 client.25133 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f5367e00700 1 -- 172.21.15.7:0/2750488501 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561f74784fb0 con 0x561f7483f930 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f5367e00700 10 client.25133 handle_command_reply: tid=0 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 client.25133 shutdown 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 msgr2=0x561f74832180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f537800a010 tx=0x7f537800e6a0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 shutdown_connections 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f7483f930 0x561f74840da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482c460 0x561f7482f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f748261f0 0x561f74832180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 --2- 172.21.15.7:0/2750488501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482bb40 0x561f7482f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 >> 172.21.15.7:0/2750488501 conn(0x561f74752120 msgr2=0x561f7481a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.634+0000 7f5367e00700 0 client.25133 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.635+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 shutdown_connections 2024-06-13T06:55:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.635+0000 7f53979bab80 1 -- 172.21.15.7:0/2750488501 wait complete. 2024-06-13T06:55:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.635+0000 7f53979bab80 20 client.25133 trim_cache size 0 max 16384 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 Processor -- start 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 -- start start 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 0x561f74825cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f74821150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f7483f0e0 con 0x561f748261f0 2024-06-13T06:55:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f7483f7b0 con 0x561f7482c460 2024-06-13T06:55:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.659+0000 7f53979bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561f7483fe80 con 0x561f7482bb40 2024-06-13T06:55:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5385e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 0x561f74825cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56088/0 (socket says 172.21.15.7:56088) 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5385400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f74821150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 -- 172.21.15.7:0/1412526995 learned_addr learned my addr 172.21.15.7:0/1412526995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5385e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 0x561f74825cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47648/0 (socket says 172.21.15.7:47648) 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 -- 172.21.15.7:0/1412526995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 msgr2=0x561f74821150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f74821150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 -- 172.21.15.7:0/1412526995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 msgr2=0x561f74825cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 0x561f74825cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.660+0000 7f5386800700 1 -- 172.21.15.7:0/1412526995 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7482d9f0 con 0x561f7482c460 2024-06-13T06:55:08.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.661+0000 7f5386800700 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f538000ba40 tx=0x7f538001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3591b64f0c95f5a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.661+0000 7f5372a00700 1 -- 172.21.15.7:0/1412526995 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538002d070 con 0x561f7482c460 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.661+0000 7f5372a00700 1 -- 172.21.15.7:0/1412526995 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5380021040 con 0x561f7482c460 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.661+0000 7f5372a00700 1 -- 172.21.15.7:0/1412526995 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538001db20 con 0x561f7482c460 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 -- 172.21.15.7:0/1412526995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 msgr2=0x561f7481b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f538000ba40 tx=0x7f538001f000 comp rx=0 tx=0).stop 2024-06-13T06:55:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 -- 172.21.15.7:0/1412526995 shutdown_connections 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482c460 0x561f7481b0e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f748261f0 0x561f74825cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 --2- 172.21.15.7:0/1412526995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f7482bb40 0x561f74821150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 -- 172.21.15.7:0/1412526995 >> 172.21.15.7:0/1412526995 conn(0x561f74752120 msgr2=0x561f74820a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 -- 172.21.15.7:0/1412526995 shutdown_connections 2024-06-13T06:55:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.662+0000 7f53979bab80 1 -- 172.21.15.7:0/1412526995 wait complete. 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.663+0000 7f53979bab80 1 -- start start 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.663+0000 7f53979bab80 1 Processor -- start 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.663+0000 7f53979bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 0x561f7484a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482bb40 0x561f74830f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f748314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f74840c20 con 0x561f7482c460 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f74840550 con 0x561f7482bb40 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561f748412f0 con 0x561f748261f0 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f53979bab80 10 client.0 _ll_register_callbacks cb 0x561f7470d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5386800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f748314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 0x561f7484a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5386800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f748314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47662/0 (socket says 172.21.15.7:47662) 2024-06-13T06:55:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5386800700 1 -- 172.21.15.7:0/1931320367 learned_addr learned my addr 172.21.15.7:0/1931320367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385400700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482bb40 0x561f74830f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 -- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482bb40 msgr2=0x561f74830f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482bb40 0x561f74830f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 -- 172.21.15.7:0/1931320367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 msgr2=0x561f748314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f748314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.664+0000 7f5385e00700 1 -- 172.21.15.7:0/1931320367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f74831ac0 con 0x561f748261f0 2024-06-13T06:55:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.665+0000 7f5385e00700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 0x561f7484a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5378009e60 tx=0x7f5378005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d217b1fedaf465f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.665+0000 7f5367e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.665+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378027040 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.665+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5380021040 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.665+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5378005200 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f537801e070 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 10 client.25132 fetch_fsmap learned FSMap version 25 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f74849070 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5380021040 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5378005440 con 0x561f748261f0 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 10 client.25132 fetch_fsmap finished waiting for FSMap version 25 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 10 client.25132 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 20 client.25132 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:08.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f748afb10 0x561f748ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 4 client.25132 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-13T06:55:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.666+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "status"}) -- 0x561f7477f530 con 0x561f748afb10 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.667+0000 7f5385400700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f748afb10 0x561f748ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.667+0000 7f5385400700 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f748afb10 0x561f748ae820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176242390a4aa91d server_cookie=c5d30c76ae659929 in_seq=0 out_seq=0 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.667+0000 7f5367e00700 10 client.25132 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f5367e00700 1 -- 172.21.15.7:0/1931320367 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f7477f530 con 0x561f748afb10 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f5367e00700 10 client.25132 handle_command_reply: tid=0 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 client.25132 shutdown 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 msgr2=0x561f7484a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 0x561f7484a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5378009e60 tx=0x7f5378005af0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 shutdown_connections 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561f748afb10 0x561f748ae820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f7482c460 0x561f748314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561f748261f0 0x561f7484a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 --2- 172.21.15.7:0/1931320367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f7482bb40 0x561f74830f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f5367e00700 0 client.25132 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 >> 172.21.15.7:0/1931320367 conn(0x561f74752120 msgr2=0x561f7482c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 shutdown_connections 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 1 -- 172.21.15.7:0/1931320367 wait complete. 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.668+0000 7f53979bab80 20 client.25132 trim_cache size 0 max 16384 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "b9295806-294f-11ef-bc9c-c7b262605968", 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-06-13T06:55:08.669 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24539, 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 289.16685859699999, 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 311.04393804099999 2024-06-13T06:55:08.670 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 msgr2=0x7f536c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 0x7f536c0a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5380009da0 tx=0x7f5380009920 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 msgr2=0x7f538819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 0x7f538819f7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400ed00 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 shutdown_connections 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5388113660 0x7f53881a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53880746c0 0x7f538819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f536c0a2ca0 0x7f536c0a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 --2- 172.21.15.7:0/2987611114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5388074fe0 0x7f538819f7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:08.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.669+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 >> 172.21.15.7:0/2987611114 conn(0x7f5388079640 msgr2=0x7f53881123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:08.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.670+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 shutdown_connections 2024-06-13T06:55:08.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:08.670+0000 7f538d000700 1 -- 172.21.15.7:0/2987611114 wait complete. 2024-06-13T06:55:09.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:09 smithi007 ceph-mon[32077]: pgmap v482: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:55:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:09 smithi189 ceph-mon[36616]: pgmap v482: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:55:09.607 INFO:teuthology.run_tasks:Running task sequential... 2024-06-13T06:55:09.617 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-13T06:55:09.617 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-13T06:55:09.617 INFO:teuthology.task.print:Enabling mgr modules 2024-06-13T06:55:09.617 INFO:teuthology.run_tasks:Running task kclient... 2024-06-13T06:55:09.648 INFO:tasks.kclient:Mounting kernel clients... 2024-06-13T06:55:09.648 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-13T06:55:09.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:09.648 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-13T06:55:09.649 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:09 smithi175 ceph-mon[36886]: pgmap v482: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-13T06:55:09.702 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-13T06:55:09.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:09.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-13T06:55:09.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:09.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-06-13T06:55:09.895 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-06-13T06:55:09.895 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-13T06:55:09.895 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-06-13T06:55:09.895 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-06-13T06:55:09.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs ls 2024-06-13T06:55:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:11.490 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:11 smithi007 ceph-mon[32077]: pgmap v483: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:11 smithi189 ceph-mon[36616]: pgmap v483: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:11 smithi175 ceph-mon[36886]: pgmap v483: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-13T06:55:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 -- 172.21.15.7:0/1951305499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c58105070 msgr2=0x7f9c58111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 --2- 172.21.15.7:0/1951305499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c58105070 0x7f9c58111580 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000b0a0 tx=0x7f9c5000b3b0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 -- 172.21.15.7:0/1951305499 shutdown_connections 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 --2- 172.21.15.7:0/1951305499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c58105070 0x7f9c58111580 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 --2- 172.21.15.7:0/1951305499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c5810a6b0 0x7f9c5810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 --2- 172.21.15.7:0/1951305499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c581046b0 0x7f9c58104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 -- 172.21.15.7:0/1951305499 >> 172.21.15.7:0/1951305499 conn(0x7f9c581001a0 msgr2=0x7f9c581025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 -- 172.21.15.7:0/1951305499 shutdown_connections 2024-06-13T06:55:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.642+0000 7f9c5d200700 1 -- 172.21.15.7:0/1951305499 wait complete. 2024-06-13T06:55:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.643+0000 7f9c5d200700 1 Processor -- start 2024-06-13T06:55:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 -- start start 2024-06-13T06:55:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 0x7f9c5819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c5810a6b0 0x7f9c5819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58114a40 con 0x7f9c5810a6b0 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581148c0 con 0x7f9c581046b0 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c5d200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58114bc0 con 0x7f9c58105070 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c55e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c55400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 0x7f9c5819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c55e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47416/0 (socket says 172.21.15.7:47416) 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.644+0000 7f9c55e00700 1 -- 172.21.15.7:0/3783059694 learned_addr learned my addr 172.21.15.7:0/3783059694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c56800700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c5810a6b0 0x7f9c5819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 -- 172.21.15.7:0/3783059694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 msgr2=0x7f9c5819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 0x7f9c5819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 -- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c5810a6b0 msgr2=0x7f9c5819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c5810a6b0 0x7f9c5819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 -- 172.21.15.7:0/3783059694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5819fde0 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55400700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 0x7f9c5819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.645+0000 7f9c55e00700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800e9a0 tx=0x7f9c4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.646+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800ca10 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.646+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4800cbb0 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.646+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48018680 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.646+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c581a00d0 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.646+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581a7950 con 0x7f9c581046b0 2024-06-13T06:55:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.647+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58069180 con 0x7f9c581046b0 2024-06-13T06:55:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.649+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9c48010430 con 0x7f9c581046b0 2024-06-13T06:55:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.650+0000 7f9c4ea00700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 0x7f9c3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.651+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9c480cf9f0 con 0x7f9c581046b0 2024-06-13T06:55:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.651+0000 7f9c55400700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 0x7f9c3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.651+0000 7f9c55400700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 0x7f9c3c0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009680 tx=0x7f9c44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.653+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c4809c5b0 con 0x7f9c581046b0 2024-06-13T06:55:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.939+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9c581a7b80 con 0x7f9c581046b0 2024-06-13T06:55:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.940+0000 7f9c4ea00700 1 -- 172.21.15.7:0/3783059694 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f9c4809beb0 con 0x7f9c581046b0 2024-06-13T06:55:12.939 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-13T06:55:12.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.942+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 msgr2=0x7f9c3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:12.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.942+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 0x7f9c3c0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009680 tx=0x7f9c44009260 comp rx=0 tx=0).stop 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 msgr2=0x7f9c5819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800e9a0 tx=0x7f9c4800ecb0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 shutdown_connections 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c5810a6b0 0x7f9c5819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9c3c0a2c20 0x7f9c3c0a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c581046b0 0x7f9c5819add0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 --2- 172.21.15.7:0/3783059694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c58105070 0x7f9c5819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:12.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 >> 172.21.15.7:0/3783059694 conn(0x7f9c581001a0 msgr2=0x7f9c58100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:12.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 shutdown_connections 2024-06-13T06:55:12.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:12.943+0000 7f9c5d200700 1 -- 172.21.15.7:0/3783059694 wait complete. 2024-06-13T06:55:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:13 smithi189 ceph-mon[36616]: pgmap v484: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:13 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3783059694' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:13 smithi175 ceph-mon[36886]: pgmap v484: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:13 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3783059694' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:13.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:13 smithi007 ceph-mon[32077]: pgmap v484: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-13T06:55:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:13 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3783059694' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-13T06:55:13.836 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-06-13T06:55:13.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:13.837 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-06-13T06:55:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-13T06:55:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-13T06:55:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-06-13T06:55:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41728sec preferred_lft 41728sec 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-06-13T06:55:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-06-13T06:55:13.899 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-13T06:55:13.899 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:13.899 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:55:13.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-06-13T06:55:13.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-06-13T06:55:13.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-06-13T06:55:13.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-13T06:55:13.900 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:55:14.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:14.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:14.160 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-13T06:55:14.230 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-06-13T06:55:14.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:14.234 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-06-13T06:55:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-06-13T06:55:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-13T06:55:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-13T06:55:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-13T06:55:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-13T06:55:14.285 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:14.285 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:55:14.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-13T06:55:14.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-13T06:55:14.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-13T06:55:14.286 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:55:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:14.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:14.513 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-13T06:55:14.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:14.564 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-06-13T06:55:14.618 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:14.618 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:55:14.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--tmp-cephfs 2024-06-13T06:55:14.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-06-13T06:55:14.618 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:55:14.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:14.815 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-06-13T06:55:14.815 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:14.815 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-06-13T06:55:14.816 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:55:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:15.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:15.099 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-13T06:55:15.100 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-06-13T06:55:15.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-06-13T06:55:15.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-06-13T06:55:15.100 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-06-13T06:55:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:15.278 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:15 smithi007 ceph-mon[32077]: pgmap v485: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:15.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:55:15.303 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /tmp/cephfs 2024-06-13T06:55:15.353 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/tmp/cephfs' 2024-06-13T06:55:15.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-13T06:55:15.353 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /tmp/cephfs 2024-06-13T06:55:15.406 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-13T06:55:15.406 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-13T06:55:15.406 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 2024-06-13T06:55:15.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:15.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-06-13T06:55:15.543 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-06-13T06:55:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-06-13T06:55:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:15 smithi189 ceph-mon[36616]: pgmap v485: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:15.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:15 smithi175 ceph-mon[36886]: pgmap v485: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:16.285 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-06-13T06:55:16.286 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-06-13T06:55:16.286 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.175:3300,172.21.15.189:3300" 2024-06-13T06:55:16.286 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@b9295806-294f-11ef-bc9c-c7b262605968.cephfs=/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 2024-06-13T06:55:16.286 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.7:3300/172.21.15.175:3300/172.21.15.189:3300" will pass to kernel 2024-06-13T06:55:16.299 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-13T06:55:16.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T06:55:16.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /tmp/cephfs 2024-06-13T06:55:16.400 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T06:55:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T06:55:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:client25141 2024-06-13T06:55:16.536 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T06:55:16.536 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T06:55:16.594 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T06:55:16.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T06:55:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:16.717 INFO:teuthology.orchestra.run.smithi007.stdout:client25141 2024-06-13T06:55:16.717 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-13T06:55:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-13T06:55:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:16.840 INFO:teuthology.orchestra.run.smithi007.stdout:client25141 2024-06-13T06:55:16.840 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b9295806-294f-11ef-bc9c-c7b262605968.client25141/status') 2024-06-13T06:55:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-13T06:55:16.983 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.25141 (3)172.21.15.7:0/4112401381 2024-06-13T06:55:16.984 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-06-13T06:55:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-06-13T06:55:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-06-13T06:55:16.984 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 0.000102556 s, 683 kB/s 2024-06-13T06:55:16.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-13T06:55:16.998 INFO:teuthology.run_tasks:Running task exec... 2024-06-13T06:55:17.010 INFO:teuthology.task.exec:Executing custom commands... 2024-06-13T06:55:17.010 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:55:17.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-13T06:55:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 -- 172.21.15.7:0/2749459036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701033f0 msgr2=0x7f8170109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 --2- 172.21.15.7:0/2749459036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701033f0 0x7f8170109c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8168009ca0 tx=0x7f816802a8c0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 -- 172.21.15.7:0/2749459036 shutdown_connections 2024-06-13T06:55:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 --2- 172.21.15.7:0/2749459036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81701033f0 0x7f8170109c80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 --2- 172.21.15.7:0/2749459036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8170102a30 0x7f8170102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 --2- 172.21.15.7:0/2749459036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170101840 0x7f8170101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 -- 172.21.15.7:0/2749459036 >> 172.21.15.7:0/2749459036 conn(0x7f81700fcdb0 msgr2=0x7f81700ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 -- 172.21.15.7:0/2749459036 shutdown_connections 2024-06-13T06:55:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.220+0000 7f8176800640 1 -- 172.21.15.7:0/2749459036 wait complete. 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.221+0000 7f8176800640 1 Processor -- start 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.221+0000 7f8176800640 1 -- start start 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170102a30 0x7f8170198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701033f0 0x7f817019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d510 con 0x7f8170101840 2024-06-13T06:55:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d390 con 0x7f81701033f0 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8176800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f817010d690 con 0x7f8170102a30 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8175400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8175400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58436/0 (socket says 172.21.15.7:58436) 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8175400640 1 -- 172.21.15.7:0/3846580600 learned_addr learned my addr 172.21.15.7:0/3846580600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8174a00640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170102a30 0x7f8170198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8175e00640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701033f0 0x7f817019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.222+0000 7f8175400640 1 -- 172.21.15.7:0/3846580600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170102a30 msgr2=0x7f8170198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8175400640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170102a30 0x7f8170198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8175400640 1 -- 172.21.15.7:0/3846580600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701033f0 msgr2=0x7f817019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8175400640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701033f0 0x7f817019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8175400640 1 -- 172.21.15.7:0/3846580600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817019f720 con 0x7f8170101840 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8175400640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9c0 tx=0x7f816400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400cd80 con 0x7f8170101840 2024-06-13T06:55:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81640035f0 con 0x7f8170101840 2024-06-13T06:55:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010690 con 0x7f8170101840 2024-06-13T06:55:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817019fa10 con 0x7f8170101840 2024-06-13T06:55:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.223+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817019ff60 con 0x7f8170101840 2024-06-13T06:55:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.225+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8170108b60 con 0x7f8170101840 2024-06-13T06:55:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.227+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f81640037a0 con 0x7f8170101840 2024-06-13T06:55:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.228+0000 7f816e000640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 0x7f81580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.228+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f81640cf6e0 con 0x7f8170101840 2024-06-13T06:55:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.228+0000 7f8174a00640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 0x7f81580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.229+0000 7f8174a00640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 0x7f81580a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8170199010 tx=0x7f8160005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.233+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f816409c2a0 con 0x7f8170101840 2024-06-13T06:55:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:17 smithi007 ceph-mon[32077]: pgmap v486: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:17.481+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f817002d2e0 con 0x7f8170101840 2024-06-13T06:55:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:17 smithi189 ceph-mon[36616]: pgmap v486: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:17.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:17 smithi175 ceph-mon[36886]: pgmap v486: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-13T06:55:18.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.213+0000 7f816e000640 1 -- 172.21.15.7:0/3846580600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f816409bba0 con 0x7f8170101840 2024-06-13T06:55:18.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 msgr2=0x7f81580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 0x7f81580a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8170199010 tx=0x7f8160005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:18.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 msgr2=0x7f8170197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9c0 tx=0x7f816400ee80 comp rx=0 tx=0).stop 2024-06-13T06:55:18.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 shutdown_connections 2024-06-13T06:55:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81580a2c00 0x7f81580a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81701033f0 0x7f817019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8170102a30 0x7f8170198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 --2- 172.21.15.7:0/3846580600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8170101840 0x7f8170197b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.217+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 >> 172.21.15.7:0/3846580600 conn(0x7f81700fcdb0 msgr2=0x7f8170105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.218+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 shutdown_connections 2024-06-13T06:55:18.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.218+0000 7f8176800640 1 -- 172.21.15.7:0/3846580600 wait complete. 2024-06-13T06:55:18.239 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-13T06:55:18.251 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-13T06:55:18.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-13T06:55:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 -- 172.21.15.7:0/3879372227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 msgr2=0x7f71f8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 --2- 172.21.15.7:0/3879372227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f8102e80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009900 tx=0x7f71e002a3f0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 -- 172.21.15.7:0/3879372227 shutdown_connections 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 --2- 172.21.15.7:0/3879372227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f81033c0 0x7f71f8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 --2- 172.21.15.7:0/3879372227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f8102e80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 --2- 172.21.15.7:0/3879372227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8101810 0x7f71f8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 -- 172.21.15.7:0/3879372227 >> 172.21.15.7:0/3879372227 conn(0x7f71f80fcd80 msgr2=0x7f71f80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 -- 172.21.15.7:0/3879372227 shutdown_connections 2024-06-13T06:55:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.501+0000 7f71f7200640 1 -- 172.21.15.7:0/3879372227 wait complete. 2024-06-13T06:55:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.502+0000 7f71f7200640 1 Processor -- start 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.502+0000 7f71f7200640 1 -- start start 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f8101810 0x7f71f8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 0x7f71f81a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810d490 con 0x7f71f8102a00 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810d310 con 0x7f71f8101810 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71f810d610 con 0x7f71f81033c0 2024-06-13T06:55:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58448/0 (socket says 172.21.15.7:58448) 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71efe00640 1 -- 172.21.15.7:0/445734562 learned_addr learned my addr 172.21.15.7:0/445734562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f8101810 0x7f71f8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 0x7f71f81a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 -- 172.21.15.7:0/445734562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f8101810 msgr2=0x7f71f8199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f8101810 0x7f71f8199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 -- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 msgr2=0x7f71f819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.503+0000 7f71f6800640 1 -- 172.21.15.7:0/445734562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f81a19b0 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71f6800640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 0x7f71f81a1310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009fd0 tx=0x7f71f000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f000ce20 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81a1ca0 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0005020 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010720 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.504+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a21a0 con 0x7f71f81033c0 2024-06-13T06:55:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.505+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f81a23d0 con 0x7f71f81033c0 2024-06-13T06:55:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.507+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f71f0005540 con 0x7f71f81033c0 2024-06-13T06:55:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.509+0000 7f71eea00640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.509+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f71f00cfb80 con 0x7f71f81033c0 2024-06-13T06:55:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.509+0000 7f71f5e00640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.510+0000 7f71f5e00640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 0x7f71dc0a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71f819b270 tx=0x7f71e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.513+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71f009c740 con 0x7f71f81033c0 2024-06-13T06:55:18.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:18 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T06:55:18.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:18 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T06:55:18.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:18 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-13T06:55:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.719+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71f81a25a0 con 0x7f71f81033c0 2024-06-13T06:55:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.721+0000 7f71eea00640 1 -- 172.21.15.7:0/445734562 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f71f007ae20 con 0x7f71f81033c0 2024-06-13T06:55:18.721 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:18.721 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 msgr2=0x7f71dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 0x7f71dc0a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71f819b270 tx=0x7f71e8005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 msgr2=0x7f71f81a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 0x7f71f81a1310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009fd0 tx=0x7f71f000ef10 comp rx=0 tx=0).stop 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 shutdown_connections 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71dc0a2b90 0x7f71dc0a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f81033c0 0x7f71f81a1310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8102a00 0x7f71f819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 --2- 172.21.15.7:0/445734562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f8101810 0x7f71f8199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 >> 172.21.15.7:0/445734562 conn(0x7f71f80fcd80 msgr2=0x7f71f8073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.725+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 shutdown_connections 2024-06-13T06:55:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.726+0000 7f71f7200640 1 -- 172.21.15.7:0/445734562 wait complete. 2024-06-13T06:55:18.744 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:44:44.175120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-13T06:50:10.636472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-13T06:50:15.402440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-13T06:50:28.238859+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-13T06:55:18.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.951+0000 7ff47be00640 1 -- 172.21.15.7:0/1122928286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 msgr2=0x7ff47c10bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.951+0000 7ff47be00640 1 --2- 172.21.15.7:0/1122928286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c10bf20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff474009ca0 tx=0x7ff47402a8c0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 -- 172.21.15.7:0/1122928286 shutdown_connections 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 --2- 172.21.15.7:0/1122928286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c10bf20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 --2- 172.21.15.7:0/1122928286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c104cd0 0x7ff47c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 --2- 172.21.15.7:0/1122928286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c103ae0 0x7ff47c103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 -- 172.21.15.7:0/1122928286 >> 172.21.15.7:0/1122928286 conn(0x7ff47c0ff050 msgr2=0x7ff47c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 -- 172.21.15.7:0/1122928286 shutdown_connections 2024-06-13T06:55:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.952+0000 7ff47be00640 1 -- 172.21.15.7:0/1122928286 wait complete. 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 Processor -- start 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 -- start start 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c103ae0 0x7ff47c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10f760 con 0x7ff47c105690 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10f5e0 con 0x7ff47c104cd0 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.953+0000 7ff47be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c10f8e0 con 0x7ff47c103ae0 2024-06-13T06:55:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47460/0 (socket says 172.21.15.7:47460) 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 -- 172.21.15.7:0/2267118176 learned_addr learned my addr 172.21.15.7:0/2267118176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff47aa00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c103ae0 0x7ff47c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff47b400640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 -- 172.21.15.7:0/2267118176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c103ae0 msgr2=0x7ff47c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c103ae0 0x7ff47c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 -- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 msgr2=0x7ff47c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 -- 172.21.15.7:0/2267118176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c1a19d0 con 0x7ff47c104cd0 2024-06-13T06:55:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff47b400640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.954+0000 7ff473e00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e9f0 tx=0x7ff46400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.955+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46400cd80 con 0x7ff47c104cd0 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.955+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c1a1cc0 con 0x7ff47c104cd0 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.955+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff464005500 con 0x7ff47c104cd0 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.955+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464010690 con 0x7ff47c104cd0 2024-06-13T06:55:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.955+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c1a21c0 con 0x7ff47c104cd0 2024-06-13T06:55:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.956+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c10ae00 con 0x7ff47c104cd0 2024-06-13T06:55:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.958+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff4640046c0 con 0x7ff47c104cd0 2024-06-13T06:55:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.959+0000 7ff473400640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 0x7ff4600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.960+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff4640cf940 con 0x7ff47c104cd0 2024-06-13T06:55:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.960+0000 7ff47aa00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 0x7ff4600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.960+0000 7ff47aa00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 0x7ff4600a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff47c19b290 tx=0x7ff46c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:18.962+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff46409c500 con 0x7ff47c104cd0 2024-06-13T06:55:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.164+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff47c02d470 con 0x7ff47c104cd0 2024-06-13T06:55:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.165+0000 7ff473400640 1 -- 172.21.15.7:0/2267118176 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7ff46409be00 con 0x7ff47c104cd0 2024-06-13T06:55:19.164 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-13T06:55:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 msgr2=0x7ff4600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 0x7ff4600a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff47c19b290 tx=0x7ff46c005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 msgr2=0x7ff47c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff46400e9f0 tx=0x7ff46400edf0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 shutdown_connections 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff4600a2b90 0x7ff4600a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff47c105690 0x7ff47c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff47c104cd0 0x7ff47c19a300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 --2- 172.21.15.7:0/2267118176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff47c103ae0 0x7ff47c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 >> 172.21.15.7:0/2267118176 conn(0x7ff47c0ff050 msgr2=0x7ff47c107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 shutdown_connections 2024-06-13T06:55:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.167+0000 7ff47be00640 1 -- 172.21.15.7:0/2267118176 wait complete. 2024-06-13T06:55:19.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-13T06:55:19.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 -- 172.21.15.7:0/3440510343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 msgr2=0x7fc900077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 --2- 172.21.15.7:0/3440510343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc900077a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009900 tx=0x7fc8ec02a3f0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 -- 172.21.15.7:0/3440510343 shutdown_connections 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 --2- 172.21.15.7:0/3440510343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900077f40 0x7fc9000787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 --2- 172.21.15.7:0/3440510343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc900077a00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 --2- 172.21.15.7:0/3440510343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900076360 0x7fc900076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 -- 172.21.15.7:0/3440510343 >> 172.21.15.7:0/3440510343 conn(0x7fc9000ff050 msgr2=0x7fc900101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 -- 172.21.15.7:0/3440510343 shutdown_connections 2024-06-13T06:55:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.400+0000 7fc906e00640 1 -- 172.21.15.7:0/3440510343 wait complete. 2024-06-13T06:55:19.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.402+0000 7fc906e00640 1 Processor -- start 2024-06-13T06:55:19.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.402+0000 7fc906e00640 1 -- start start 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.402+0000 7fc906e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.402+0000 7fc906e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc90019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.402+0000 7fc906e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 0x7fc9001a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc906e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc900113b30 con 0x7fc9000775a0 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc906e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001139b0 con 0x7fc900076360 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc906e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc900113cb0 con 0x7fc900077f40 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc90019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc906400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 0x7fc9001a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47484/0 (socket says 172.21.15.7:47484) 2024-06-13T06:55:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 -- 172.21.15.7:0/1416018018 learned_addr learned my addr 172.21.15.7:0/1416018018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 -- 172.21.15.7:0/1416018018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 msgr2=0x7fc9001a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 0x7fc9001a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 -- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 msgr2=0x7fc90019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc90019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc905a00640 1 -- 172.21.15.7:0/1416018018 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9001a5e00 con 0x7fc900076360 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.403+0000 7fc906400640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 0x7fc9001a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc905a00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ea60 tx=0x7fc8f400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400cde0 con 0x7fc900076360 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f40035f0 con 0x7fc900076360 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001a60f0 con 0x7fc900076360 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4010670 con 0x7fc900076360 2024-06-13T06:55:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.404+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a6610 con 0x7fc900076360 2024-06-13T06:55:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.406+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9001105d0 con 0x7fc900076360 2024-06-13T06:55:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.407+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc8f40037a0 con 0x7fc900076360 2024-06-13T06:55:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.409+0000 7fc8fe000640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 0x7fc8e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.409+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc8f40cf690 con 0x7fc900076360 2024-06-13T06:55:19.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.409+0000 7fc905000640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 0x7fc8e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.410+0000 7fc905000640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 0x7fc8e80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0314e0 tx=0x7fc8ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.413+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc8f409c250 con 0x7fc900076360 2024-06-13T06:55:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:19 smithi189 ceph-mon[36616]: pgmap v487: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-13T06:55:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:19 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T06:55:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:19 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:19 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/445734562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:19 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2267118176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:55:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:19 smithi175 ceph-mon[36886]: pgmap v487: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-13T06:55:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:19 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T06:55:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:19 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:19 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/445734562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:19 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2267118176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:55:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.621+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc9000625a0 con 0x7fc900076360 2024-06-13T06:55:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.622+0000 7fc8fe000640 1 -- 172.21.15.7:0/1416018018 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fc8f409bb50 con 0x7fc900076360 2024-06-13T06:55:19.621 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.624+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 msgr2=0x7fc8e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.624+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 0x7fc8e80a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0314e0 tx=0x7fc8ec000bc0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.624+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 msgr2=0x7fc90019e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.624+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ea60 tx=0x7fc8f400ef20 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 shutdown_connections 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc8e80a2b90 0x7fc8e80a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc900077f40 0x7fc9001a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9000775a0 0x7fc90019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 --2- 172.21.15.7:0/1416018018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc900076360 0x7fc90019e1e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 >> 172.21.15.7:0/1416018018 conn(0x7fc9000ff050 msgr2=0x7fc9001007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 shutdown_connections 2024-06-13T06:55:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.625+0000 7fc906e00640 1 -- 172.21.15.7:0/1416018018 wait complete. 2024-06-13T06:55:19.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:19 smithi007 ceph-mon[32077]: pgmap v487: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-06-13T06:55:19.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:19 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3846580600' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-13T06:55:19.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:19 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:19.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:19 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/445734562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:19.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:19 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2267118176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T06:55:19.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-13T06:55:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 -- 172.21.15.7:0/3900854831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49581018b0 msgr2=0x7f4958101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 --2- 172.21.15.7:0/3900854831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49581018b0 0x7f4958101cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4948009900 tx=0x7f494802a410 comp rx=0 tx=0).stop 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 -- 172.21.15.7:0/3900854831 shutdown_connections 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 --2- 172.21.15.7:0/3900854831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f4958109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 --2- 172.21.15.7:0/3900854831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4958102aa0 0x7f4958102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 --2- 172.21.15.7:0/3900854831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49581018b0 0x7f4958101cd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 -- 172.21.15.7:0/3900854831 >> 172.21.15.7:0/3900854831 conn(0x7f49580fce40 msgr2=0x7f49580ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 -- 172.21.15.7:0/3900854831 shutdown_connections 2024-06-13T06:55:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.870+0000 7f4957200640 1 -- 172.21.15.7:0/3900854831 wait complete. 2024-06-13T06:55:19.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.871+0000 7f4957200640 1 Processor -- start 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.871+0000 7f4957200640 1 -- start start 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49581018b0 0x7f4958197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f495819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f495810d570 con 0x7f4958102aa0 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f495810d3f0 con 0x7f4958103460 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4957200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f495810d6f0 con 0x7f49581018b0 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4955400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4955400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58512/0 (socket says 172.21.15.7:58512) 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4955400640 1 -- 172.21.15.7:0/315660610 learned_addr learned my addr 172.21.15.7:0/315660610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4955e00640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49581018b0 0x7f4958197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.872+0000 7f4956800640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f495819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 -- 172.21.15.7:0/315660610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49581018b0 msgr2=0x7f4958197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49581018b0 0x7f4958197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 -- 172.21.15.7:0/315660610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 msgr2=0x7f495819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f495819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 -- 172.21.15.7:0/315660610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819f800 con 0x7f4958102aa0 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4956800640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f495819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4955400640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f494400eb90 tx=0x7f494400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944004d20 con 0x7f4958102aa0 2024-06-13T06:55:19.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494400ee90 con 0x7f4958102aa0 2024-06-13T06:55:19.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819faf0 con 0x7f4958102aa0 2024-06-13T06:55:19.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944010830 con 0x7f4958102aa0 2024-06-13T06:55:19.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.873+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495819fff0 con 0x7f4958102aa0 2024-06-13T06:55:19.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.875+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4958108bd0 con 0x7f4958102aa0 2024-06-13T06:55:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.878+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4944005020 con 0x7f4958102aa0 2024-06-13T06:55:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.879+0000 7f494ea00640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.880+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49440cfa10 con 0x7f4958102aa0 2024-06-13T06:55:19.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.880+0000 7f4955e00640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:19.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.880+0000 7f4955e00640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 0x7f493c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f494802aa00 tx=0x7f49480315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:19.884+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f494409c5d0 con 0x7f4958102aa0 2024-06-13T06:55:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.091+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f49580625a0 con 0x7f4958102aa0 2024-06-13T06:55:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.092+0000 7f494ea00640 1 -- 172.21.15.7:0/315660610 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f494409bed0 con 0x7f4958102aa0 2024-06-13T06:55:20.091 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-06-13T06:55:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 msgr2=0x7f493c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 0x7f493c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f494802aa00 tx=0x7f49480315b0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 msgr2=0x7f4958198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f494400eb90 tx=0x7f494400c6a0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 shutdown_connections 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f493c0a2b90 0x7f493c0a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4958103460 0x7f495819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4958102aa0 0x7f4958198120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 --2- 172.21.15.7:0/315660610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49581018b0 0x7f4958197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.095+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 >> 172.21.15.7:0/315660610 conn(0x7f49580fce40 msgr2=0x7f4958105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.096+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 shutdown_connections 2024-06-13T06:55:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.096+0000 7f4957200640 1 -- 172.21.15.7:0/315660610 wait complete. 2024-06-13T06:55:20.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-13T06:55:20.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 -- 172.21.15.7:0/1848479742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 msgr2=0x7f390c073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 --2- 172.21.15.7:0/1848479742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c073e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009900 tx=0x7f38fc02a3f0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 -- 172.21.15.7:0/1848479742 shutdown_connections 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 --2- 172.21.15.7:0/1848479742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c109770 0x7f390c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 --2- 172.21.15.7:0/1848479742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c074400 0x7f390c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 --2- 172.21.15.7:0/1848479742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c073e30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.340+0000 7f390be00640 1 -- 172.21.15.7:0/1848479742 >> 172.21.15.7:0/1848479742 conn(0x7f390c0fce40 msgr2=0x7f390c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.341+0000 7f390be00640 1 -- 172.21.15.7:0/1848479742 shutdown_connections 2024-06-13T06:55:20.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.341+0000 7f390be00640 1 -- 172.21.15.7:0/1848479742 wait complete. 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 Processor -- start 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 -- start start 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c109770 0x7f390c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10f760 con 0x7f390c109770 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10f5e0 con 0x7f390c0759e0 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10f8e0 con 0x7f390c074400 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c109770 0x7f390c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49736/0 (socket says 172.21.15.7:49736) 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.342+0000 7f390aa00640 1 -- 172.21.15.7:0/3154328508 learned_addr learned my addr 172.21.15.7:0/3154328508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390a000640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 -- 172.21.15.7:0/3154328508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 msgr2=0x7f390c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 -- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c109770 msgr2=0x7f390c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c109770 0x7f390c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 -- 172.21.15.7:0/3154328508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1a1990 con 0x7f390c074400 2024-06-13T06:55:20.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390a000640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.343+0000 7f390aa00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38fc031440 tx=0x7f38fc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.344+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc03f030 con 0x7f390c074400 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.344+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1a1c20 con 0x7f390c074400 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.344+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc003ce0 con 0x7f390c074400 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.344+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc035070 con 0x7f390c074400 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.344+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a2160 con 0x7f390c074400 2024-06-13T06:55:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.345+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c1084a0 con 0x7f390c074400 2024-06-13T06:55:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.347+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f38fc042030 con 0x7f390c074400 2024-06-13T06:55:20.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.348+0000 7f3903400640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 0x7f38f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:20.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.348+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f38fc0ef8e0 con 0x7f390c074400 2024-06-13T06:55:20.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.349+0000 7f390a000640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 0x7f38f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:20.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.349+0000 7f390a000640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 0x7f38f00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f390c19b280 tx=0x7f38f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:20.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.351+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38fc0bc4a0 con 0x7f390c074400 2024-06-13T06:55:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:20 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1416018018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T06:55:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:20 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/315660610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T06:55:20.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.571+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f390c02d530 con 0x7f390c074400 2024-06-13T06:55:20.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.572+0000 7f3903400640 1 -- 172.21.15.7:0/3154328508 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f38fc0bbda0 con 0x7f390c074400 2024-06-13T06:55:20.571 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-13T06:55:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 msgr2=0x7f38f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 0x7f38f00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f390c19b280 tx=0x7f38f8005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 msgr2=0x7f390c199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38fc031440 tx=0x7f38fc032ce0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 shutdown_connections 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38f00a2b90 0x7f38f00a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c109770 0x7f390c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0759e0 0x7f390c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 --2- 172.21.15.7:0/3154328508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f390c074400 0x7f390c199d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.574+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 >> 172.21.15.7:0/3154328508 conn(0x7f390c0fce40 msgr2=0x7f390c105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.575+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 shutdown_connections 2024-06-13T06:55:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:20.575+0000 7f390be00640 1 -- 172.21.15.7:0/3154328508 wait complete. 2024-06-13T06:55:20.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:20 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1416018018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T06:55:20.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:20 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/315660610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T06:55:20.593 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-13T06:55:20.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:55:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:20 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1416018018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T06:55:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:20 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/315660610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T06:55:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:21.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:21 smithi007 ceph-mon[32077]: pgmap v488: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-13T06:55:21.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:21 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3154328508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T06:55:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:21 smithi189 ceph-mon[36616]: pgmap v488: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-13T06:55:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:21 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3154328508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T06:55:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:21 smithi175 ceph-mon[36886]: pgmap v488: 129 pgs: 129 active+clean; 644 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-13T06:55:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:21 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3154328508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T06:55:23.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:23 smithi007 ceph-mon[32077]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 -- 172.21.15.7:0/844860867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078102510 msgr2=0x7f3078102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 --2- 172.21.15.7:0/844860867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078102510 0x7f3078102990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3060009990 tx=0x7f3060009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 -- 172.21.15.7:0/844860867 shutdown_connections 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 --2- 172.21.15.7:0/844860867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102ed0 0x7f307810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 --2- 172.21.15.7:0/844860867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078108520 0x7f3078108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 --2- 172.21.15.7:0/844860867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078102510 0x7f3078102990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 -- 172.21.15.7:0/844860867 >> 172.21.15.7:0/844860867 conn(0x7f3078079560 msgr2=0x7f3078079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 -- 172.21.15.7:0/844860867 shutdown_connections 2024-06-13T06:55:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.539+0000 7f307e000700 1 -- 172.21.15.7:0/844860867 wait complete. 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.541+0000 7f307e000700 1 Processor -- start 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 -- start start 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078102ed0 0x7f307819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078108520 0x7f307819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30781127f0 con 0x7f3078102ed0 2024-06-13T06:55:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3078112670 con 0x7f3078102510 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.542+0000 7f307e000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3078112970 con 0x7f3078108520 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f306ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078102ed0 0x7f307819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078108520 0x7f307819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54744/0 (socket says 172.21.15.7:54744) 2024-06-13T06:55:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 -- 172.21.15.7:0/88123291 learned_addr learned my addr 172.21.15.7:0/88123291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 -- 172.21.15.7:0/88123291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078108520 msgr2=0x7f307819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078108520 0x7f307819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 -- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078102ed0 msgr2=0x7f307819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078102ed0 0x7f307819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.543+0000 7f3077400700 1 -- 172.21.15.7:0/88123291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307819fda0 con 0x7f3078102510 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f3077400700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f306800ef10 tx=0x7f306800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306800c9c0 con 0x7f3078102510 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3068019070 con 0x7f3078102510 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068007e60 con 0x7f3078102510 2024-06-13T06:55:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781a0030 con 0x7f3078102510 2024-06-13T06:55:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.544+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a7910 con 0x7f3078102510 2024-06-13T06:55:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.545+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3078069180 con 0x7f3078102510 2024-06-13T06:55:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.547+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3068012050 con 0x7f3078102510 2024-06-13T06:55:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.549+0000 7f3074c00700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 0x7f305c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.549+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f30680cf2f0 con 0x7f3078102510 2024-06-13T06:55:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.549+0000 7f306ea00700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 0x7f305c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.550+0000 7f306ea00700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 0x7f305c0a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3078079d90 tx=0x7f306000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.553+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3068010040 con 0x7f3078102510 2024-06-13T06:55:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:23 smithi189 ceph-mon[36616]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:55:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:23 smithi175 ceph-mon[36886]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-13T06:55:23.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.832+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f307819c0f0 con 0x7f3078102510 2024-06-13T06:55:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.833+0000 7f3074c00700 1 -- 172.21.15.7:0/88123291 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11868 (secure 0 0 0) 0x7f306809beb0 con 0x7f3078102510 2024-06-13T06:55:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:23.833 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:55:17.483515+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 msgr2=0x7f305c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 0x7f305c0a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3078079d90 tx=0x7f306000b260 comp rx=0 tx=0).stop 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 msgr2=0x7f307819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f306800ef10 tx=0x7f306800c3a0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 shutdown_connections 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3078108520 0x7f307819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f305c0a2ca0 0x7f305c0a5160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3078102510 0x7f307819ad90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 --2- 172.21.15.7:0/88123291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3078102ed0 0x7f307819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 >> 172.21.15.7:0/88123291 conn(0x7f3078079560 msgr2=0x7f3078074950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 shutdown_connections 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:23.837+0000 7f307e000700 1 -- 172.21.15.7:0/88123291 wait complete. 2024-06-13T06:55:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-13T06:55:24.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:24 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/88123291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:24.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:24 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/88123291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:24.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:24 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/88123291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:24.768 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-13T06:55:24.768 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-13T06:55:24.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.990+0000 7f0ebf200640 1 -- 172.21.15.7:0/4066295748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec01053b0 msgr2=0x7f0ec0105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.990+0000 7f0ebf200640 1 --2- 172.21.15.7:0/4066295748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec01053b0 0x7f0ec0105830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb802a8c0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 -- 172.21.15.7:0/4066295748 shutdown_connections 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 --2- 172.21.15.7:0/4066295748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec01053b0 0x7f0ec0105830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 --2- 172.21.15.7:0/4066295748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 0x7f0ec0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 --2- 172.21.15.7:0/4066295748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0100090 0x7f0ec01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 -- 172.21.15.7:0/4066295748 >> 172.21.15.7:0/4066295748 conn(0x7f0ec00fbc70 msgr2=0x7f0ec00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 -- 172.21.15.7:0/4066295748 shutdown_connections 2024-06-13T06:55:24.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.991+0000 7f0ebf200640 1 -- 172.21.15.7:0/4066295748 wait complete. 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 Processor -- start 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 -- start start 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 0x7f0ec019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 0x7f0ec01a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0113ae0 con 0x7f0ec0100090 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0113960 con 0x7f0ec0102a00 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebf200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0113c60 con 0x7f0ec01053b0 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39460/0 (socket says 172.21.15.7:39460) 2024-06-13T06:55:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.992+0000 7f0ebde00640 1 -- 172.21.15.7:0/3834143839 learned_addr learned my addr 172.21.15.7:0/3834143839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebd400640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 0x7f0ec019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebe800640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 0x7f0ec01a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 -- 172.21.15.7:0/3834143839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 msgr2=0x7f0ec01a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 0x7f0ec01a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 -- 172.21.15.7:0/3834143839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 msgr2=0x7f0ec019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 0x7f0ec019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 -- 172.21.15.7:0/3834143839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec01a5cb0 con 0x7f0ec0100090 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebe800640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 0x7f0ec01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0ebde00640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9f0 tx=0x7f0eb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000cd80 con 0x7f0ec0100090 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0005500 con 0x7f0ec0100090 2024-06-13T06:55:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.993+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0010690 con 0x7f0ec0100090 2024-06-13T06:55:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.994+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a5fa0 con 0x7f0ec0100090 2024-06-13T06:55:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.994+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a63b0 con 0x7f0ec0100090 2024-06-13T06:55:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.995+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec00fd760 con 0x7f0ec0100090 2024-06-13T06:55:24.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.997+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0eb00046c0 con 0x7f0ec0100090 2024-06-13T06:55:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.998+0000 7f0eb6a00640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 0x7f0ea00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.999+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0eb00cf9f0 con 0x7f0ec0100090 2024-06-13T06:55:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:24.999+0000 7f0ebd400640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 0x7f0ea00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.000+0000 7f0ebd400640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 0x7f0ea00a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ec019f5a0 tx=0x7f0ea8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:25.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.001+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0eb0053850 con 0x7f0ec0100090 2024-06-13T06:55:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.202+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ec0061de0 con 0x7f0ea00a2c10 2024-06-13T06:55:25.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.233+0000 7f0eb6a00640 1 -- 172.21.15.7:0/3834143839 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340792 (secure 0 0 0) 0x7f0ec0061de0 con 0x7f0ea00a2c10 2024-06-13T06:55:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 msgr2=0x7f0ea00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 0x7f0ea00a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ec019f5a0 tx=0x7f0ea8005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 msgr2=0x7f0ec019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000e9f0 tx=0x7f0eb000edf0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 shutdown_connections 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0ea00a2c10 0x7f0ea00a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec01053b0 0x7f0ec01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ec0102a00 0x7f0ec019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 --2- 172.21.15.7:0/3834143839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0100090 0x7f0ec019e090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.239+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 >> 172.21.15.7:0/3834143839 conn(0x7f0ec00fbc70 msgr2=0x7f0ec0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.241+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 shutdown_connections 2024-06-13T06:55:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.241+0000 7f0ebf200640 1 -- 172.21.15.7:0/3834143839 wait complete. 2024-06-13T06:55:25.261 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-06-13T06:55:23.752843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663976,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1451,"num_read_kb":850,"num_write":368,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378928,"kb_used_data":13136,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562896,"statfs":{"total":1151940427776,"available":1151552405504,"internally_reserved":0,"allocated":13451264,"data_stored":8615222,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":37,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006921"},"pg_stats":[{"pgid":"3.3f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632165+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:53:29.632165+0000","last_peered":"2024-06-13T06:53:29.632165+0000","last_clean":"2024-06-13T06:53:29.632165+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:53:29.632165+0000","last_undegraded":"2024-06-13T06:53:29.632165+0000","last_fullsized":"2024-06-13T06:53:29.632165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626282+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:53:29.626282+0000","last_peered":"2024-06-13T06:53:29.626282+0000","last_clean":"2024-06-13T06:53:29.626282+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:53:29.626282+0000","last_undegraded":"2024-06-13T06:53:29.626282+0000","last_fullsized":"2024-06-13T06:53:29.626282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626415+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:53:29.626415+0000","last_peered":"2024-06-13T06:53:29.626415+0000","last_clean":"2024-06-13T06:53:29.626415+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:53:29.626415+0000","last_undegraded":"2024-06-13T06:53:29.626415+0000","last_fullsized":"2024-06-13T06:53:29.626415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"98'8","reported_seq":21,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622619+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:53:29.622619+0000","last_peered":"2024-06-13T06:53:29.622619+0000","last_clean":"2024-06-13T06:53:29.622619+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:53:29.622619+0000","last_undegraded":"2024-06-13T06:53:29.622619+0000","last_fullsized":"2024-06-13T06:53:29.622619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633492+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:53:29.633492+0000","last_peered":"2024-06-13T06:53:29.633492+0000","last_clean":"2024-06-13T06:53:29.633492+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:53:29.633492+0000","last_undegraded":"2024-06-13T06:53:29.633492+0000","last_fullsized":"2024-06-13T06:53:29.633492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633554+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:53:29.633554+0000","last_peered":"2024-06-13T06:53:29.633554+0000","last_clean":"2024-06-13T06:53:29.633554+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:53:29.633554+0000","last_undegraded":"2024-06-13T06:53:29.633554+0000","last_fullsized":"2024-06-13T06:53:29.633554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628843+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:53:29.628843+0000","last_peered":"2024-06-13T06:53:29.628843+0000","last_clean":"2024-06-13T06:53:29.628843+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:53:29.628843+0000","last_undegraded":"2024-06-13T06:53:29.628843+0000","last_fullsized":"2024-06-13T06:53:29.628843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638046+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:53:29.638046+0000","last_peered":"2024-06-13T06:53:29.638046+0000","last_clean":"2024-06-13T06:53:29.638046+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:53:29.638046+0000","last_undegraded":"2024-06-13T06:53:29.638046+0000","last_fullsized":"2024-06-13T06:53:29.638046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631256+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:53:29.631256+0000","last_peered":"2024-06-13T06:53:29.631256+0000","last_clean":"2024-06-13T06:53:29.631256+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:53:29.631256+0000","last_undegraded":"2024-06-13T06:53:29.631256+0000","last_fullsized":"2024-06-13T06:53:29.631256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"98'16","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627241+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:53:29.627241+0000","last_peered":"2024-06-13T06:53:29.627241+0000","last_clean":"2024-06-13T06:53:29.627241+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:53:29.627241+0000","last_undegraded":"2024-06-13T06:53:29.627241+0000","last_fullsized":"2024-06-13T06:53:29.627241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632454+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:53:29.632454+0000","last_peered":"2024-06-13T06:53:29.632454+0000","last_clean":"2024-06-13T06:53:29.632454+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:53:29.632454+0000","last_undegraded":"2024-06-13T06:53:29.632454+0000","last_fullsized":"2024-06-13T06:53:29.632454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"98'7","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635014+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:53:29.635014+0000","last_peered":"2024-06-13T06:53:29.635014+0000","last_clean":"2024-06-13T06:53:29.635014+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:53:29.635014+0000","last_undegraded":"2024-06-13T06:53:29.635014+0000","last_fullsized":"2024-06-13T06:53:29.635014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631788+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:53:29.631788+0000","last_peered":"2024-06-13T06:53:29.631788+0000","last_clean":"2024-06-13T06:53:29.631788+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:53:29.631788+0000","last_undegraded":"2024-06-13T06:53:29.631788+0000","last_fullsized":"2024-06-13T06:53:29.631788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630960+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:53:29.630960+0000","last_peered":"2024-06-13T06:53:29.630960+0000","last_clean":"2024-06-13T06:53:29.630960+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:53:29.630960+0000","last_undegraded":"2024-06-13T06:53:29.630960+0000","last_fullsized":"2024-06-13T06:53:29.630960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622955+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:53:29.622955+0000","last_peered":"2024-06-13T06:53:29.622955+0000","last_clean":"2024-06-13T06:53:29.622955+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:53:29.622955+0000","last_undegraded":"2024-06-13T06:53:29.622955+0000","last_fullsized":"2024-06-13T06:53:29.622955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632867+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:53:29.632867+0000","last_peered":"2024-06-13T06:53:29.632867+0000","last_clean":"2024-06-13T06:53:29.632867+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:53:29.632867+0000","last_undegraded":"2024-06-13T06:53:29.632867+0000","last_fullsized":"2024-06-13T06:53:29.632867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632172+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:53:29.632172+0000","last_peered":"2024-06-13T06:53:29.632172+0000","last_clean":"2024-06-13T06:53:29.632172+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:53:29.632172+0000","last_undegraded":"2024-06-13T06:53:29.632172+0000","last_fullsized":"2024-06-13T06:53:29.632172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634623+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:53:29.634623+0000","last_peered":"2024-06-13T06:53:29.634623+0000","last_clean":"2024-06-13T06:53:29.634623+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:53:29.634623+0000","last_undegraded":"2024-06-13T06:53:29.634623+0000","last_fullsized":"2024-06-13T06:53:29.634623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630073+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:53:29.630073+0000","last_peered":"2024-06-13T06:53:29.630073+0000","last_clean":"2024-06-13T06:53:29.630073+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:53:29.630073+0000","last_undegraded":"2024-06-13T06:53:29.630073+0000","last_fullsized":"2024-06-13T06:53:29.630073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.768380+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:52:57.768380+0000","last_peered":"2024-06-13T06:52:57.768380+0000","last_clean":"2024-06-13T06:52:57.768380+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:52:57.768380+0000","last_undegraded":"2024-06-13T06:52:57.768380+0000","last_fullsized":"2024-06-13T06:52:57.768380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464726+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:52:18.464726+0000","last_peered":"2024-06-13T06:52:18.464726+0000","last_clean":"2024-06-13T06:52:18.464726+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:52:18.464726+0000","last_undegraded":"2024-06-13T06:52:18.464726+0000","last_fullsized":"2024-06-13T06:52:18.464726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635549+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:53:29.635549+0000","last_peered":"2024-06-13T06:53:29.635549+0000","last_clean":"2024-06-13T06:53:29.635549+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:53:29.635549+0000","last_undegraded":"2024-06-13T06:53:29.635549+0000","last_fullsized":"2024-06-13T06:53:29.635549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.087653+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:52:57.087653+0000","last_peered":"2024-06-13T06:52:57.087653+0000","last_clean":"2024-06-13T06:52:57.087653+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:52:57.087653+0000","last_undegraded":"2024-06-13T06:52:57.087653+0000","last_fullsized":"2024-06-13T06:52:57.087653+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:15.257089+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:52:15.257089+0000","last_peered":"2024-06-13T06:52:15.257089+0000","last_clean":"2024-06-13T06:52:15.257089+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:52:15.257089+0000","last_undegraded":"2024-06-13T06:52:15.257089+0000","last_fullsized":"2024-06-13T06:52:15.257089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635752+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:53:29.635752+0000","last_peered":"2024-06-13T06:53:29.635752+0000","last_clean":"2024-06-13T06:53:29.635752+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:53:29.635752+0000","last_undegraded":"2024-06-13T06:53:29.635752+0000","last_fullsized":"2024-06-13T06:53:29.635752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891648+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891648+0000","last_peered":"2024-06-13T06:51:00.891648+0000","last_clean":"2024-06-13T06:51:00.891648+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891648+0000","last_undegraded":"2024-06-13T06:51:00.891648+0000","last_fullsized":"2024-06-13T06:51:00.891648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957691+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:52:42.957691+0000","last_peered":"2024-06-13T06:52:42.957691+0000","last_clean":"2024-06-13T06:52:42.957691+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:52:42.957691+0000","last_undegraded":"2024-06-13T06:52:42.957691+0000","last_fullsized":"2024-06-13T06:52:42.957691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634711+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:53:29.634711+0000","last_peered":"2024-06-13T06:53:29.634711+0000","last_clean":"2024-06-13T06:53:29.634711+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:53:29.634711+0000","last_undegraded":"2024-06-13T06:53:29.634711+0000","last_fullsized":"2024-06-13T06:53:29.634711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891222+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891222+0000","last_peered":"2024-06-13T06:51:00.891222+0000","last_clean":"2024-06-13T06:51:00.891222+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891222+0000","last_undegraded":"2024-06-13T06:51:00.891222+0000","last_fullsized":"2024-06-13T06:51:00.891222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957657+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:52:42.957657+0000","last_peered":"2024-06-13T06:52:42.957657+0000","last_clean":"2024-06-13T06:52:42.957657+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:52:42.957657+0000","last_undegraded":"2024-06-13T06:52:42.957657+0000","last_fullsized":"2024-06-13T06:52:42.957657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632012+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:53:29.632012+0000","last_peered":"2024-06-13T06:53:29.632012+0000","last_clean":"2024-06-13T06:53:29.632012+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:53:29.632012+0000","last_undegraded":"2024-06-13T06:53:29.632012+0000","last_fullsized":"2024-06-13T06:53:29.632012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881987+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881987+0000","last_peered":"2024-06-13T06:51:00.881987+0000","last_clean":"2024-06-13T06:51:00.881987+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881987+0000","last_undegraded":"2024-06-13T06:51:00.881987+0000","last_fullsized":"2024-06-13T06:51:00.881987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:19.156377+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:55:19.156377+0000","last_peered":"2024-06-13T06:55:19.156377+0000","last_clean":"2024-06-13T06:55:19.156377+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:55:19.156377+0000","last_undegraded":"2024-06-13T06:55:19.156377+0000","last_fullsized":"2024-06-13T06:55:19.156377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617187+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:53:29.617187+0000","last_peered":"2024-06-13T06:53:29.617187+0000","last_clean":"2024-06-13T06:53:29.617187+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:53:29.617187+0000","last_undegraded":"2024-06-13T06:53:29.617187+0000","last_fullsized":"2024-06-13T06:53:29.617187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882139+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882139+0000","last_peered":"2024-06-13T06:51:00.882139+0000","last_clean":"2024-06-13T06:51:00.882139+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882139+0000","last_undegraded":"2024-06-13T06:51:00.882139+0000","last_fullsized":"2024-06-13T06:51:00.882139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:45.352623+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:52:45.352623+0000","last_peered":"2024-06-13T06:52:45.352623+0000","last_clean":"2024-06-13T06:52:45.352623+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:52:45.352623+0000","last_undegraded":"2024-06-13T06:52:45.352623+0000","last_fullsized":"2024-06-13T06:52:45.352623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640132+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:53:29.640132+0000","last_peered":"2024-06-13T06:53:29.640132+0000","last_clean":"2024-06-13T06:53:29.640132+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:53:29.640132+0000","last_undegraded":"2024-06-13T06:53:29.640132+0000","last_fullsized":"2024-06-13T06:53:29.640132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108566+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:52:36.108566+0000","last_peered":"2024-06-13T06:52:36.108566+0000","last_clean":"2024-06-13T06:52:36.108566+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:52:36.108566+0000","last_undegraded":"2024-06-13T06:52:36.108566+0000","last_fullsized":"2024-06-13T06:52:36.108566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606565+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:52:36.606565+0000","last_peered":"2024-06-13T06:52:36.606565+0000","last_clean":"2024-06-13T06:52:36.606565+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:52:36.606565+0000","last_undegraded":"2024-06-13T06:52:36.606565+0000","last_fullsized":"2024-06-13T06:52:36.606565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637458+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:53:29.637458+0000","last_peered":"2024-06-13T06:53:29.637458+0000","last_clean":"2024-06-13T06:53:29.637458+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:53:29.637458+0000","last_undegraded":"2024-06-13T06:53:29.637458+0000","last_fullsized":"2024-06-13T06:53:29.637458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:29.820545+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:52:29.820545+0000","last_peered":"2024-06-13T06:52:29.820545+0000","last_clean":"2024-06-13T06:52:29.820545+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:52:29.820545+0000","last_undegraded":"2024-06-13T06:52:29.820545+0000","last_fullsized":"2024-06-13T06:52:29.820545+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250016+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:52:40.250016+0000","last_peered":"2024-06-13T06:52:40.250016+0000","last_clean":"2024-06-13T06:52:40.250016+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:52:40.250016+0000","last_undegraded":"2024-06-13T06:52:40.250016+0000","last_fullsized":"2024-06-13T06:52:40.250016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639877+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:53:29.639877+0000","last_peered":"2024-06-13T06:53:29.639877+0000","last_clean":"2024-06-13T06:53:29.639877+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:53:29.639877+0000","last_undegraded":"2024-06-13T06:53:29.639877+0000","last_fullsized":"2024-06-13T06:53:29.639877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:50.263717+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:52:50.263717+0000","last_peered":"2024-06-13T06:52:50.263717+0000","last_clean":"2024-06-13T06:52:50.263717+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:52:50.263717+0000","last_undegraded":"2024-06-13T06:52:50.263717+0000","last_fullsized":"2024-06-13T06:52:50.263717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:19.155728+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:55:19.155728+0000","last_peered":"2024-06-13T06:55:19.155728+0000","last_clean":"2024-06-13T06:55:19.155728+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:55:19.155728+0000","last_undegraded":"2024-06-13T06:55:19.155728+0000","last_fullsized":"2024-06-13T06:55:19.155728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634544+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:53:29.634544+0000","last_peered":"2024-06-13T06:53:29.634544+0000","last_clean":"2024-06-13T06:53:29.634544+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:53:29.634544+0000","last_undegraded":"2024-06-13T06:53:29.634544+0000","last_fullsized":"2024-06-13T06:53:29.634544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470112+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:52:53.470112+0000","last_peered":"2024-06-13T06:52:53.470112+0000","last_clean":"2024-06-13T06:52:53.470112+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:52:53.470112+0000","last_undegraded":"2024-06-13T06:52:53.470112+0000","last_fullsized":"2024-06-13T06:52:53.470112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033534+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:52:36.033534+0000","last_peered":"2024-06-13T06:52:36.033534+0000","last_clean":"2024-06-13T06:52:36.033534+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:52:36.033534+0000","last_undegraded":"2024-06-13T06:52:36.033534+0000","last_fullsized":"2024-06-13T06:52:36.033534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630475+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:53:29.630475+0000","last_peered":"2024-06-13T06:53:29.630475+0000","last_clean":"2024-06-13T06:53:29.630475+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:53:29.630475+0000","last_undegraded":"2024-06-13T06:53:29.630475+0000","last_fullsized":"2024-06-13T06:53:29.630475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:52.086907+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:52:52.086907+0000","last_peered":"2024-06-13T06:52:52.086907+0000","last_clean":"2024-06-13T06:52:52.086907+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:52:52.086907+0000","last_undegraded":"2024-06-13T06:52:52.086907+0000","last_fullsized":"2024-06-13T06:52:52.086907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250061+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:52:40.250061+0000","last_peered":"2024-06-13T06:52:40.250061+0000","last_clean":"2024-06-13T06:52:40.250061+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:52:40.250061+0000","last_undegraded":"2024-06-13T06:52:40.250061+0000","last_fullsized":"2024-06-13T06:52:40.250061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633279+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:53:29.633279+0000","last_peered":"2024-06-13T06:53:29.633279+0000","last_clean":"2024-06-13T06:53:29.633279+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:53:29.633279+0000","last_undegraded":"2024-06-13T06:53:29.633279+0000","last_fullsized":"2024-06-13T06:53:29.633279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470157+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:52:53.470157+0000","last_peered":"2024-06-13T06:52:53.470157+0000","last_clean":"2024-06-13T06:52:53.470157+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:52:53.470157+0000","last_undegraded":"2024-06-13T06:52:53.470157+0000","last_fullsized":"2024-06-13T06:52:53.470157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'40","reported_seq":524,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.167306+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:55:20.167306+0000","last_peered":"2024-06-13T06:55:20.167306+0000","last_clean":"2024-06-13T06:55:20.167306+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:55:20.167306+0000","last_undegraded":"2024-06-13T06:55:20.167306+0000","last_fullsized":"2024-06-13T06:55:20.167306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":442,"num_read_kb":289,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.612063+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:53:29.612063+0000","last_peered":"2024-06-13T06:53:29.612063+0000","last_clean":"2024-06-13T06:53:29.612063+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:53:29.612063+0000","last_undegraded":"2024-06-13T06:53:29.612063+0000","last_fullsized":"2024-06-13T06:53:29.612063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:55.251812+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:52:55.251812+0000","last_peered":"2024-06-13T06:52:55.251812+0000","last_clean":"2024-06-13T06:52:55.251812+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:52:55.251812+0000","last_undegraded":"2024-06-13T06:52:55.251812+0000","last_fullsized":"2024-06-13T06:52:55.251812+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.765822+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:52:32.765822+0000","last_peered":"2024-06-13T06:52:32.765822+0000","last_clean":"2024-06-13T06:52:32.765822+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:52:32.765822+0000","last_undegraded":"2024-06-13T06:52:32.765822+0000","last_fullsized":"2024-06-13T06:52:32.765822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632382+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:53:29.632382+0000","last_peered":"2024-06-13T06:53:29.632382+0000","last_clean":"2024-06-13T06:53:29.632382+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:53:29.632382+0000","last_undegraded":"2024-06-13T06:53:29.632382+0000","last_fullsized":"2024-06-13T06:53:29.632382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:51.110319+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:52:51.110319+0000","last_peered":"2024-06-13T06:52:51.110319+0000","last_clean":"2024-06-13T06:52:51.110319+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:52:51.110319+0000","last_undegraded":"2024-06-13T06:52:51.110319+0000","last_fullsized":"2024-06-13T06:52:51.110319+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:35.249437+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:52:35.249437+0000","last_peered":"2024-06-13T06:52:35.249437+0000","last_clean":"2024-06-13T06:52:35.249437+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:52:35.249437+0000","last_undegraded":"2024-06-13T06:52:35.249437+0000","last_fullsized":"2024-06-13T06:52:35.249437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633901+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:53:29.633901+0000","last_peered":"2024-06-13T06:53:29.633901+0000","last_clean":"2024-06-13T06:53:29.633901+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:53:29.633901+0000","last_undegraded":"2024-06-13T06:53:29.633901+0000","last_fullsized":"2024-06-13T06:53:29.633901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:58.471021+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:52:58.471021+0000","last_peered":"2024-06-13T06:52:58.471021+0000","last_clean":"2024-06-13T06:52:58.471021+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:52:58.471021+0000","last_undegraded":"2024-06-13T06:52:58.471021+0000","last_fullsized":"2024-06-13T06:52:58.471021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.326188+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:55:20.326188+0000","last_peered":"2024-06-13T06:55:20.326188+0000","last_clean":"2024-06-13T06:55:20.326188+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:55:20.326188+0000","last_undegraded":"2024-06-13T06:55:20.326188+0000","last_fullsized":"2024-06-13T06:55:20.326188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1425,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633677+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:53:29.633677+0000","last_peered":"2024-06-13T06:53:29.633677+0000","last_clean":"2024-06-13T06:53:29.633677+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:53:29.633677+0000","last_undegraded":"2024-06-13T06:53:29.633677+0000","last_fullsized":"2024-06-13T06:53:29.633677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:56.609813+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:52:56.609813+0000","last_peered":"2024-06-13T06:52:56.609813+0000","last_clean":"2024-06-13T06:52:56.609813+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:52:56.609813+0000","last_undegraded":"2024-06-13T06:52:56.609813+0000","last_fullsized":"2024-06-13T06:52:56.609813+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.956450+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:52:32.956450+0000","last_peered":"2024-06-13T06:52:32.956450+0000","last_clean":"2024-06-13T06:52:32.956450+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:52:32.956450+0000","last_undegraded":"2024-06-13T06:52:32.956450+0000","last_fullsized":"2024-06-13T06:52:32.956450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631663+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:53:29.631663+0000","last_peered":"2024-06-13T06:53:29.631663+0000","last_clean":"2024-06-13T06:53:29.631663+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:53:29.631663+0000","last_undegraded":"2024-06-13T06:53:29.631663+0000","last_fullsized":"2024-06-13T06:53:29.631663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250034+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:52:40.250034+0000","last_peered":"2024-06-13T06:52:40.250034+0000","last_clean":"2024-06-13T06:52:40.250034+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:52:40.250034+0000","last_undegraded":"2024-06-13T06:52:40.250034+0000","last_fullsized":"2024-06-13T06:52:40.250034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:42.970288+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:54:42.970288+0000","last_peered":"2024-06-13T06:54:42.970288+0000","last_clean":"2024-06-13T06:54:42.970288+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:54:42.970288+0000","last_undegraded":"2024-06-13T06:54:42.970288+0000","last_fullsized":"2024-06-13T06:54:42.970288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627895+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:53:29.627895+0000","last_peered":"2024-06-13T06:53:29.627895+0000","last_clean":"2024-06-13T06:53:29.627895+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:53:29.627895+0000","last_undegraded":"2024-06-13T06:53:29.627895+0000","last_fullsized":"2024-06-13T06:53:29.627895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.957155+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:52:37.957155+0000","last_peered":"2024-06-13T06:52:37.957155+0000","last_clean":"2024-06-13T06:52:37.957155+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:52:37.957155+0000","last_undegraded":"2024-06-13T06:52:37.957155+0000","last_fullsized":"2024-06-13T06:52:37.957155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464620+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:52:18.464620+0000","last_peered":"2024-06-13T06:52:18.464620+0000","last_clean":"2024-06-13T06:52:18.464620+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:52:18.464620+0000","last_undegraded":"2024-06-13T06:52:18.464620+0000","last_fullsized":"2024-06-13T06:52:18.464620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636785+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:53:29.636785+0000","last_peered":"2024-06-13T06:53:29.636785+0000","last_clean":"2024-06-13T06:53:29.636785+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:53:29.636785+0000","last_undegraded":"2024-06-13T06:53:29.636785+0000","last_fullsized":"2024-06-13T06:53:29.636785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606649+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:52:36.606649+0000","last_peered":"2024-06-13T06:52:36.606649+0000","last_clean":"2024-06-13T06:52:36.606649+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:52:36.606649+0000","last_undegraded":"2024-06-13T06:52:36.606649+0000","last_fullsized":"2024-06-13T06:52:36.606649+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:16.031317+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:52:16.031317+0000","last_peered":"2024-06-13T06:52:16.031317+0000","last_clean":"2024-06-13T06:52:16.031317+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:52:16.031317+0000","last_undegraded":"2024-06-13T06:52:16.031317+0000","last_fullsized":"2024-06-13T06:52:16.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633623+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:53:29.633623+0000","last_peered":"2024-06-13T06:53:29.633623+0000","last_clean":"2024-06-13T06:53:29.633623+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:53:29.633623+0000","last_undegraded":"2024-06-13T06:53:29.633623+0000","last_fullsized":"2024-06-13T06:53:29.633623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:41.607532+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:52:41.607532+0000","last_peered":"2024-06-13T06:52:41.607532+0000","last_clean":"2024-06-13T06:52:41.607532+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:52:41.607532+0000","last_undegraded":"2024-06-13T06:52:41.607532+0000","last_fullsized":"2024-06-13T06:52:41.607532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623878+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:53:29.623878+0000","last_peered":"2024-06-13T06:53:29.623878+0000","last_clean":"2024-06-13T06:53:29.623878+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:53:29.623878+0000","last_undegraded":"2024-06-13T06:53:29.623878+0000","last_fullsized":"2024-06-13T06:53:29.623878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'6","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.324967+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:55:20.324967+0000","last_peered":"2024-06-13T06:55:20.324967+0000","last_clean":"2024-06-13T06:55:20.324967+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:55:20.324967+0000","last_undegraded":"2024-06-13T06:55:20.324967+0000","last_fullsized":"2024-06-13T06:55:20.324967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033518+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:52:36.033518+0000","last_peered":"2024-06-13T06:52:36.033518+0000","last_clean":"2024-06-13T06:52:36.033518+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:52:36.033518+0000","last_undegraded":"2024-06-13T06:52:36.033518+0000","last_fullsized":"2024-06-13T06:52:36.033518+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636741+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:53:29.636741+0000","last_peered":"2024-06-13T06:53:29.636741+0000","last_clean":"2024-06-13T06:53:29.636741+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:53:29.636741+0000","last_undegraded":"2024-06-13T06:53:29.636741+0000","last_fullsized":"2024-06-13T06:53:29.636741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.082967+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:52:17.082967+0000","last_peered":"2024-06-13T06:52:17.082967+0000","last_clean":"2024-06-13T06:52:17.082967+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:52:17.082967+0000","last_undegraded":"2024-06-13T06:52:17.082967+0000","last_fullsized":"2024-06-13T06:52:17.082967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.037470+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:53:11.037470+0000","last_peered":"2024-06-13T06:53:11.037470+0000","last_clean":"2024-06-13T06:53:11.037470+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:53:11.037470+0000","last_undegraded":"2024-06-13T06:53:11.037470+0000","last_fullsized":"2024-06-13T06:53:11.037470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250077+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:52:40.250077+0000","last_peered":"2024-06-13T06:52:40.250077+0000","last_clean":"2024-06-13T06:52:40.250077+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:52:40.250077+0000","last_undegraded":"2024-06-13T06:52:40.250077+0000","last_fullsized":"2024-06-13T06:52:40.250077+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631450+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:53:29.631450+0000","last_peered":"2024-06-13T06:53:29.631450+0000","last_clean":"2024-06-13T06:53:29.631450+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:53:29.631450+0000","last_undegraded":"2024-06-13T06:53:29.631450+0000","last_fullsized":"2024-06-13T06:53:29.631450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:11.030897+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:52:11.030897+0000","last_peered":"2024-06-13T06:52:11.030897+0000","last_clean":"2024-06-13T06:52:11.030897+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:52:11.030897+0000","last_undegraded":"2024-06-13T06:52:11.030897+0000","last_fullsized":"2024-06-13T06:52:11.030897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108536+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:52:36.108536+0000","last_peered":"2024-06-13T06:52:36.108536+0000","last_clean":"2024-06-13T06:52:36.108536+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:52:36.108536+0000","last_undegraded":"2024-06-13T06:52:36.108536+0000","last_fullsized":"2024-06-13T06:52:36.108536+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.955053+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:52:17.955053+0000","last_peered":"2024-06-13T06:52:17.955053+0000","last_clean":"2024-06-13T06:52:17.955053+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:52:17.955053+0000","last_undegraded":"2024-06-13T06:52:17.955053+0000","last_fullsized":"2024-06-13T06:52:17.955053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628081+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:53:29.628081+0000","last_peered":"2024-06-13T06:53:29.628081+0000","last_clean":"2024-06-13T06:53:29.628081+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:53:29.628081+0000","last_undegraded":"2024-06-13T06:53:29.628081+0000","last_fullsized":"2024-06-13T06:53:29.628081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.259402+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:52:30.259402+0000","last_peered":"2024-06-13T06:52:30.259402+0000","last_clean":"2024-06-13T06:52:30.259402+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:52:30.259402+0000","last_undegraded":"2024-06-13T06:52:30.259402+0000","last_fullsized":"2024-06-13T06:52:30.259402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'13","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:33.486327+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:54:33.486327+0000","last_peered":"2024-06-13T06:54:33.486327+0000","last_clean":"2024-06-13T06:54:33.486327+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:54:33.486327+0000","last_undegraded":"2024-06-13T06:54:33.486327+0000","last_fullsized":"2024-06-13T06:54:33.486327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633370+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:53:29.633370+0000","last_peered":"2024-06-13T06:53:29.633370+0000","last_clean":"2024-06-13T06:53:29.633370+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:53:29.633370+0000","last_undegraded":"2024-06-13T06:53:29.633370+0000","last_fullsized":"2024-06-13T06:53:29.633370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.766412+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:52:37.766412+0000","last_peered":"2024-06-13T06:52:37.766412+0000","last_clean":"2024-06-13T06:52:37.766412+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:52:37.766412+0000","last_undegraded":"2024-06-13T06:52:37.766412+0000","last_fullsized":"2024-06-13T06:52:37.766412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:22.083460+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:52:22.083460+0000","last_peered":"2024-06-13T06:52:22.083460+0000","last_clean":"2024-06-13T06:52:22.083460+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:52:22.083460+0000","last_undegraded":"2024-06-13T06:52:22.083460+0000","last_fullsized":"2024-06-13T06:52:22.083460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635347+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:53:29.635347+0000","last_peered":"2024-06-13T06:53:29.635347+0000","last_clean":"2024-06-13T06:53:29.635347+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:53:29.635347+0000","last_undegraded":"2024-06-13T06:53:29.635347+0000","last_fullsized":"2024-06-13T06:53:29.635347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086435+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:52:47.086435+0000","last_peered":"2024-06-13T06:52:47.086435+0000","last_clean":"2024-06-13T06:52:47.086435+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:52:47.086435+0000","last_undegraded":"2024-06-13T06:52:47.086435+0000","last_fullsized":"2024-06-13T06:52:47.086435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:24.165534+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:52:24.165534+0000","last_peered":"2024-06-13T06:52:24.165534+0000","last_clean":"2024-06-13T06:52:24.165534+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:52:24.165534+0000","last_undegraded":"2024-06-13T06:52:24.165534+0000","last_fullsized":"2024-06-13T06:52:24.165534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.620463+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:53:29.620463+0000","last_peered":"2024-06-13T06:53:29.620463+0000","last_clean":"2024-06-13T06:53:29.620463+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:53:29.620463+0000","last_undegraded":"2024-06-13T06:53:29.620463+0000","last_fullsized":"2024-06-13T06:53:29.620463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:48.469183+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:52:48.469183+0000","last_peered":"2024-06-13T06:52:48.469183+0000","last_clean":"2024-06-13T06:52:48.469183+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:52:48.469183+0000","last_undegraded":"2024-06-13T06:52:48.469183+0000","last_fullsized":"2024-06-13T06:52:48.469183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:25.258413+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:52:25.258413+0000","last_peered":"2024-06-13T06:52:25.258413+0000","last_clean":"2024-06-13T06:52:25.258413+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:52:25.258413+0000","last_undegraded":"2024-06-13T06:52:25.258413+0000","last_fullsized":"2024-06-13T06:52:25.258413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.625157+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:53:29.625157+0000","last_peered":"2024-06-13T06:53:29.625157+0000","last_clean":"2024-06-13T06:53:29.625157+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:53:29.625157+0000","last_undegraded":"2024-06-13T06:53:29.625157+0000","last_fullsized":"2024-06-13T06:53:29.625157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":27,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086403+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:52:47.086403+0000","last_peered":"2024-06-13T06:52:47.086403+0000","last_clean":"2024-06-13T06:52:47.086403+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:52:47.086403+0000","last_undegraded":"2024-06-13T06:52:47.086403+0000","last_fullsized":"2024-06-13T06:52:47.086403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.112337+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:53:11.112337+0000","last_peered":"2024-06-13T06:53:11.112337+0000","last_clean":"2024-06-13T06:53:11.112337+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:53:11.112337+0000","last_undegraded":"2024-06-13T06:53:11.112337+0000","last_fullsized":"2024-06-13T06:53:11.112337+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639239+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:53:29.639239+0000","last_peered":"2024-06-13T06:53:29.639239+0000","last_clean":"2024-06-13T06:53:29.639239+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:53:29.639239+0000","last_undegraded":"2024-06-13T06:53:29.639239+0000","last_fullsized":"2024-06-13T06:53:29.639239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086424+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:52:47.086424+0000","last_peered":"2024-06-13T06:52:47.086424+0000","last_clean":"2024-06-13T06:52:47.086424+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:52:47.086424+0000","last_undegraded":"2024-06-13T06:52:47.086424+0000","last_fullsized":"2024-06-13T06:52:47.086424+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.955951+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:52:27.955951+0000","last_peered":"2024-06-13T06:52:27.955951+0000","last_clean":"2024-06-13T06:52:27.955951+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:52:27.955951+0000","last_undegraded":"2024-06-13T06:52:27.955951+0000","last_fullsized":"2024-06-13T06:52:27.955951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631970+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:53:29.631970+0000","last_peered":"2024-06-13T06:53:29.631970+0000","last_clean":"2024-06-13T06:53:29.631970+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:53:29.631970+0000","last_undegraded":"2024-06-13T06:53:29.631970+0000","last_fullsized":"2024-06-13T06:53:29.631970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:39.167112+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:52:39.167112+0000","last_peered":"2024-06-13T06:52:39.167112+0000","last_clean":"2024-06-13T06:52:39.167112+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:52:39.167112+0000","last_undegraded":"2024-06-13T06:52:39.167112+0000","last_fullsized":"2024-06-13T06:52:39.167112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.765245+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:52:27.765245+0000","last_peered":"2024-06-13T06:52:27.765245+0000","last_clean":"2024-06-13T06:52:27.765245+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:52:27.765245+0000","last_undegraded":"2024-06-13T06:52:27.765245+0000","last_fullsized":"2024-06-13T06:52:27.765245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632700+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:53:29.632700+0000","last_peered":"2024-06-13T06:53:29.632700+0000","last_clean":"2024-06-13T06:53:29.632700+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:53:29.632700+0000","last_undegraded":"2024-06-13T06:53:29.632700+0000","last_fullsized":"2024-06-13T06:53:29.632700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:44.822664+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:52:44.822664+0000","last_peered":"2024-06-13T06:52:44.822664+0000","last_clean":"2024-06-13T06:52:44.822664+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:52:44.822664+0000","last_undegraded":"2024-06-13T06:52:44.822664+0000","last_fullsized":"2024-06-13T06:52:44.822664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.248910+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:52:30.248910+0000","last_peered":"2024-06-13T06:52:30.248910+0000","last_clean":"2024-06-13T06:52:30.248910+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:52:30.248910+0000","last_undegraded":"2024-06-13T06:52:30.248910+0000","last_fullsized":"2024-06-13T06:52:30.248910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629774+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:53:29.629774+0000","last_peered":"2024-06-13T06:53:29.629774+0000","last_clean":"2024-06-13T06:53:29.629774+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:53:29.629774+0000","last_undegraded":"2024-06-13T06:53:29.629774+0000","last_fullsized":"2024-06-13T06:53:29.629774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.249974+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:52:40.249974+0000","last_peered":"2024-06-13T06:52:40.249974+0000","last_clean":"2024-06-13T06:52:40.249974+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:52:40.249974+0000","last_undegraded":"2024-06-13T06:52:40.249974+0000","last_fullsized":"2024-06-13T06:52:40.249974+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:26.605224+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:52:26.605224+0000","last_peered":"2024-06-13T06:52:26.605224+0000","last_clean":"2024-06-13T06:52:26.605224+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:52:26.605224+0000","last_undegraded":"2024-06-13T06:52:26.605224+0000","last_fullsized":"2024-06-13T06:52:26.605224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.621192+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:53:29.621192+0000","last_peered":"2024-06-13T06:53:29.621192+0000","last_clean":"2024-06-13T06:53:29.621192+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:53:29.621192+0000","last_undegraded":"2024-06-13T06:53:29.621192+0000","last_fullsized":"2024-06-13T06:53:29.621192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623559+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:53:29.623559+0000","last_peered":"2024-06-13T06:53:29.623559+0000","last_clean":"2024-06-13T06:53:29.623559+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:53:29.623559+0000","last_undegraded":"2024-06-13T06:53:29.623559+0000","last_fullsized":"2024-06-13T06:53:29.623559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"98'12","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.624643+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:53:29.624643+0000","last_peered":"2024-06-13T06:53:29.624643+0000","last_clean":"2024-06-13T06:53:29.624643+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:53:29.624643+0000","last_undegraded":"2024-06-13T06:53:29.624643+0000","last_fullsized":"2024-06-13T06:53:29.624643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629702+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:53:29.629702+0000","last_peered":"2024-06-13T06:53:29.629702+0000","last_clean":"2024-06-13T06:53:29.629702+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:53:29.629702+0000","last_undegraded":"2024-06-13T06:53:29.629702+0000","last_fullsized":"2024-06-13T06:53:29.629702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638325+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:53:29.638325+0000","last_peered":"2024-06-13T06:53:29.638325+0000","last_clean":"2024-06-13T06:53:29.638325+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:53:29.638325+0000","last_undegraded":"2024-06-13T06:53:29.638325+0000","last_fullsized":"2024-06-13T06:53:29.638325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631155+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:53:29.631155+0000","last_peered":"2024-06-13T06:53:29.631155+0000","last_clean":"2024-06-13T06:53:29.631155+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:53:29.631155+0000","last_undegraded":"2024-06-13T06:53:29.631155+0000","last_fullsized":"2024-06-13T06:53:29.631155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617420+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:53:29.617420+0000","last_peered":"2024-06-13T06:53:29.617420+0000","last_clean":"2024-06-13T06:53:29.617420+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:53:29.617420+0000","last_undegraded":"2024-06-13T06:53:29.617420+0000","last_fullsized":"2024-06-13T06:53:29.617420+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630989+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:53:29.630989+0000","last_peered":"2024-06-13T06:53:29.630989+0000","last_clean":"2024-06-13T06:53:29.630989+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:53:29.630989+0000","last_undegraded":"2024-06-13T06:53:29.630989+0000","last_fullsized":"2024-06-13T06:53:29.630989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629961+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:53:29.629961+0000","last_peered":"2024-06-13T06:53:29.629961+0000","last_clean":"2024-06-13T06:53:29.629961+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:53:29.629961+0000","last_undegraded":"2024-06-13T06:53:29.629961+0000","last_fullsized":"2024-06-13T06:53:29.629961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640037+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:53:29.640037+0000","last_peered":"2024-06-13T06:53:29.640037+0000","last_clean":"2024-06-13T06:53:29.640037+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:53:29.640037+0000","last_undegraded":"2024-06-13T06:53:29.640037+0000","last_fullsized":"2024-06-13T06:53:29.640037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632710+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:53:29.632710+0000","last_peered":"2024-06-13T06:53:29.632710+0000","last_clean":"2024-06-13T06:53:29.632710+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:53:29.632710+0000","last_undegraded":"2024-06-13T06:53:29.632710+0000","last_fullsized":"2024-06-13T06:53:29.632710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640563+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:53:29.640563+0000","last_peered":"2024-06-13T06:53:29.640563+0000","last_clean":"2024-06-13T06:53:29.640563+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:53:29.640563+0000","last_undegraded":"2024-06-13T06:53:29.640563+0000","last_fullsized":"2024-06-13T06:53:29.640563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632622+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:53:29.632622+0000","last_peered":"2024-06-13T06:53:29.632622+0000","last_clean":"2024-06-13T06:53:29.632622+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:53:29.632622+0000","last_undegraded":"2024-06-13T06:53:29.632622+0000","last_fullsized":"2024-06-13T06:53:29.632622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628373+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:53:29.628373+0000","last_peered":"2024-06-13T06:53:29.628373+0000","last_clean":"2024-06-13T06:53:29.628373+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:53:29.628373+0000","last_undegraded":"2024-06-13T06:53:29.628373+0000","last_fullsized":"2024-06-13T06:53:29.628373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73374,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1325,"num_read_kb":741,"num_write":185,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":220122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351055,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":1215075,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776217,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299493,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150458,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463019,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":618196,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1171456,"data_stored":619546,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149819,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554846,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724658,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:55:25.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T06:55:25.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 -- 172.21.15.7:0/2158793965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad781033f0 msgr2=0x7fad78109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 --2- 172.21.15.7:0/2158793965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad781033f0 0x7fad78109c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad70009ca0 tx=0x7fad7002a8c0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 -- 172.21.15.7:0/2158793965 shutdown_connections 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 --2- 172.21.15.7:0/2158793965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad781033f0 0x7fad78109c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 --2- 172.21.15.7:0/2158793965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad78102a30 0x7fad78102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 --2- 172.21.15.7:0/2158793965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 -- 172.21.15.7:0/2158793965 >> 172.21.15.7:0/2158793965 conn(0x7fad780fcdb0 msgr2=0x7fad780ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 -- 172.21.15.7:0/2158793965 shutdown_connections 2024-06-13T06:55:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.501+0000 7fad7f000640 1 -- 172.21.15.7:0/2158793965 wait complete. 2024-06-13T06:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.502+0000 7fad7f000640 1 Processor -- start 2024-06-13T06:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 -- start start 2024-06-13T06:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad781033f0 0x7fad780742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810d4f0 con 0x7fad78101840 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810d370 con 0x7fad78102a30 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810d670 con 0x7fad781033f0 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.503+0000 7fad7d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54782/0 (socket says 172.21.15.7:54782) 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39476/0 (socket says 172.21.15.7:39476) 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7e600640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad781033f0 0x7fad780742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 -- 172.21.15.7:0/3886040584 learned_addr learned my addr 172.21.15.7:0/3886040584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 -- 172.21.15.7:0/3886040584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad781033f0 msgr2=0x7fad780742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad781033f0 0x7fad780742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 -- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 msgr2=0x7fad78075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 -- 172.21.15.7:0/3886040584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad78074b60 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.504+0000 7fad7d200640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9f0 tx=0x7fad6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.505+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6400cd80 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.505+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad64010910 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.505+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64018990 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.505+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad781a1c80 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.505+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad781a21d0 con 0x7fad78102a30 2024-06-13T06:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.506+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78108a90 con 0x7fad78102a30 2024-06-13T06:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.510+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fad64010ac0 con 0x7fad78102a30 2024-06-13T06:55:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.511+0000 7fad76000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 0x7fad600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.511+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fad640cf630 con 0x7fad78102a30 2024-06-13T06:55:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.511+0000 7fad7dc00640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 0x7fad600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.512+0000 7fad7dc00640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 0x7fad600a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad78102890 tx=0x7fad6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:25.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.515+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad6409bba0 con 0x7fad78102a30 2024-06-13T06:55:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:25 smithi189 ceph-mon[36616]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:25 smithi175 ceph-mon[36886]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:25.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:25 smithi007 ceph-mon[32077]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.717+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad78061de0 con 0x7fad600a2b90 2024-06-13T06:55:25.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.739+0000 7fad76000640 1 -- 172.21.15.7:0/3886040584 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340792 (secure 0 0 0) 0x7fad78061de0 con 0x7fad600a2b90 2024-06-13T06:55:25.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.745+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 msgr2=0x7fad600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.745+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 0x7fad600a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad78102890 tx=0x7fad6c005f90 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.745+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 msgr2=0x7fad78073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.745+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad6400e9f0 tx=0x7fad6400eeb0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 shutdown_connections 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad600a2b90 0x7fad600a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad781033f0 0x7fad780742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad78102a30 0x7fad78073d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 --2- 172.21.15.7:0/3886040584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101840 0x7fad78075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 >> 172.21.15.7:0/3886040584 conn(0x7fad780fcdb0 msgr2=0x7fad78105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 shutdown_connections 2024-06-13T06:55:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:25.746+0000 7fad7f000640 1 -- 172.21.15.7:0/3886040584 wait complete. 2024-06-13T06:55:25.769 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-06-13T06:55:23.752843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663976,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1451,"num_read_kb":850,"num_write":368,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378928,"kb_used_data":13136,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562896,"statfs":{"total":1151940427776,"available":1151552405504,"internally_reserved":0,"allocated":13451264,"data_stored":8615222,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":37,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006921"},"pg_stats":[{"pgid":"3.3f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632165+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:53:29.632165+0000","last_peered":"2024-06-13T06:53:29.632165+0000","last_clean":"2024-06-13T06:53:29.632165+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:53:29.632165+0000","last_undegraded":"2024-06-13T06:53:29.632165+0000","last_fullsized":"2024-06-13T06:53:29.632165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626282+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:53:29.626282+0000","last_peered":"2024-06-13T06:53:29.626282+0000","last_clean":"2024-06-13T06:53:29.626282+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:53:29.626282+0000","last_undegraded":"2024-06-13T06:53:29.626282+0000","last_fullsized":"2024-06-13T06:53:29.626282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626415+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:53:29.626415+0000","last_peered":"2024-06-13T06:53:29.626415+0000","last_clean":"2024-06-13T06:53:29.626415+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:53:29.626415+0000","last_undegraded":"2024-06-13T06:53:29.626415+0000","last_fullsized":"2024-06-13T06:53:29.626415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"98'8","reported_seq":21,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622619+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:53:29.622619+0000","last_peered":"2024-06-13T06:53:29.622619+0000","last_clean":"2024-06-13T06:53:29.622619+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:53:29.622619+0000","last_undegraded":"2024-06-13T06:53:29.622619+0000","last_fullsized":"2024-06-13T06:53:29.622619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633492+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:53:29.633492+0000","last_peered":"2024-06-13T06:53:29.633492+0000","last_clean":"2024-06-13T06:53:29.633492+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:53:29.633492+0000","last_undegraded":"2024-06-13T06:53:29.633492+0000","last_fullsized":"2024-06-13T06:53:29.633492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633554+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:53:29.633554+0000","last_peered":"2024-06-13T06:53:29.633554+0000","last_clean":"2024-06-13T06:53:29.633554+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:53:29.633554+0000","last_undegraded":"2024-06-13T06:53:29.633554+0000","last_fullsized":"2024-06-13T06:53:29.633554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628843+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:53:29.628843+0000","last_peered":"2024-06-13T06:53:29.628843+0000","last_clean":"2024-06-13T06:53:29.628843+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:53:29.628843+0000","last_undegraded":"2024-06-13T06:53:29.628843+0000","last_fullsized":"2024-06-13T06:53:29.628843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638046+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:53:29.638046+0000","last_peered":"2024-06-13T06:53:29.638046+0000","last_clean":"2024-06-13T06:53:29.638046+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:53:29.638046+0000","last_undegraded":"2024-06-13T06:53:29.638046+0000","last_fullsized":"2024-06-13T06:53:29.638046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631256+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:53:29.631256+0000","last_peered":"2024-06-13T06:53:29.631256+0000","last_clean":"2024-06-13T06:53:29.631256+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:53:29.631256+0000","last_undegraded":"2024-06-13T06:53:29.631256+0000","last_fullsized":"2024-06-13T06:53:29.631256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"98'16","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627241+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:53:29.627241+0000","last_peered":"2024-06-13T06:53:29.627241+0000","last_clean":"2024-06-13T06:53:29.627241+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:53:29.627241+0000","last_undegraded":"2024-06-13T06:53:29.627241+0000","last_fullsized":"2024-06-13T06:53:29.627241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632454+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:53:29.632454+0000","last_peered":"2024-06-13T06:53:29.632454+0000","last_clean":"2024-06-13T06:53:29.632454+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:53:29.632454+0000","last_undegraded":"2024-06-13T06:53:29.632454+0000","last_fullsized":"2024-06-13T06:53:29.632454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"98'7","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635014+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:53:29.635014+0000","last_peered":"2024-06-13T06:53:29.635014+0000","last_clean":"2024-06-13T06:53:29.635014+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:53:29.635014+0000","last_undegraded":"2024-06-13T06:53:29.635014+0000","last_fullsized":"2024-06-13T06:53:29.635014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631788+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:53:29.631788+0000","last_peered":"2024-06-13T06:53:29.631788+0000","last_clean":"2024-06-13T06:53:29.631788+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:53:29.631788+0000","last_undegraded":"2024-06-13T06:53:29.631788+0000","last_fullsized":"2024-06-13T06:53:29.631788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630960+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:53:29.630960+0000","last_peered":"2024-06-13T06:53:29.630960+0000","last_clean":"2024-06-13T06:53:29.630960+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:53:29.630960+0000","last_undegraded":"2024-06-13T06:53:29.630960+0000","last_fullsized":"2024-06-13T06:53:29.630960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622955+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:53:29.622955+0000","last_peered":"2024-06-13T06:53:29.622955+0000","last_clean":"2024-06-13T06:53:29.622955+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:53:29.622955+0000","last_undegraded":"2024-06-13T06:53:29.622955+0000","last_fullsized":"2024-06-13T06:53:29.622955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632867+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:53:29.632867+0000","last_peered":"2024-06-13T06:53:29.632867+0000","last_clean":"2024-06-13T06:53:29.632867+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:53:29.632867+0000","last_undegraded":"2024-06-13T06:53:29.632867+0000","last_fullsized":"2024-06-13T06:53:29.632867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632172+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:53:29.632172+0000","last_peered":"2024-06-13T06:53:29.632172+0000","last_clean":"2024-06-13T06:53:29.632172+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:53:29.632172+0000","last_undegraded":"2024-06-13T06:53:29.632172+0000","last_fullsized":"2024-06-13T06:53:29.632172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634623+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:53:29.634623+0000","last_peered":"2024-06-13T06:53:29.634623+0000","last_clean":"2024-06-13T06:53:29.634623+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:53:29.634623+0000","last_undegraded":"2024-06-13T06:53:29.634623+0000","last_fullsized":"2024-06-13T06:53:29.634623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630073+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:53:29.630073+0000","last_peered":"2024-06-13T06:53:29.630073+0000","last_clean":"2024-06-13T06:53:29.630073+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:53:29.630073+0000","last_undegraded":"2024-06-13T06:53:29.630073+0000","last_fullsized":"2024-06-13T06:53:29.630073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.768380+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:52:57.768380+0000","last_peered":"2024-06-13T06:52:57.768380+0000","last_clean":"2024-06-13T06:52:57.768380+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:52:57.768380+0000","last_undegraded":"2024-06-13T06:52:57.768380+0000","last_fullsized":"2024-06-13T06:52:57.768380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464726+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:52:18.464726+0000","last_peered":"2024-06-13T06:52:18.464726+0000","last_clean":"2024-06-13T06:52:18.464726+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:52:18.464726+0000","last_undegraded":"2024-06-13T06:52:18.464726+0000","last_fullsized":"2024-06-13T06:52:18.464726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635549+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:53:29.635549+0000","last_peered":"2024-06-13T06:53:29.635549+0000","last_clean":"2024-06-13T06:53:29.635549+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:53:29.635549+0000","last_undegraded":"2024-06-13T06:53:29.635549+0000","last_fullsized":"2024-06-13T06:53:29.635549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.087653+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:52:57.087653+0000","last_peered":"2024-06-13T06:52:57.087653+0000","last_clean":"2024-06-13T06:52:57.087653+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:52:57.087653+0000","last_undegraded":"2024-06-13T06:52:57.087653+0000","last_fullsized":"2024-06-13T06:52:57.087653+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:15.257089+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:52:15.257089+0000","last_peered":"2024-06-13T06:52:15.257089+0000","last_clean":"2024-06-13T06:52:15.257089+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:52:15.257089+0000","last_undegraded":"2024-06-13T06:52:15.257089+0000","last_fullsized":"2024-06-13T06:52:15.257089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635752+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:53:29.635752+0000","last_peered":"2024-06-13T06:53:29.635752+0000","last_clean":"2024-06-13T06:53:29.635752+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:53:29.635752+0000","last_undegraded":"2024-06-13T06:53:29.635752+0000","last_fullsized":"2024-06-13T06:53:29.635752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891648+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891648+0000","last_peered":"2024-06-13T06:51:00.891648+0000","last_clean":"2024-06-13T06:51:00.891648+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891648+0000","last_undegraded":"2024-06-13T06:51:00.891648+0000","last_fullsized":"2024-06-13T06:51:00.891648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957691+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:52:42.957691+0000","last_peered":"2024-06-13T06:52:42.957691+0000","last_clean":"2024-06-13T06:52:42.957691+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:52:42.957691+0000","last_undegraded":"2024-06-13T06:52:42.957691+0000","last_fullsized":"2024-06-13T06:52:42.957691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634711+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:53:29.634711+0000","last_peered":"2024-06-13T06:53:29.634711+0000","last_clean":"2024-06-13T06:53:29.634711+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:53:29.634711+0000","last_undegraded":"2024-06-13T06:53:29.634711+0000","last_fullsized":"2024-06-13T06:53:29.634711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891222+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891222+0000","last_peered":"2024-06-13T06:51:00.891222+0000","last_clean":"2024-06-13T06:51:00.891222+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891222+0000","last_undegraded":"2024-06-13T06:51:00.891222+0000","last_fullsized":"2024-06-13T06:51:00.891222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957657+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:52:42.957657+0000","last_peered":"2024-06-13T06:52:42.957657+0000","last_clean":"2024-06-13T06:52:42.957657+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:52:42.957657+0000","last_undegraded":"2024-06-13T06:52:42.957657+0000","last_fullsized":"2024-06-13T06:52:42.957657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632012+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:53:29.632012+0000","last_peered":"2024-06-13T06:53:29.632012+0000","last_clean":"2024-06-13T06:53:29.632012+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:53:29.632012+0000","last_undegraded":"2024-06-13T06:53:29.632012+0000","last_fullsized":"2024-06-13T06:53:29.632012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881987+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881987+0000","last_peered":"2024-06-13T06:51:00.881987+0000","last_clean":"2024-06-13T06:51:00.881987+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881987+0000","last_undegraded":"2024-06-13T06:51:00.881987+0000","last_fullsized":"2024-06-13T06:51:00.881987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:19.156377+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:55:19.156377+0000","last_peered":"2024-06-13T06:55:19.156377+0000","last_clean":"2024-06-13T06:55:19.156377+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:55:19.156377+0000","last_undegraded":"2024-06-13T06:55:19.156377+0000","last_fullsized":"2024-06-13T06:55:19.156377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":217,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617187+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:53:29.617187+0000","last_peered":"2024-06-13T06:53:29.617187+0000","last_clean":"2024-06-13T06:53:29.617187+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:53:29.617187+0000","last_undegraded":"2024-06-13T06:53:29.617187+0000","last_fullsized":"2024-06-13T06:53:29.617187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882139+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882139+0000","last_peered":"2024-06-13T06:51:00.882139+0000","last_clean":"2024-06-13T06:51:00.882139+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882139+0000","last_undegraded":"2024-06-13T06:51:00.882139+0000","last_fullsized":"2024-06-13T06:51:00.882139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:45.352623+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:52:45.352623+0000","last_peered":"2024-06-13T06:52:45.352623+0000","last_clean":"2024-06-13T06:52:45.352623+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:52:45.352623+0000","last_undegraded":"2024-06-13T06:52:45.352623+0000","last_fullsized":"2024-06-13T06:52:45.352623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640132+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:53:29.640132+0000","last_peered":"2024-06-13T06:53:29.640132+0000","last_clean":"2024-06-13T06:53:29.640132+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:53:29.640132+0000","last_undegraded":"2024-06-13T06:53:29.640132+0000","last_fullsized":"2024-06-13T06:53:29.640132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108566+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:52:36.108566+0000","last_peered":"2024-06-13T06:52:36.108566+0000","last_clean":"2024-06-13T06:52:36.108566+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:52:36.108566+0000","last_undegraded":"2024-06-13T06:52:36.108566+0000","last_fullsized":"2024-06-13T06:52:36.108566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606565+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:52:36.606565+0000","last_peered":"2024-06-13T06:52:36.606565+0000","last_clean":"2024-06-13T06:52:36.606565+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:52:36.606565+0000","last_undegraded":"2024-06-13T06:52:36.606565+0000","last_fullsized":"2024-06-13T06:52:36.606565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637458+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:53:29.637458+0000","last_peered":"2024-06-13T06:53:29.637458+0000","last_clean":"2024-06-13T06:53:29.637458+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:53:29.637458+0000","last_undegraded":"2024-06-13T06:53:29.637458+0000","last_fullsized":"2024-06-13T06:53:29.637458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:29.820545+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:52:29.820545+0000","last_peered":"2024-06-13T06:52:29.820545+0000","last_clean":"2024-06-13T06:52:29.820545+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:52:29.820545+0000","last_undegraded":"2024-06-13T06:52:29.820545+0000","last_fullsized":"2024-06-13T06:52:29.820545+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250016+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:52:40.250016+0000","last_peered":"2024-06-13T06:52:40.250016+0000","last_clean":"2024-06-13T06:52:40.250016+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:52:40.250016+0000","last_undegraded":"2024-06-13T06:52:40.250016+0000","last_fullsized":"2024-06-13T06:52:40.250016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639877+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:53:29.639877+0000","last_peered":"2024-06-13T06:53:29.639877+0000","last_clean":"2024-06-13T06:53:29.639877+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:53:29.639877+0000","last_undegraded":"2024-06-13T06:53:29.639877+0000","last_fullsized":"2024-06-13T06:53:29.639877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:50.263717+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:52:50.263717+0000","last_peered":"2024-06-13T06:52:50.263717+0000","last_clean":"2024-06-13T06:52:50.263717+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:52:50.263717+0000","last_undegraded":"2024-06-13T06:52:50.263717+0000","last_fullsized":"2024-06-13T06:52:50.263717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:19.155728+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:55:19.155728+0000","last_peered":"2024-06-13T06:55:19.155728+0000","last_clean":"2024-06-13T06:55:19.155728+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:55:19.155728+0000","last_undegraded":"2024-06-13T06:55:19.155728+0000","last_fullsized":"2024-06-13T06:55:19.155728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":215,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634544+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:53:29.634544+0000","last_peered":"2024-06-13T06:53:29.634544+0000","last_clean":"2024-06-13T06:53:29.634544+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:53:29.634544+0000","last_undegraded":"2024-06-13T06:53:29.634544+0000","last_fullsized":"2024-06-13T06:53:29.634544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470112+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:52:53.470112+0000","last_peered":"2024-06-13T06:52:53.470112+0000","last_clean":"2024-06-13T06:52:53.470112+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:52:53.470112+0000","last_undegraded":"2024-06-13T06:52:53.470112+0000","last_fullsized":"2024-06-13T06:52:53.470112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033534+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:52:36.033534+0000","last_peered":"2024-06-13T06:52:36.033534+0000","last_clean":"2024-06-13T06:52:36.033534+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:52:36.033534+0000","last_undegraded":"2024-06-13T06:52:36.033534+0000","last_fullsized":"2024-06-13T06:52:36.033534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630475+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:53:29.630475+0000","last_peered":"2024-06-13T06:53:29.630475+0000","last_clean":"2024-06-13T06:53:29.630475+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:53:29.630475+0000","last_undegraded":"2024-06-13T06:53:29.630475+0000","last_fullsized":"2024-06-13T06:53:29.630475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:52.086907+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:52:52.086907+0000","last_peered":"2024-06-13T06:52:52.086907+0000","last_clean":"2024-06-13T06:52:52.086907+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:52:52.086907+0000","last_undegraded":"2024-06-13T06:52:52.086907+0000","last_fullsized":"2024-06-13T06:52:52.086907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250061+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:52:40.250061+0000","last_peered":"2024-06-13T06:52:40.250061+0000","last_clean":"2024-06-13T06:52:40.250061+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:52:40.250061+0000","last_undegraded":"2024-06-13T06:52:40.250061+0000","last_fullsized":"2024-06-13T06:52:40.250061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633279+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:53:29.633279+0000","last_peered":"2024-06-13T06:53:29.633279+0000","last_clean":"2024-06-13T06:53:29.633279+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:53:29.633279+0000","last_undegraded":"2024-06-13T06:53:29.633279+0000","last_fullsized":"2024-06-13T06:53:29.633279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470157+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:52:53.470157+0000","last_peered":"2024-06-13T06:52:53.470157+0000","last_clean":"2024-06-13T06:52:53.470157+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:52:53.470157+0000","last_undegraded":"2024-06-13T06:52:53.470157+0000","last_fullsized":"2024-06-13T06:52:53.470157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'40","reported_seq":524,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.167306+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:55:20.167306+0000","last_peered":"2024-06-13T06:55:20.167306+0000","last_clean":"2024-06-13T06:55:20.167306+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:55:20.167306+0000","last_undegraded":"2024-06-13T06:55:20.167306+0000","last_fullsized":"2024-06-13T06:55:20.167306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":442,"num_read_kb":289,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.612063+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:53:29.612063+0000","last_peered":"2024-06-13T06:53:29.612063+0000","last_clean":"2024-06-13T06:53:29.612063+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:53:29.612063+0000","last_undegraded":"2024-06-13T06:53:29.612063+0000","last_fullsized":"2024-06-13T06:53:29.612063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:55.251812+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:52:55.251812+0000","last_peered":"2024-06-13T06:52:55.251812+0000","last_clean":"2024-06-13T06:52:55.251812+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:52:55.251812+0000","last_undegraded":"2024-06-13T06:52:55.251812+0000","last_fullsized":"2024-06-13T06:52:55.251812+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.765822+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:52:32.765822+0000","last_peered":"2024-06-13T06:52:32.765822+0000","last_clean":"2024-06-13T06:52:32.765822+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:52:32.765822+0000","last_undegraded":"2024-06-13T06:52:32.765822+0000","last_fullsized":"2024-06-13T06:52:32.765822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632382+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:53:29.632382+0000","last_peered":"2024-06-13T06:53:29.632382+0000","last_clean":"2024-06-13T06:53:29.632382+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:53:29.632382+0000","last_undegraded":"2024-06-13T06:53:29.632382+0000","last_fullsized":"2024-06-13T06:53:29.632382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:51.110319+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:52:51.110319+0000","last_peered":"2024-06-13T06:52:51.110319+0000","last_clean":"2024-06-13T06:52:51.110319+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:52:51.110319+0000","last_undegraded":"2024-06-13T06:52:51.110319+0000","last_fullsized":"2024-06-13T06:52:51.110319+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:35.249437+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:52:35.249437+0000","last_peered":"2024-06-13T06:52:35.249437+0000","last_clean":"2024-06-13T06:52:35.249437+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:52:35.249437+0000","last_undegraded":"2024-06-13T06:52:35.249437+0000","last_fullsized":"2024-06-13T06:52:35.249437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633901+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:53:29.633901+0000","last_peered":"2024-06-13T06:53:29.633901+0000","last_clean":"2024-06-13T06:53:29.633901+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:53:29.633901+0000","last_undegraded":"2024-06-13T06:53:29.633901+0000","last_fullsized":"2024-06-13T06:53:29.633901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:58.471021+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:52:58.471021+0000","last_peered":"2024-06-13T06:52:58.471021+0000","last_clean":"2024-06-13T06:52:58.471021+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:52:58.471021+0000","last_undegraded":"2024-06-13T06:52:58.471021+0000","last_fullsized":"2024-06-13T06:52:58.471021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.326188+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:55:20.326188+0000","last_peered":"2024-06-13T06:55:20.326188+0000","last_clean":"2024-06-13T06:55:20.326188+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:55:20.326188+0000","last_undegraded":"2024-06-13T06:55:20.326188+0000","last_fullsized":"2024-06-13T06:55:20.326188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1425,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633677+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:53:29.633677+0000","last_peered":"2024-06-13T06:53:29.633677+0000","last_clean":"2024-06-13T06:53:29.633677+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:53:29.633677+0000","last_undegraded":"2024-06-13T06:53:29.633677+0000","last_fullsized":"2024-06-13T06:53:29.633677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:56.609813+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:52:56.609813+0000","last_peered":"2024-06-13T06:52:56.609813+0000","last_clean":"2024-06-13T06:52:56.609813+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:52:56.609813+0000","last_undegraded":"2024-06-13T06:52:56.609813+0000","last_fullsized":"2024-06-13T06:52:56.609813+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.956450+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:52:32.956450+0000","last_peered":"2024-06-13T06:52:32.956450+0000","last_clean":"2024-06-13T06:52:32.956450+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:52:32.956450+0000","last_undegraded":"2024-06-13T06:52:32.956450+0000","last_fullsized":"2024-06-13T06:52:32.956450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631663+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:53:29.631663+0000","last_peered":"2024-06-13T06:53:29.631663+0000","last_clean":"2024-06-13T06:53:29.631663+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:53:29.631663+0000","last_undegraded":"2024-06-13T06:53:29.631663+0000","last_fullsized":"2024-06-13T06:53:29.631663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250034+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:52:40.250034+0000","last_peered":"2024-06-13T06:52:40.250034+0000","last_clean":"2024-06-13T06:52:40.250034+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:52:40.250034+0000","last_undegraded":"2024-06-13T06:52:40.250034+0000","last_fullsized":"2024-06-13T06:52:40.250034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:42.970288+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:54:42.970288+0000","last_peered":"2024-06-13T06:54:42.970288+0000","last_clean":"2024-06-13T06:54:42.970288+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:54:42.970288+0000","last_undegraded":"2024-06-13T06:54:42.970288+0000","last_fullsized":"2024-06-13T06:54:42.970288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627895+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:53:29.627895+0000","last_peered":"2024-06-13T06:53:29.627895+0000","last_clean":"2024-06-13T06:53:29.627895+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:53:29.627895+0000","last_undegraded":"2024-06-13T06:53:29.627895+0000","last_fullsized":"2024-06-13T06:53:29.627895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.957155+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:52:37.957155+0000","last_peered":"2024-06-13T06:52:37.957155+0000","last_clean":"2024-06-13T06:52:37.957155+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:52:37.957155+0000","last_undegraded":"2024-06-13T06:52:37.957155+0000","last_fullsized":"2024-06-13T06:52:37.957155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464620+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:52:18.464620+0000","last_peered":"2024-06-13T06:52:18.464620+0000","last_clean":"2024-06-13T06:52:18.464620+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:52:18.464620+0000","last_undegraded":"2024-06-13T06:52:18.464620+0000","last_fullsized":"2024-06-13T06:52:18.464620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636785+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:53:29.636785+0000","last_peered":"2024-06-13T06:53:29.636785+0000","last_clean":"2024-06-13T06:53:29.636785+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:53:29.636785+0000","last_undegraded":"2024-06-13T06:53:29.636785+0000","last_fullsized":"2024-06-13T06:53:29.636785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606649+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:52:36.606649+0000","last_peered":"2024-06-13T06:52:36.606649+0000","last_clean":"2024-06-13T06:52:36.606649+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:52:36.606649+0000","last_undegraded":"2024-06-13T06:52:36.606649+0000","last_fullsized":"2024-06-13T06:52:36.606649+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:16.031317+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:52:16.031317+0000","last_peered":"2024-06-13T06:52:16.031317+0000","last_clean":"2024-06-13T06:52:16.031317+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:52:16.031317+0000","last_undegraded":"2024-06-13T06:52:16.031317+0000","last_fullsized":"2024-06-13T06:52:16.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633623+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:53:29.633623+0000","last_peered":"2024-06-13T06:53:29.633623+0000","last_clean":"2024-06-13T06:53:29.633623+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:53:29.633623+0000","last_undegraded":"2024-06-13T06:53:29.633623+0000","last_fullsized":"2024-06-13T06:53:29.633623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:41.607532+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:52:41.607532+0000","last_peered":"2024-06-13T06:52:41.607532+0000","last_clean":"2024-06-13T06:52:41.607532+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:52:41.607532+0000","last_undegraded":"2024-06-13T06:52:41.607532+0000","last_fullsized":"2024-06-13T06:52:41.607532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623878+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:53:29.623878+0000","last_peered":"2024-06-13T06:53:29.623878+0000","last_clean":"2024-06-13T06:53:29.623878+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:53:29.623878+0000","last_undegraded":"2024-06-13T06:53:29.623878+0000","last_fullsized":"2024-06-13T06:53:29.623878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'6","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.324967+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:55:20.324967+0000","last_peered":"2024-06-13T06:55:20.324967+0000","last_clean":"2024-06-13T06:55:20.324967+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:55:20.324967+0000","last_undegraded":"2024-06-13T06:55:20.324967+0000","last_fullsized":"2024-06-13T06:55:20.324967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033518+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:52:36.033518+0000","last_peered":"2024-06-13T06:52:36.033518+0000","last_clean":"2024-06-13T06:52:36.033518+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:52:36.033518+0000","last_undegraded":"2024-06-13T06:52:36.033518+0000","last_fullsized":"2024-06-13T06:52:36.033518+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636741+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:53:29.636741+0000","last_peered":"2024-06-13T06:53:29.636741+0000","last_clean":"2024-06-13T06:53:29.636741+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:53:29.636741+0000","last_undegraded":"2024-06-13T06:53:29.636741+0000","last_fullsized":"2024-06-13T06:53:29.636741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.082967+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:52:17.082967+0000","last_peered":"2024-06-13T06:52:17.082967+0000","last_clean":"2024-06-13T06:52:17.082967+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:52:17.082967+0000","last_undegraded":"2024-06-13T06:52:17.082967+0000","last_fullsized":"2024-06-13T06:52:17.082967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.037470+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:53:11.037470+0000","last_peered":"2024-06-13T06:53:11.037470+0000","last_clean":"2024-06-13T06:53:11.037470+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:53:11.037470+0000","last_undegraded":"2024-06-13T06:53:11.037470+0000","last_fullsized":"2024-06-13T06:53:11.037470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250077+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:52:40.250077+0000","last_peered":"2024-06-13T06:52:40.250077+0000","last_clean":"2024-06-13T06:52:40.250077+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:52:40.250077+0000","last_undegraded":"2024-06-13T06:52:40.250077+0000","last_fullsized":"2024-06-13T06:52:40.250077+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631450+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:53:29.631450+0000","last_peered":"2024-06-13T06:53:29.631450+0000","last_clean":"2024-06-13T06:53:29.631450+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:53:29.631450+0000","last_undegraded":"2024-06-13T06:53:29.631450+0000","last_fullsized":"2024-06-13T06:53:29.631450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:11.030897+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:52:11.030897+0000","last_peered":"2024-06-13T06:52:11.030897+0000","last_clean":"2024-06-13T06:52:11.030897+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:52:11.030897+0000","last_undegraded":"2024-06-13T06:52:11.030897+0000","last_fullsized":"2024-06-13T06:52:11.030897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108536+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:52:36.108536+0000","last_peered":"2024-06-13T06:52:36.108536+0000","last_clean":"2024-06-13T06:52:36.108536+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:52:36.108536+0000","last_undegraded":"2024-06-13T06:52:36.108536+0000","last_fullsized":"2024-06-13T06:52:36.108536+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.955053+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:52:17.955053+0000","last_peered":"2024-06-13T06:52:17.955053+0000","last_clean":"2024-06-13T06:52:17.955053+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:52:17.955053+0000","last_undegraded":"2024-06-13T06:52:17.955053+0000","last_fullsized":"2024-06-13T06:52:17.955053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628081+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:53:29.628081+0000","last_peered":"2024-06-13T06:53:29.628081+0000","last_clean":"2024-06-13T06:53:29.628081+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:53:29.628081+0000","last_undegraded":"2024-06-13T06:53:29.628081+0000","last_fullsized":"2024-06-13T06:53:29.628081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.259402+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:52:30.259402+0000","last_peered":"2024-06-13T06:52:30.259402+0000","last_clean":"2024-06-13T06:52:30.259402+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:52:30.259402+0000","last_undegraded":"2024-06-13T06:52:30.259402+0000","last_fullsized":"2024-06-13T06:52:30.259402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'13","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:33.486327+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:54:33.486327+0000","last_peered":"2024-06-13T06:54:33.486327+0000","last_clean":"2024-06-13T06:54:33.486327+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:54:33.486327+0000","last_undegraded":"2024-06-13T06:54:33.486327+0000","last_fullsized":"2024-06-13T06:54:33.486327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633370+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:53:29.633370+0000","last_peered":"2024-06-13T06:53:29.633370+0000","last_clean":"2024-06-13T06:53:29.633370+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:53:29.633370+0000","last_undegraded":"2024-06-13T06:53:29.633370+0000","last_fullsized":"2024-06-13T06:53:29.633370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.766412+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:52:37.766412+0000","last_peered":"2024-06-13T06:52:37.766412+0000","last_clean":"2024-06-13T06:52:37.766412+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:52:37.766412+0000","last_undegraded":"2024-06-13T06:52:37.766412+0000","last_fullsized":"2024-06-13T06:52:37.766412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:22.083460+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:52:22.083460+0000","last_peered":"2024-06-13T06:52:22.083460+0000","last_clean":"2024-06-13T06:52:22.083460+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:52:22.083460+0000","last_undegraded":"2024-06-13T06:52:22.083460+0000","last_fullsized":"2024-06-13T06:52:22.083460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635347+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:53:29.635347+0000","last_peered":"2024-06-13T06:53:29.635347+0000","last_clean":"2024-06-13T06:53:29.635347+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:53:29.635347+0000","last_undegraded":"2024-06-13T06:53:29.635347+0000","last_fullsized":"2024-06-13T06:53:29.635347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086435+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:52:47.086435+0000","last_peered":"2024-06-13T06:52:47.086435+0000","last_clean":"2024-06-13T06:52:47.086435+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:52:47.086435+0000","last_undegraded":"2024-06-13T06:52:47.086435+0000","last_fullsized":"2024-06-13T06:52:47.086435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:24.165534+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:52:24.165534+0000","last_peered":"2024-06-13T06:52:24.165534+0000","last_clean":"2024-06-13T06:52:24.165534+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:52:24.165534+0000","last_undegraded":"2024-06-13T06:52:24.165534+0000","last_fullsized":"2024-06-13T06:52:24.165534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.620463+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:53:29.620463+0000","last_peered":"2024-06-13T06:53:29.620463+0000","last_clean":"2024-06-13T06:53:29.620463+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:53:29.620463+0000","last_undegraded":"2024-06-13T06:53:29.620463+0000","last_fullsized":"2024-06-13T06:53:29.620463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:48.469183+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:52:48.469183+0000","last_peered":"2024-06-13T06:52:48.469183+0000","last_clean":"2024-06-13T06:52:48.469183+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:52:48.469183+0000","last_undegraded":"2024-06-13T06:52:48.469183+0000","last_fullsized":"2024-06-13T06:52:48.469183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:25.258413+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:52:25.258413+0000","last_peered":"2024-06-13T06:52:25.258413+0000","last_clean":"2024-06-13T06:52:25.258413+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:52:25.258413+0000","last_undegraded":"2024-06-13T06:52:25.258413+0000","last_fullsized":"2024-06-13T06:52:25.258413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.625157+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:53:29.625157+0000","last_peered":"2024-06-13T06:53:29.625157+0000","last_clean":"2024-06-13T06:53:29.625157+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:53:29.625157+0000","last_undegraded":"2024-06-13T06:53:29.625157+0000","last_fullsized":"2024-06-13T06:53:29.625157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":27,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086403+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:52:47.086403+0000","last_peered":"2024-06-13T06:52:47.086403+0000","last_clean":"2024-06-13T06:52:47.086403+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:52:47.086403+0000","last_undegraded":"2024-06-13T06:52:47.086403+0000","last_fullsized":"2024-06-13T06:52:47.086403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.112337+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:53:11.112337+0000","last_peered":"2024-06-13T06:53:11.112337+0000","last_clean":"2024-06-13T06:53:11.112337+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:53:11.112337+0000","last_undegraded":"2024-06-13T06:53:11.112337+0000","last_fullsized":"2024-06-13T06:53:11.112337+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639239+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:53:29.639239+0000","last_peered":"2024-06-13T06:53:29.639239+0000","last_clean":"2024-06-13T06:53:29.639239+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:53:29.639239+0000","last_undegraded":"2024-06-13T06:53:29.639239+0000","last_fullsized":"2024-06-13T06:53:29.639239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086424+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:52:47.086424+0000","last_peered":"2024-06-13T06:52:47.086424+0000","last_clean":"2024-06-13T06:52:47.086424+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:52:47.086424+0000","last_undegraded":"2024-06-13T06:52:47.086424+0000","last_fullsized":"2024-06-13T06:52:47.086424+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.955951+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:52:27.955951+0000","last_peered":"2024-06-13T06:52:27.955951+0000","last_clean":"2024-06-13T06:52:27.955951+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:52:27.955951+0000","last_undegraded":"2024-06-13T06:52:27.955951+0000","last_fullsized":"2024-06-13T06:52:27.955951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631970+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:53:29.631970+0000","last_peered":"2024-06-13T06:53:29.631970+0000","last_clean":"2024-06-13T06:53:29.631970+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:53:29.631970+0000","last_undegraded":"2024-06-13T06:53:29.631970+0000","last_fullsized":"2024-06-13T06:53:29.631970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:39.167112+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:52:39.167112+0000","last_peered":"2024-06-13T06:52:39.167112+0000","last_clean":"2024-06-13T06:52:39.167112+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:52:39.167112+0000","last_undegraded":"2024-06-13T06:52:39.167112+0000","last_fullsized":"2024-06-13T06:52:39.167112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.765245+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:52:27.765245+0000","last_peered":"2024-06-13T06:52:27.765245+0000","last_clean":"2024-06-13T06:52:27.765245+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:52:27.765245+0000","last_undegraded":"2024-06-13T06:52:27.765245+0000","last_fullsized":"2024-06-13T06:52:27.765245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632700+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:53:29.632700+0000","last_peered":"2024-06-13T06:53:29.632700+0000","last_clean":"2024-06-13T06:53:29.632700+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:53:29.632700+0000","last_undegraded":"2024-06-13T06:53:29.632700+0000","last_fullsized":"2024-06-13T06:53:29.632700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:44.822664+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:52:44.822664+0000","last_peered":"2024-06-13T06:52:44.822664+0000","last_clean":"2024-06-13T06:52:44.822664+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:52:44.822664+0000","last_undegraded":"2024-06-13T06:52:44.822664+0000","last_fullsized":"2024-06-13T06:52:44.822664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.248910+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:52:30.248910+0000","last_peered":"2024-06-13T06:52:30.248910+0000","last_clean":"2024-06-13T06:52:30.248910+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:52:30.248910+0000","last_undegraded":"2024-06-13T06:52:30.248910+0000","last_fullsized":"2024-06-13T06:52:30.248910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629774+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:53:29.629774+0000","last_peered":"2024-06-13T06:53:29.629774+0000","last_clean":"2024-06-13T06:53:29.629774+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:53:29.629774+0000","last_undegraded":"2024-06-13T06:53:29.629774+0000","last_fullsized":"2024-06-13T06:53:29.629774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.249974+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:52:40.249974+0000","last_peered":"2024-06-13T06:52:40.249974+0000","last_clean":"2024-06-13T06:52:40.249974+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:52:40.249974+0000","last_undegraded":"2024-06-13T06:52:40.249974+0000","last_fullsized":"2024-06-13T06:52:40.249974+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:26.605224+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:52:26.605224+0000","last_peered":"2024-06-13T06:52:26.605224+0000","last_clean":"2024-06-13T06:52:26.605224+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:52:26.605224+0000","last_undegraded":"2024-06-13T06:52:26.605224+0000","last_fullsized":"2024-06-13T06:52:26.605224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.621192+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:53:29.621192+0000","last_peered":"2024-06-13T06:53:29.621192+0000","last_clean":"2024-06-13T06:53:29.621192+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:53:29.621192+0000","last_undegraded":"2024-06-13T06:53:29.621192+0000","last_fullsized":"2024-06-13T06:53:29.621192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623559+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:53:29.623559+0000","last_peered":"2024-06-13T06:53:29.623559+0000","last_clean":"2024-06-13T06:53:29.623559+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:53:29.623559+0000","last_undegraded":"2024-06-13T06:53:29.623559+0000","last_fullsized":"2024-06-13T06:53:29.623559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"98'12","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.624643+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:53:29.624643+0000","last_peered":"2024-06-13T06:53:29.624643+0000","last_clean":"2024-06-13T06:53:29.624643+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:53:29.624643+0000","last_undegraded":"2024-06-13T06:53:29.624643+0000","last_fullsized":"2024-06-13T06:53:29.624643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629702+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:53:29.629702+0000","last_peered":"2024-06-13T06:53:29.629702+0000","last_clean":"2024-06-13T06:53:29.629702+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:53:29.629702+0000","last_undegraded":"2024-06-13T06:53:29.629702+0000","last_fullsized":"2024-06-13T06:53:29.629702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638325+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:53:29.638325+0000","last_peered":"2024-06-13T06:53:29.638325+0000","last_clean":"2024-06-13T06:53:29.638325+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:53:29.638325+0000","last_undegraded":"2024-06-13T06:53:29.638325+0000","last_fullsized":"2024-06-13T06:53:29.638325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631155+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:53:29.631155+0000","last_peered":"2024-06-13T06:53:29.631155+0000","last_clean":"2024-06-13T06:53:29.631155+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:53:29.631155+0000","last_undegraded":"2024-06-13T06:53:29.631155+0000","last_fullsized":"2024-06-13T06:53:29.631155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617420+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:53:29.617420+0000","last_peered":"2024-06-13T06:53:29.617420+0000","last_clean":"2024-06-13T06:53:29.617420+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:53:29.617420+0000","last_undegraded":"2024-06-13T06:53:29.617420+0000","last_fullsized":"2024-06-13T06:53:29.617420+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630989+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:53:29.630989+0000","last_peered":"2024-06-13T06:53:29.630989+0000","last_clean":"2024-06-13T06:53:29.630989+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:53:29.630989+0000","last_undegraded":"2024-06-13T06:53:29.630989+0000","last_fullsized":"2024-06-13T06:53:29.630989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629961+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:53:29.629961+0000","last_peered":"2024-06-13T06:53:29.629961+0000","last_clean":"2024-06-13T06:53:29.629961+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:53:29.629961+0000","last_undegraded":"2024-06-13T06:53:29.629961+0000","last_fullsized":"2024-06-13T06:53:29.629961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640037+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:53:29.640037+0000","last_peered":"2024-06-13T06:53:29.640037+0000","last_clean":"2024-06-13T06:53:29.640037+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:53:29.640037+0000","last_undegraded":"2024-06-13T06:53:29.640037+0000","last_fullsized":"2024-06-13T06:53:29.640037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632710+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:53:29.632710+0000","last_peered":"2024-06-13T06:53:29.632710+0000","last_clean":"2024-06-13T06:53:29.632710+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:53:29.632710+0000","last_undegraded":"2024-06-13T06:53:29.632710+0000","last_fullsized":"2024-06-13T06:53:29.632710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640563+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:53:29.640563+0000","last_peered":"2024-06-13T06:53:29.640563+0000","last_clean":"2024-06-13T06:53:29.640563+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:53:29.640563+0000","last_undegraded":"2024-06-13T06:53:29.640563+0000","last_fullsized":"2024-06-13T06:53:29.640563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632622+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:53:29.632622+0000","last_peered":"2024-06-13T06:53:29.632622+0000","last_clean":"2024-06-13T06:53:29.632622+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:53:29.632622+0000","last_undegraded":"2024-06-13T06:53:29.632622+0000","last_fullsized":"2024-06-13T06:53:29.632622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628373+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:53:29.628373+0000","last_peered":"2024-06-13T06:53:29.628373+0000","last_clean":"2024-06-13T06:53:29.628373+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:53:29.628373+0000","last_undegraded":"2024-06-13T06:53:29.628373+0000","last_fullsized":"2024-06-13T06:53:29.628373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73374,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1325,"num_read_kb":741,"num_write":185,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":220122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351055,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":1215075,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776217,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299493,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150458,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463019,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":618196,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1171456,"data_stored":619546,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149819,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554846,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724658,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:55:25.774 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-13T06:55:25.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-13T06:55:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 -- 172.21.15.7:0/2668874404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc101a10 msgr2=0x7fcddc101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 --2- 172.21.15.7:0/2668874404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc101a10 0x7fcddc101e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a3f0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 -- 172.21.15.7:0/2668874404 shutdown_connections 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 --2- 172.21.15.7:0/2668874404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc1035c0 0x7fcddc109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 --2- 172.21.15.7:0/2668874404 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc102c00 0x7fcddc103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 --2- 172.21.15.7:0/2668874404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc101a10 0x7fcddc101e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 -- 172.21.15.7:0/2668874404 >> 172.21.15.7:0/2668874404 conn(0x7fcddc0fcf80 msgr2=0x7fcddc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 -- 172.21.15.7:0/2668874404 shutdown_connections 2024-06-13T06:55:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.012+0000 7fcde1600640 1 -- 172.21.15.7:0/2668874404 wait complete. 2024-06-13T06:55:26.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.013+0000 7fcde1600640 1 Processor -- start 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.013+0000 7fcde1600640 1 -- start start 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 0x7fcddc073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc1035c0 0x7fcddc0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10d6e0 con 0x7fcddc1035c0 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10d560 con 0x7fcddc101a10 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcde1600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc10d860 con 0x7fcddc102c00 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcddaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcddaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54804/0 (socket says 172.21.15.7:54804) 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcddaa00640 1 -- 172.21.15.7:0/1333241030 learned_addr learned my addr 172.21.15.7:0/1333241030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcddb400640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc1035c0 0x7fcddc0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.014+0000 7fcdda000640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 0x7fcddc073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 -- 172.21.15.7:0/1333241030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 msgr2=0x7fcddc075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 -- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc1035c0 msgr2=0x7fcddc0742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc1035c0 0x7fcddc0742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 -- 172.21.15.7:0/1333241030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc074b60 con 0x7fcddc102c00 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcddaa00640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdda000640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 0x7fcddc073d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400ccc0 con 0x7fcddc102c00 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4010910 con 0x7fcddc102c00 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc1a1e10 con 0x7fcddc102c00 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4018960 con 0x7fcddc102c00 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.015+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a2270 con 0x7fcddc102c00 2024-06-13T06:55:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.017+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc108cd0 con 0x7fcddc102c00 2024-06-13T06:55:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.019+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcdc4010ac0 con 0x7fcddc102c00 2024-06-13T06:55:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.020+0000 7fcdd3400640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 0x7fcdc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.020+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcdc40cf6a0 con 0x7fcddc102c00 2024-06-13T06:55:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.021+0000 7fcddaa00640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 0x7fcdc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.021+0000 7fcddaa00640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 0x7fcdc00a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc000c00 tx=0x7fcdcc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.024+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcdc409bc10 con 0x7fcddc102c00 2024-06-13T06:55:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.228+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 --> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcddc061de0 con 0x7fcdc00a2b90 2024-06-13T06:55:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.243+0000 7fcdd3400640 1 -- 172.21.15.7:0/1333241030 <== mgr.14152 v2:172.21.15.7:6800/2901233239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340768 (secure 0 0 0) 0x7fcddc061de0 con 0x7fcdc00a2b90 2024-06-13T06:55:26.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 msgr2=0x7fcdc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 0x7fcdc00a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc000c00 tx=0x7fcdcc0315b0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 msgr2=0x7fcddc073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 0x7fcddc073d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400eeb0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 shutdown_connections 2024-06-13T06:55:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcdc00a2b90 0x7fcdc00a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcddc1035c0 0x7fcddc0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcddc102c00 0x7fcddc073d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 --2- 172.21.15.7:0/1333241030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcddc101a10 0x7fcddc075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 >> 172.21.15.7:0/1333241030 conn(0x7fcddc0fcf80 msgr2=0x7fcddc105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 shutdown_connections 2024-06-13T06:55:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:26.249+0000 7fcde1600640 1 -- 172.21.15.7:0/1333241030 wait complete. 2024-06-13T06:55:26.269 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-06-13T06:55:25.753979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663976,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1471,"num_read_kb":860,"num_write":368,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378928,"kb_used_data":13136,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562896,"statfs":{"total":1151940427776,"available":1151552405504,"internally_reserved":0,"allocated":13451264,"data_stored":8615222,"data_compressed":427329,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4926,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":47,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006762"},"pg_stats":[{"pgid":"3.3f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632165+0000","last_change":"2024-06-13T06:50:16.489420+0000","last_active":"2024-06-13T06:53:29.632165+0000","last_peered":"2024-06-13T06:53:29.632165+0000","last_clean":"2024-06-13T06:53:29.632165+0000","last_became_active":"2024-06-13T06:50:16.489034+0000","last_became_peered":"2024-06-13T06:50:16.489034+0000","last_unstale":"2024-06-13T06:53:29.632165+0000","last_undegraded":"2024-06-13T06:53:29.632165+0000","last_fullsized":"2024-06-13T06:53:29.632165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:10:28.792195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626282+0000","last_change":"2024-06-13T06:50:16.486723+0000","last_active":"2024-06-13T06:53:29.626282+0000","last_peered":"2024-06-13T06:53:29.626282+0000","last_clean":"2024-06-13T06:53:29.626282+0000","last_became_active":"2024-06-13T06:50:16.486247+0000","last_became_peered":"2024-06-13T06:50:16.486247+0000","last_unstale":"2024-06-13T06:53:29.626282+0000","last_undegraded":"2024-06-13T06:53:29.626282+0000","last_fullsized":"2024-06-13T06:53:29.626282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:37:55.372065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.626415+0000","last_change":"2024-06-13T06:50:16.488520+0000","last_active":"2024-06-13T06:53:29.626415+0000","last_peered":"2024-06-13T06:53:29.626415+0000","last_clean":"2024-06-13T06:53:29.626415+0000","last_became_active":"2024-06-13T06:50:16.488198+0000","last_became_peered":"2024-06-13T06:50:16.488198+0000","last_unstale":"2024-06-13T06:53:29.626415+0000","last_undegraded":"2024-06-13T06:53:29.626415+0000","last_fullsized":"2024-06-13T06:53:29.626415+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:13.629614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"98'8","reported_seq":21,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622619+0000","last_change":"2024-06-13T06:51:01.255825+0000","last_active":"2024-06-13T06:53:29.622619+0000","last_peered":"2024-06-13T06:53:29.622619+0000","last_clean":"2024-06-13T06:53:29.622619+0000","last_became_active":"2024-06-13T06:51:01.255233+0000","last_became_peered":"2024-06-13T06:51:01.255233+0000","last_unstale":"2024-06-13T06:53:29.622619+0000","last_undegraded":"2024-06-13T06:53:29.622619+0000","last_fullsized":"2024-06-13T06:53:29.622619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:38:28.304572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633492+0000","last_change":"2024-06-13T06:50:16.498156+0000","last_active":"2024-06-13T06:53:29.633492+0000","last_peered":"2024-06-13T06:53:29.633492+0000","last_clean":"2024-06-13T06:53:29.633492+0000","last_became_active":"2024-06-13T06:50:16.497791+0000","last_became_peered":"2024-06-13T06:50:16.497791+0000","last_unstale":"2024-06-13T06:53:29.633492+0000","last_undegraded":"2024-06-13T06:53:29.633492+0000","last_fullsized":"2024-06-13T06:53:29.633492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:01:44.156039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633554+0000","last_change":"2024-06-13T06:50:16.490085+0000","last_active":"2024-06-13T06:53:29.633554+0000","last_peered":"2024-06-13T06:53:29.633554+0000","last_clean":"2024-06-13T06:53:29.633554+0000","last_became_active":"2024-06-13T06:50:16.489660+0000","last_became_peered":"2024-06-13T06:50:16.489660+0000","last_unstale":"2024-06-13T06:53:29.633554+0000","last_undegraded":"2024-06-13T06:53:29.633554+0000","last_fullsized":"2024-06-13T06:53:29.633554+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:00:02.734934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628843+0000","last_change":"2024-06-13T06:50:16.485023+0000","last_active":"2024-06-13T06:53:29.628843+0000","last_peered":"2024-06-13T06:53:29.628843+0000","last_clean":"2024-06-13T06:53:29.628843+0000","last_became_active":"2024-06-13T06:50:16.483679+0000","last_became_peered":"2024-06-13T06:50:16.483679+0000","last_unstale":"2024-06-13T06:53:29.628843+0000","last_undegraded":"2024-06-13T06:53:29.628843+0000","last_fullsized":"2024-06-13T06:53:29.628843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:05:00.576718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638046+0000","last_change":"2024-06-13T06:50:16.486917+0000","last_active":"2024-06-13T06:53:29.638046+0000","last_peered":"2024-06-13T06:53:29.638046+0000","last_clean":"2024-06-13T06:53:29.638046+0000","last_became_active":"2024-06-13T06:50:16.482459+0000","last_became_peered":"2024-06-13T06:50:16.482459+0000","last_unstale":"2024-06-13T06:53:29.638046+0000","last_undegraded":"2024-06-13T06:53:29.638046+0000","last_fullsized":"2024-06-13T06:53:29.638046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:47.021489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631256+0000","last_change":"2024-06-13T06:50:16.486474+0000","last_active":"2024-06-13T06:53:29.631256+0000","last_peered":"2024-06-13T06:53:29.631256+0000","last_clean":"2024-06-13T06:53:29.631256+0000","last_became_active":"2024-06-13T06:50:16.484926+0000","last_became_peered":"2024-06-13T06:50:16.484926+0000","last_unstale":"2024-06-13T06:53:29.631256+0000","last_undegraded":"2024-06-13T06:53:29.631256+0000","last_fullsized":"2024-06-13T06:53:29.631256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:39:41.356830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"98'16","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627241+0000","last_change":"2024-06-13T06:50:16.484998+0000","last_active":"2024-06-13T06:53:29.627241+0000","last_peered":"2024-06-13T06:53:29.627241+0000","last_clean":"2024-06-13T06:53:29.627241+0000","last_became_active":"2024-06-13T06:50:16.484636+0000","last_became_peered":"2024-06-13T06:50:16.484636+0000","last_unstale":"2024-06-13T06:53:29.627241+0000","last_undegraded":"2024-06-13T06:53:29.627241+0000","last_fullsized":"2024-06-13T06:53:29.627241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:05:08.834692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632454+0000","last_change":"2024-06-13T06:50:16.482831+0000","last_active":"2024-06-13T06:53:29.632454+0000","last_peered":"2024-06-13T06:53:29.632454+0000","last_clean":"2024-06-13T06:53:29.632454+0000","last_became_active":"2024-06-13T06:50:16.481866+0000","last_became_peered":"2024-06-13T06:50:16.481866+0000","last_unstale":"2024-06-13T06:53:29.632454+0000","last_undegraded":"2024-06-13T06:53:29.632454+0000","last_fullsized":"2024-06-13T06:53:29.632454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:17:05.930088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"98'7","reported_seq":19,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635014+0000","last_change":"2024-06-13T06:51:01.255759+0000","last_active":"2024-06-13T06:53:29.635014+0000","last_peered":"2024-06-13T06:53:29.635014+0000","last_clean":"2024-06-13T06:53:29.635014+0000","last_became_active":"2024-06-13T06:51:01.255219+0000","last_became_peered":"2024-06-13T06:51:01.255219+0000","last_unstale":"2024-06-13T06:53:29.635014+0000","last_undegraded":"2024-06-13T06:53:29.635014+0000","last_fullsized":"2024-06-13T06:53:29.635014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:25:35.879034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631788+0000","last_change":"2024-06-13T06:50:16.482300+0000","last_active":"2024-06-13T06:53:29.631788+0000","last_peered":"2024-06-13T06:53:29.631788+0000","last_clean":"2024-06-13T06:53:29.631788+0000","last_became_active":"2024-06-13T06:50:16.481856+0000","last_became_peered":"2024-06-13T06:50:16.481856+0000","last_unstale":"2024-06-13T06:53:29.631788+0000","last_undegraded":"2024-06-13T06:53:29.631788+0000","last_fullsized":"2024-06-13T06:53:29.631788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:12:58.133440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630960+0000","last_change":"2024-06-13T06:50:16.496677+0000","last_active":"2024-06-13T06:53:29.630960+0000","last_peered":"2024-06-13T06:53:29.630960+0000","last_clean":"2024-06-13T06:53:29.630960+0000","last_became_active":"2024-06-13T06:50:16.496229+0000","last_became_peered":"2024-06-13T06:50:16.496229+0000","last_unstale":"2024-06-13T06:53:29.630960+0000","last_undegraded":"2024-06-13T06:53:29.630960+0000","last_fullsized":"2024-06-13T06:53:29.630960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:49:06.154117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.622955+0000","last_change":"2024-06-13T06:50:16.480771+0000","last_active":"2024-06-13T06:53:29.622955+0000","last_peered":"2024-06-13T06:53:29.622955+0000","last_clean":"2024-06-13T06:53:29.622955+0000","last_became_active":"2024-06-13T06:50:16.480314+0000","last_became_peered":"2024-06-13T06:50:16.480314+0000","last_unstale":"2024-06-13T06:53:29.622955+0000","last_undegraded":"2024-06-13T06:53:29.622955+0000","last_fullsized":"2024-06-13T06:53:29.622955+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:14:18.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632867+0000","last_change":"2024-06-13T06:50:16.497347+0000","last_active":"2024-06-13T06:53:29.632867+0000","last_peered":"2024-06-13T06:53:29.632867+0000","last_clean":"2024-06-13T06:53:29.632867+0000","last_became_active":"2024-06-13T06:50:16.497015+0000","last_became_peered":"2024-06-13T06:50:16.497015+0000","last_unstale":"2024-06-13T06:53:29.632867+0000","last_undegraded":"2024-06-13T06:53:29.632867+0000","last_fullsized":"2024-06-13T06:53:29.632867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:58:29.992942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632172+0000","last_change":"2024-06-13T06:50:16.502623+0000","last_active":"2024-06-13T06:53:29.632172+0000","last_peered":"2024-06-13T06:53:29.632172+0000","last_clean":"2024-06-13T06:53:29.632172+0000","last_became_active":"2024-06-13T06:50:16.502294+0000","last_became_peered":"2024-06-13T06:50:16.502294+0000","last_unstale":"2024-06-13T06:53:29.632172+0000","last_undegraded":"2024-06-13T06:53:29.632172+0000","last_fullsized":"2024-06-13T06:53:29.632172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:05:56.712637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634623+0000","last_change":"2024-06-13T06:50:16.491559+0000","last_active":"2024-06-13T06:53:29.634623+0000","last_peered":"2024-06-13T06:53:29.634623+0000","last_clean":"2024-06-13T06:53:29.634623+0000","last_became_active":"2024-06-13T06:50:16.491167+0000","last_became_peered":"2024-06-13T06:50:16.491167+0000","last_unstale":"2024-06-13T06:53:29.634623+0000","last_undegraded":"2024-06-13T06:53:29.634623+0000","last_fullsized":"2024-06-13T06:53:29.634623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:28:38.027847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630073+0000","last_change":"2024-06-13T06:50:16.498451+0000","last_active":"2024-06-13T06:53:29.630073+0000","last_peered":"2024-06-13T06:53:29.630073+0000","last_clean":"2024-06-13T06:53:29.630073+0000","last_became_active":"2024-06-13T06:50:16.498062+0000","last_became_peered":"2024-06-13T06:50:16.498062+0000","last_unstale":"2024-06-13T06:53:29.630073+0000","last_undegraded":"2024-06-13T06:53:29.630073+0000","last_fullsized":"2024-06-13T06:53:29.630073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:35.045939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.768380+0000","last_change":"2024-06-13T06:50:29.744383+0000","last_active":"2024-06-13T06:52:57.768380+0000","last_peered":"2024-06-13T06:52:57.768380+0000","last_clean":"2024-06-13T06:52:57.768380+0000","last_became_active":"2024-06-13T06:50:29.743907+0000","last_became_peered":"2024-06-13T06:50:29.743907+0000","last_unstale":"2024-06-13T06:52:57.768380+0000","last_undegraded":"2024-06-13T06:52:57.768380+0000","last_fullsized":"2024-06-13T06:52:57.768380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:45:31.147546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464726+0000","last_change":"2024-06-13T06:50:11.851632+0000","last_active":"2024-06-13T06:52:18.464726+0000","last_peered":"2024-06-13T06:52:18.464726+0000","last_clean":"2024-06-13T06:52:18.464726+0000","last_became_active":"2024-06-13T06:50:11.851193+0000","last_became_peered":"2024-06-13T06:50:11.851193+0000","last_unstale":"2024-06-13T06:52:18.464726+0000","last_undegraded":"2024-06-13T06:52:18.464726+0000","last_fullsized":"2024-06-13T06:52:18.464726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:55:19.821193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635549+0000","last_change":"2024-06-13T06:50:16.501554+0000","last_active":"2024-06-13T06:53:29.635549+0000","last_peered":"2024-06-13T06:53:29.635549+0000","last_clean":"2024-06-13T06:53:29.635549+0000","last_became_active":"2024-06-13T06:50:16.499651+0000","last_became_peered":"2024-06-13T06:50:16.499651+0000","last_unstale":"2024-06-13T06:53:29.635549+0000","last_undegraded":"2024-06-13T06:53:29.635549+0000","last_fullsized":"2024-06-13T06:53:29.635549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:18:43.253482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:57.087653+0000","last_change":"2024-06-13T06:50:29.735758+0000","last_active":"2024-06-13T06:52:57.087653+0000","last_peered":"2024-06-13T06:52:57.087653+0000","last_clean":"2024-06-13T06:52:57.087653+0000","last_became_active":"2024-06-13T06:50:29.735375+0000","last_became_peered":"2024-06-13T06:50:29.735375+0000","last_unstale":"2024-06-13T06:52:57.087653+0000","last_undegraded":"2024-06-13T06:52:57.087653+0000","last_fullsized":"2024-06-13T06:52:57.087653+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:10:11.670129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:15.257089+0000","last_change":"2024-06-13T06:50:11.862392+0000","last_active":"2024-06-13T06:52:15.257089+0000","last_peered":"2024-06-13T06:52:15.257089+0000","last_clean":"2024-06-13T06:52:15.257089+0000","last_became_active":"2024-06-13T06:50:11.860753+0000","last_became_peered":"2024-06-13T06:50:11.860753+0000","last_unstale":"2024-06-13T06:52:15.257089+0000","last_undegraded":"2024-06-13T06:52:15.257089+0000","last_fullsized":"2024-06-13T06:52:15.257089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:27:23.691186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635752+0000","last_change":"2024-06-13T06:50:16.500250+0000","last_active":"2024-06-13T06:53:29.635752+0000","last_peered":"2024-06-13T06:53:29.635752+0000","last_clean":"2024-06-13T06:53:29.635752+0000","last_became_active":"2024-06-13T06:50:16.495467+0000","last_became_peered":"2024-06-13T06:50:16.495467+0000","last_unstale":"2024-06-13T06:53:29.635752+0000","last_undegraded":"2024-06-13T06:53:29.635752+0000","last_fullsized":"2024-06-13T06:53:29.635752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:13:06.437130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891648+0000","last_change":"2024-06-13T06:50:29.739247+0000","last_active":"2024-06-13T06:51:00.891648+0000","last_peered":"2024-06-13T06:51:00.891648+0000","last_clean":"2024-06-13T06:51:00.891648+0000","last_became_active":"2024-06-13T06:50:29.738580+0000","last_became_peered":"2024-06-13T06:50:29.738580+0000","last_unstale":"2024-06-13T06:51:00.891648+0000","last_undegraded":"2024-06-13T06:51:00.891648+0000","last_fullsized":"2024-06-13T06:51:00.891648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:41:20.488857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957691+0000","last_change":"2024-06-13T06:50:11.843949+0000","last_active":"2024-06-13T06:52:42.957691+0000","last_peered":"2024-06-13T06:52:42.957691+0000","last_clean":"2024-06-13T06:52:42.957691+0000","last_became_active":"2024-06-13T06:50:11.843350+0000","last_became_peered":"2024-06-13T06:50:11.843350+0000","last_unstale":"2024-06-13T06:52:42.957691+0000","last_undegraded":"2024-06-13T06:52:42.957691+0000","last_fullsized":"2024-06-13T06:52:42.957691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634711+0000","last_change":"2024-06-13T06:50:16.491648+0000","last_active":"2024-06-13T06:53:29.634711+0000","last_peered":"2024-06-13T06:53:29.634711+0000","last_clean":"2024-06-13T06:53:29.634711+0000","last_became_active":"2024-06-13T06:50:16.491336+0000","last_became_peered":"2024-06-13T06:50:16.491336+0000","last_unstale":"2024-06-13T06:53:29.634711+0000","last_undegraded":"2024-06-13T06:53:29.634711+0000","last_fullsized":"2024-06-13T06:53:29.634711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:12:57.442683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.891222+0000","last_change":"2024-06-13T06:50:29.727313+0000","last_active":"2024-06-13T06:51:00.891222+0000","last_peered":"2024-06-13T06:51:00.891222+0000","last_clean":"2024-06-13T06:51:00.891222+0000","last_became_active":"2024-06-13T06:50:29.726998+0000","last_became_peered":"2024-06-13T06:50:29.726998+0000","last_unstale":"2024-06-13T06:51:00.891222+0000","last_undegraded":"2024-06-13T06:51:00.891222+0000","last_fullsized":"2024-06-13T06:51:00.891222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:47.220675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:42.957657+0000","last_change":"2024-06-13T06:50:11.841851+0000","last_active":"2024-06-13T06:52:42.957657+0000","last_peered":"2024-06-13T06:52:42.957657+0000","last_clean":"2024-06-13T06:52:42.957657+0000","last_became_active":"2024-06-13T06:50:11.841432+0000","last_became_peered":"2024-06-13T06:50:11.841432+0000","last_unstale":"2024-06-13T06:52:42.957657+0000","last_undegraded":"2024-06-13T06:52:42.957657+0000","last_fullsized":"2024-06-13T06:52:42.957657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:49:00.680286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632012+0000","last_change":"2024-06-13T06:50:16.484743+0000","last_active":"2024-06-13T06:53:29.632012+0000","last_peered":"2024-06-13T06:53:29.632012+0000","last_clean":"2024-06-13T06:53:29.632012+0000","last_became_active":"2024-06-13T06:50:16.484424+0000","last_became_peered":"2024-06-13T06:50:16.484424+0000","last_unstale":"2024-06-13T06:53:29.632012+0000","last_undegraded":"2024-06-13T06:53:29.632012+0000","last_fullsized":"2024-06-13T06:53:29.632012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:05:56.382375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.881987+0000","last_change":"2024-06-13T06:50:29.719732+0000","last_active":"2024-06-13T06:51:00.881987+0000","last_peered":"2024-06-13T06:51:00.881987+0000","last_clean":"2024-06-13T06:51:00.881987+0000","last_became_active":"2024-06-13T06:50:29.718902+0000","last_became_peered":"2024-06-13T06:50:29.718902+0000","last_unstale":"2024-06-13T06:51:00.881987+0000","last_undegraded":"2024-06-13T06:51:00.881987+0000","last_fullsized":"2024-06-13T06:51:00.881987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:58:57.642635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:24.164072+0000","last_change":"2024-06-13T06:50:11.841166+0000","last_active":"2024-06-13T06:55:24.164072+0000","last_peered":"2024-06-13T06:55:24.164072+0000","last_clean":"2024-06-13T06:55:24.164072+0000","last_became_active":"2024-06-13T06:50:11.840762+0000","last_became_peered":"2024-06-13T06:50:11.840762+0000","last_unstale":"2024-06-13T06:55:24.164072+0000","last_undegraded":"2024-06-13T06:55:24.164072+0000","last_fullsized":"2024-06-13T06:55:24.164072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:30:00.344627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617187+0000","last_change":"2024-06-13T06:50:16.484663+0000","last_active":"2024-06-13T06:53:29.617187+0000","last_peered":"2024-06-13T06:53:29.617187+0000","last_clean":"2024-06-13T06:53:29.617187+0000","last_became_active":"2024-06-13T06:50:16.484292+0000","last_became_peered":"2024-06-13T06:50:16.484292+0000","last_unstale":"2024-06-13T06:53:29.617187+0000","last_undegraded":"2024-06-13T06:53:29.617187+0000","last_fullsized":"2024-06-13T06:53:29.617187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:55.846263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:51:00.882139+0000","last_change":"2024-06-13T06:50:29.741040+0000","last_active":"2024-06-13T06:51:00.882139+0000","last_peered":"2024-06-13T06:51:00.882139+0000","last_clean":"2024-06-13T06:51:00.882139+0000","last_became_active":"2024-06-13T06:50:29.740499+0000","last_became_peered":"2024-06-13T06:50:29.740499+0000","last_unstale":"2024-06-13T06:51:00.882139+0000","last_undegraded":"2024-06-13T06:51:00.882139+0000","last_fullsized":"2024-06-13T06:51:00.882139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:28:11.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:45.352623+0000","last_change":"2024-06-13T06:50:11.850366+0000","last_active":"2024-06-13T06:52:45.352623+0000","last_peered":"2024-06-13T06:52:45.352623+0000","last_clean":"2024-06-13T06:52:45.352623+0000","last_became_active":"2024-06-13T06:50:11.849913+0000","last_became_peered":"2024-06-13T06:50:11.849913+0000","last_unstale":"2024-06-13T06:52:45.352623+0000","last_undegraded":"2024-06-13T06:52:45.352623+0000","last_fullsized":"2024-06-13T06:52:45.352623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:22:07.852483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640132+0000","last_change":"2024-06-13T06:50:16.493314+0000","last_active":"2024-06-13T06:53:29.640132+0000","last_peered":"2024-06-13T06:53:29.640132+0000","last_clean":"2024-06-13T06:53:29.640132+0000","last_became_active":"2024-06-13T06:50:16.492951+0000","last_became_peered":"2024-06-13T06:50:16.492951+0000","last_unstale":"2024-06-13T06:53:29.640132+0000","last_undegraded":"2024-06-13T06:53:29.640132+0000","last_fullsized":"2024-06-13T06:53:29.640132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:47:27.885433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108566+0000","last_change":"2024-06-13T06:50:29.739241+0000","last_active":"2024-06-13T06:52:36.108566+0000","last_peered":"2024-06-13T06:52:36.108566+0000","last_clean":"2024-06-13T06:52:36.108566+0000","last_became_active":"2024-06-13T06:50:29.738818+0000","last_became_peered":"2024-06-13T06:50:29.738818+0000","last_unstale":"2024-06-13T06:52:36.108566+0000","last_undegraded":"2024-06-13T06:52:36.108566+0000","last_fullsized":"2024-06-13T06:52:36.108566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:45.308081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606565+0000","last_change":"2024-06-13T06:50:11.856866+0000","last_active":"2024-06-13T06:52:36.606565+0000","last_peered":"2024-06-13T06:52:36.606565+0000","last_clean":"2024-06-13T06:52:36.606565+0000","last_became_active":"2024-06-13T06:50:11.856464+0000","last_became_peered":"2024-06-13T06:50:11.856464+0000","last_unstale":"2024-06-13T06:52:36.606565+0000","last_undegraded":"2024-06-13T06:52:36.606565+0000","last_fullsized":"2024-06-13T06:52:36.606565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:53:37.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.637458+0000","last_change":"2024-06-13T06:50:16.493830+0000","last_active":"2024-06-13T06:53:29.637458+0000","last_peered":"2024-06-13T06:53:29.637458+0000","last_clean":"2024-06-13T06:53:29.637458+0000","last_became_active":"2024-06-13T06:50:16.492414+0000","last_became_peered":"2024-06-13T06:50:16.492414+0000","last_unstale":"2024-06-13T06:53:29.637458+0000","last_undegraded":"2024-06-13T06:53:29.637458+0000","last_fullsized":"2024-06-13T06:53:29.637458+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:58:44.348384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:29.820545+0000","last_change":"2024-06-13T06:50:29.731655+0000","last_active":"2024-06-13T06:52:29.820545+0000","last_peered":"2024-06-13T06:52:29.820545+0000","last_clean":"2024-06-13T06:52:29.820545+0000","last_became_active":"2024-06-13T06:50:29.731189+0000","last_became_peered":"2024-06-13T06:50:29.731189+0000","last_unstale":"2024-06-13T06:52:29.820545+0000","last_undegraded":"2024-06-13T06:52:29.820545+0000","last_fullsized":"2024-06-13T06:52:29.820545+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:38:25.129549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250016+0000","last_change":"2024-06-13T06:50:11.841112+0000","last_active":"2024-06-13T06:52:40.250016+0000","last_peered":"2024-06-13T06:52:40.250016+0000","last_clean":"2024-06-13T06:52:40.250016+0000","last_became_active":"2024-06-13T06:50:11.840696+0000","last_became_peered":"2024-06-13T06:50:11.840696+0000","last_unstale":"2024-06-13T06:52:40.250016+0000","last_undegraded":"2024-06-13T06:52:40.250016+0000","last_fullsized":"2024-06-13T06:52:40.250016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:47:37.026053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639877+0000","last_change":"2024-06-13T06:50:16.498549+0000","last_active":"2024-06-13T06:53:29.639877+0000","last_peered":"2024-06-13T06:53:29.639877+0000","last_clean":"2024-06-13T06:53:29.639877+0000","last_became_active":"2024-06-13T06:50:16.498168+0000","last_became_peered":"2024-06-13T06:50:16.498168+0000","last_unstale":"2024-06-13T06:53:29.639877+0000","last_undegraded":"2024-06-13T06:53:29.639877+0000","last_fullsized":"2024-06-13T06:53:29.639877+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:19:14.415633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:50.263717+0000","last_change":"2024-06-13T06:50:29.736271+0000","last_active":"2024-06-13T06:52:50.263717+0000","last_peered":"2024-06-13T06:52:50.263717+0000","last_clean":"2024-06-13T06:52:50.263717+0000","last_became_active":"2024-06-13T06:50:29.735744+0000","last_became_peered":"2024-06-13T06:50:29.735744+0000","last_unstale":"2024-06-13T06:52:50.263717+0000","last_undegraded":"2024-06-13T06:52:50.263717+0000","last_fullsized":"2024-06-13T06:52:50.263717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:01:07.301771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":265,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:24.163424+0000","last_change":"2024-06-13T06:50:11.856954+0000","last_active":"2024-06-13T06:55:24.163424+0000","last_peered":"2024-06-13T06:55:24.163424+0000","last_clean":"2024-06-13T06:55:24.163424+0000","last_became_active":"2024-06-13T06:50:11.856553+0000","last_became_peered":"2024-06-13T06:50:11.856553+0000","last_unstale":"2024-06-13T06:55:24.163424+0000","last_undegraded":"2024-06-13T06:55:24.163424+0000","last_fullsized":"2024-06-13T06:55:24.163424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:43:44.239375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.634544+0000","last_change":"2024-06-13T06:50:16.488460+0000","last_active":"2024-06-13T06:53:29.634544+0000","last_peered":"2024-06-13T06:53:29.634544+0000","last_clean":"2024-06-13T06:53:29.634544+0000","last_became_active":"2024-06-13T06:50:16.488141+0000","last_became_peered":"2024-06-13T06:50:16.488141+0000","last_unstale":"2024-06-13T06:53:29.634544+0000","last_undegraded":"2024-06-13T06:53:29.634544+0000","last_fullsized":"2024-06-13T06:53:29.634544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:31:21.752641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470112+0000","last_change":"2024-06-13T06:50:29.735220+0000","last_active":"2024-06-13T06:52:53.470112+0000","last_peered":"2024-06-13T06:52:53.470112+0000","last_clean":"2024-06-13T06:52:53.470112+0000","last_became_active":"2024-06-13T06:50:29.734117+0000","last_became_peered":"2024-06-13T06:50:29.734117+0000","last_unstale":"2024-06-13T06:52:53.470112+0000","last_undegraded":"2024-06-13T06:52:53.470112+0000","last_fullsized":"2024-06-13T06:52:53.470112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:24:43.032207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033534+0000","last_change":"2024-06-13T06:50:11.841605+0000","last_active":"2024-06-13T06:52:36.033534+0000","last_peered":"2024-06-13T06:52:36.033534+0000","last_clean":"2024-06-13T06:52:36.033534+0000","last_became_active":"2024-06-13T06:50:11.841218+0000","last_became_peered":"2024-06-13T06:50:11.841218+0000","last_unstale":"2024-06-13T06:52:36.033534+0000","last_undegraded":"2024-06-13T06:52:36.033534+0000","last_fullsized":"2024-06-13T06:52:36.033534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:29:56.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630475+0000","last_change":"2024-06-13T06:50:16.496222+0000","last_active":"2024-06-13T06:53:29.630475+0000","last_peered":"2024-06-13T06:53:29.630475+0000","last_clean":"2024-06-13T06:53:29.630475+0000","last_became_active":"2024-06-13T06:50:16.495267+0000","last_became_peered":"2024-06-13T06:50:16.495267+0000","last_unstale":"2024-06-13T06:53:29.630475+0000","last_undegraded":"2024-06-13T06:53:29.630475+0000","last_fullsized":"2024-06-13T06:53:29.630475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:02:25.793198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:52.086907+0000","last_change":"2024-06-13T06:50:29.738807+0000","last_active":"2024-06-13T06:52:52.086907+0000","last_peered":"2024-06-13T06:52:52.086907+0000","last_clean":"2024-06-13T06:52:52.086907+0000","last_became_active":"2024-06-13T06:50:29.738243+0000","last_became_peered":"2024-06-13T06:50:29.738243+0000","last_unstale":"2024-06-13T06:52:52.086907+0000","last_undegraded":"2024-06-13T06:52:52.086907+0000","last_fullsized":"2024-06-13T06:52:52.086907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:06.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250061+0000","last_change":"2024-06-13T06:50:11.848330+0000","last_active":"2024-06-13T06:52:40.250061+0000","last_peered":"2024-06-13T06:52:40.250061+0000","last_clean":"2024-06-13T06:52:40.250061+0000","last_became_active":"2024-06-13T06:50:11.847840+0000","last_became_peered":"2024-06-13T06:50:11.847840+0000","last_unstale":"2024-06-13T06:52:40.250061+0000","last_undegraded":"2024-06-13T06:52:40.250061+0000","last_fullsized":"2024-06-13T06:52:40.250061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:59:59.375200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633279+0000","last_change":"2024-06-13T06:50:16.483953+0000","last_active":"2024-06-13T06:53:29.633279+0000","last_peered":"2024-06-13T06:53:29.633279+0000","last_clean":"2024-06-13T06:53:29.633279+0000","last_became_active":"2024-06-13T06:50:16.483602+0000","last_became_peered":"2024-06-13T06:50:16.483602+0000","last_unstale":"2024-06-13T06:53:29.633279+0000","last_undegraded":"2024-06-13T06:53:29.633279+0000","last_fullsized":"2024-06-13T06:53:29.633279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:43:26.110150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:53.470157+0000","last_change":"2024-06-13T06:50:29.730498+0000","last_active":"2024-06-13T06:52:53.470157+0000","last_peered":"2024-06-13T06:52:53.470157+0000","last_clean":"2024-06-13T06:52:53.470157+0000","last_became_active":"2024-06-13T06:50:29.730067+0000","last_became_peered":"2024-06-13T06:50:29.730067+0000","last_unstale":"2024-06-13T06:52:53.470157+0000","last_undegraded":"2024-06-13T06:52:53.470157+0000","last_fullsized":"2024-06-13T06:52:53.470157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:39:31.262840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'40","reported_seq":534,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:25.174507+0000","last_change":"2024-06-13T06:50:11.841738+0000","last_active":"2024-06-13T06:55:25.174507+0000","last_peered":"2024-06-13T06:55:25.174507+0000","last_clean":"2024-06-13T06:55:25.174507+0000","last_became_active":"2024-06-13T06:50:11.841267+0000","last_became_peered":"2024-06-13T06:50:11.841267+0000","last_unstale":"2024-06-13T06:55:25.174507+0000","last_undegraded":"2024-06-13T06:55:25.174507+0000","last_fullsized":"2024-06-13T06:55:25.174507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:17:48.595229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":452,"num_read_kb":294,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.612063+0000","last_change":"2024-06-13T06:50:16.486988+0000","last_active":"2024-06-13T06:53:29.612063+0000","last_peered":"2024-06-13T06:53:29.612063+0000","last_clean":"2024-06-13T06:53:29.612063+0000","last_became_active":"2024-06-13T06:50:16.486595+0000","last_became_peered":"2024-06-13T06:50:16.486595+0000","last_unstale":"2024-06-13T06:53:29.612063+0000","last_undegraded":"2024-06-13T06:53:29.612063+0000","last_fullsized":"2024-06-13T06:53:29.612063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:49:45.345913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:55.251812+0000","last_change":"2024-06-13T06:50:29.733985+0000","last_active":"2024-06-13T06:52:55.251812+0000","last_peered":"2024-06-13T06:52:55.251812+0000","last_clean":"2024-06-13T06:52:55.251812+0000","last_became_active":"2024-06-13T06:50:29.733492+0000","last_became_peered":"2024-06-13T06:50:29.733492+0000","last_unstale":"2024-06-13T06:52:55.251812+0000","last_undegraded":"2024-06-13T06:52:55.251812+0000","last_fullsized":"2024-06-13T06:52:55.251812+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:59:19.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.765822+0000","last_change":"2024-06-13T06:50:11.858693+0000","last_active":"2024-06-13T06:52:32.765822+0000","last_peered":"2024-06-13T06:52:32.765822+0000","last_clean":"2024-06-13T06:52:32.765822+0000","last_became_active":"2024-06-13T06:50:11.858279+0000","last_became_peered":"2024-06-13T06:50:11.858279+0000","last_unstale":"2024-06-13T06:52:32.765822+0000","last_undegraded":"2024-06-13T06:52:32.765822+0000","last_fullsized":"2024-06-13T06:52:32.765822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:35:02.366337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632382+0000","last_change":"2024-06-13T06:50:16.500708+0000","last_active":"2024-06-13T06:53:29.632382+0000","last_peered":"2024-06-13T06:53:29.632382+0000","last_clean":"2024-06-13T06:53:29.632382+0000","last_became_active":"2024-06-13T06:50:16.500306+0000","last_became_peered":"2024-06-13T06:50:16.500306+0000","last_unstale":"2024-06-13T06:53:29.632382+0000","last_undegraded":"2024-06-13T06:53:29.632382+0000","last_fullsized":"2024-06-13T06:53:29.632382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:11.941654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:51.110319+0000","last_change":"2024-06-13T06:50:29.739319+0000","last_active":"2024-06-13T06:52:51.110319+0000","last_peered":"2024-06-13T06:52:51.110319+0000","last_clean":"2024-06-13T06:52:51.110319+0000","last_became_active":"2024-06-13T06:50:29.738822+0000","last_became_peered":"2024-06-13T06:50:29.738822+0000","last_unstale":"2024-06-13T06:52:51.110319+0000","last_undegraded":"2024-06-13T06:52:51.110319+0000","last_fullsized":"2024-06-13T06:52:51.110319+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:52:09.907309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:35.249437+0000","last_change":"2024-06-13T06:50:11.856085+0000","last_active":"2024-06-13T06:52:35.249437+0000","last_peered":"2024-06-13T06:52:35.249437+0000","last_clean":"2024-06-13T06:52:35.249437+0000","last_became_active":"2024-06-13T06:50:11.855646+0000","last_became_peered":"2024-06-13T06:50:11.855646+0000","last_unstale":"2024-06-13T06:52:35.249437+0000","last_undegraded":"2024-06-13T06:52:35.249437+0000","last_fullsized":"2024-06-13T06:52:35.249437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:23:53.331035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"98'15","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633901+0000","last_change":"2024-06-13T06:50:16.488859+0000","last_active":"2024-06-13T06:53:29.633901+0000","last_peered":"2024-06-13T06:53:29.633901+0000","last_clean":"2024-06-13T06:53:29.633901+0000","last_became_active":"2024-06-13T06:50:16.488216+0000","last_became_peered":"2024-06-13T06:50:16.488216+0000","last_unstale":"2024-06-13T06:53:29.633901+0000","last_undegraded":"2024-06-13T06:53:29.633901+0000","last_fullsized":"2024-06-13T06:53:29.633901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:07:36.610662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:58.471021+0000","last_change":"2024-06-13T06:50:29.745854+0000","last_active":"2024-06-13T06:52:58.471021+0000","last_peered":"2024-06-13T06:52:58.471021+0000","last_clean":"2024-06-13T06:52:58.471021+0000","last_became_active":"2024-06-13T06:50:29.744068+0000","last_became_peered":"2024-06-13T06:50:29.744068+0000","last_unstale":"2024-06-13T06:52:58.471021+0000","last_undegraded":"2024-06-13T06:52:58.471021+0000","last_fullsized":"2024-06-13T06:52:58.471021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:15:26.242816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.326188+0000","last_change":"2024-06-13T06:50:11.843479+0000","last_active":"2024-06-13T06:55:20.326188+0000","last_peered":"2024-06-13T06:55:20.326188+0000","last_clean":"2024-06-13T06:55:20.326188+0000","last_became_active":"2024-06-13T06:50:11.843118+0000","last_became_peered":"2024-06-13T06:50:11.843118+0000","last_unstale":"2024-06-13T06:55:20.326188+0000","last_undegraded":"2024-06-13T06:55:20.326188+0000","last_fullsized":"2024-06-13T06:55:20.326188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:12:01.230706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1425,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633677+0000","last_change":"2024-06-13T06:50:16.480914+0000","last_active":"2024-06-13T06:53:29.633677+0000","last_peered":"2024-06-13T06:53:29.633677+0000","last_clean":"2024-06-13T06:53:29.633677+0000","last_became_active":"2024-06-13T06:50:16.480556+0000","last_became_peered":"2024-06-13T06:50:16.480556+0000","last_unstale":"2024-06-13T06:53:29.633677+0000","last_undegraded":"2024-06-13T06:53:29.633677+0000","last_fullsized":"2024-06-13T06:53:29.633677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:12:16.619869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:56.609813+0000","last_change":"2024-06-13T06:50:29.741034+0000","last_active":"2024-06-13T06:52:56.609813+0000","last_peered":"2024-06-13T06:52:56.609813+0000","last_clean":"2024-06-13T06:52:56.609813+0000","last_became_active":"2024-06-13T06:50:29.740647+0000","last_became_peered":"2024-06-13T06:50:29.740647+0000","last_unstale":"2024-06-13T06:52:56.609813+0000","last_undegraded":"2024-06-13T06:52:56.609813+0000","last_fullsized":"2024-06-13T06:52:56.609813+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:05:03.004321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:32.956450+0000","last_change":"2024-06-13T06:50:11.853590+0000","last_active":"2024-06-13T06:52:32.956450+0000","last_peered":"2024-06-13T06:52:32.956450+0000","last_clean":"2024-06-13T06:52:32.956450+0000","last_became_active":"2024-06-13T06:50:11.853231+0000","last_became_peered":"2024-06-13T06:50:11.853231+0000","last_unstale":"2024-06-13T06:52:32.956450+0000","last_undegraded":"2024-06-13T06:52:32.956450+0000","last_fullsized":"2024-06-13T06:52:32.956450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:08:40.629084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631663+0000","last_change":"2024-06-13T06:50:16.481595+0000","last_active":"2024-06-13T06:53:29.631663+0000","last_peered":"2024-06-13T06:53:29.631663+0000","last_clean":"2024-06-13T06:53:29.631663+0000","last_became_active":"2024-06-13T06:50:16.481247+0000","last_became_peered":"2024-06-13T06:50:16.481247+0000","last_unstale":"2024-06-13T06:53:29.631663+0000","last_undegraded":"2024-06-13T06:53:29.631663+0000","last_fullsized":"2024-06-13T06:53:29.631663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:20:00.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250034+0000","last_change":"2024-06-13T06:50:29.738452+0000","last_active":"2024-06-13T06:52:40.250034+0000","last_peered":"2024-06-13T06:52:40.250034+0000","last_clean":"2024-06-13T06:52:40.250034+0000","last_became_active":"2024-06-13T06:50:29.738005+0000","last_became_peered":"2024-06-13T06:50:29.738005+0000","last_unstale":"2024-06-13T06:52:40.250034+0000","last_undegraded":"2024-06-13T06:52:40.250034+0000","last_fullsized":"2024-06-13T06:52:40.250034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:16:24.735852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:42.970288+0000","last_change":"2024-06-13T06:50:11.843605+0000","last_active":"2024-06-13T06:54:42.970288+0000","last_peered":"2024-06-13T06:54:42.970288+0000","last_clean":"2024-06-13T06:54:42.970288+0000","last_became_active":"2024-06-13T06:50:11.842617+0000","last_became_peered":"2024-06-13T06:50:11.842617+0000","last_unstale":"2024-06-13T06:54:42.970288+0000","last_undegraded":"2024-06-13T06:54:42.970288+0000","last_fullsized":"2024-06-13T06:54:42.970288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:16:25.885675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.627895+0000","last_change":"2024-06-13T06:50:16.482722+0000","last_active":"2024-06-13T06:53:29.627895+0000","last_peered":"2024-06-13T06:53:29.627895+0000","last_clean":"2024-06-13T06:53:29.627895+0000","last_became_active":"2024-06-13T06:50:16.482356+0000","last_became_peered":"2024-06-13T06:50:16.482356+0000","last_unstale":"2024-06-13T06:53:29.627895+0000","last_undegraded":"2024-06-13T06:53:29.627895+0000","last_fullsized":"2024-06-13T06:53:29.627895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:39.013577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.957155+0000","last_change":"2024-06-13T06:50:29.737506+0000","last_active":"2024-06-13T06:52:37.957155+0000","last_peered":"2024-06-13T06:52:37.957155+0000","last_clean":"2024-06-13T06:52:37.957155+0000","last_became_active":"2024-06-13T06:50:29.737014+0000","last_became_peered":"2024-06-13T06:50:29.737014+0000","last_unstale":"2024-06-13T06:52:37.957155+0000","last_undegraded":"2024-06-13T06:52:37.957155+0000","last_fullsized":"2024-06-13T06:52:37.957155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:32:19.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:18.464620+0000","last_change":"2024-06-13T06:51:00.894992+0000","last_active":"2024-06-13T06:52:18.464620+0000","last_peered":"2024-06-13T06:52:18.464620+0000","last_clean":"2024-06-13T06:52:18.464620+0000","last_became_active":"2024-06-13T06:51:00.894278+0000","last_became_peered":"2024-06-13T06:51:00.894278+0000","last_unstale":"2024-06-13T06:52:18.464620+0000","last_undegraded":"2024-06-13T06:52:18.464620+0000","last_fullsized":"2024-06-13T06:52:18.464620+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:39:37.233367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636785+0000","last_change":"2024-06-13T06:50:16.498297+0000","last_active":"2024-06-13T06:53:29.636785+0000","last_peered":"2024-06-13T06:53:29.636785+0000","last_clean":"2024-06-13T06:53:29.636785+0000","last_became_active":"2024-06-13T06:50:16.498055+0000","last_became_peered":"2024-06-13T06:50:16.498055+0000","last_unstale":"2024-06-13T06:53:29.636785+0000","last_undegraded":"2024-06-13T06:53:29.636785+0000","last_fullsized":"2024-06-13T06:53:29.636785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:16:08.155925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.606649+0000","last_change":"2024-06-13T06:50:29.732562+0000","last_active":"2024-06-13T06:52:36.606649+0000","last_peered":"2024-06-13T06:52:36.606649+0000","last_clean":"2024-06-13T06:52:36.606649+0000","last_became_active":"2024-06-13T06:50:29.732055+0000","last_became_peered":"2024-06-13T06:50:29.732055+0000","last_unstale":"2024-06-13T06:52:36.606649+0000","last_undegraded":"2024-06-13T06:52:36.606649+0000","last_fullsized":"2024-06-13T06:52:36.606649+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:47:03.284547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:16.031317+0000","last_change":"2024-06-13T06:50:11.850406+0000","last_active":"2024-06-13T06:52:16.031317+0000","last_peered":"2024-06-13T06:52:16.031317+0000","last_clean":"2024-06-13T06:52:16.031317+0000","last_became_active":"2024-06-13T06:50:11.849896+0000","last_became_peered":"2024-06-13T06:50:11.849896+0000","last_unstale":"2024-06-13T06:52:16.031317+0000","last_undegraded":"2024-06-13T06:52:16.031317+0000","last_fullsized":"2024-06-13T06:52:16.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:42.184418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633623+0000","last_change":"2024-06-13T06:50:16.502472+0000","last_active":"2024-06-13T06:53:29.633623+0000","last_peered":"2024-06-13T06:53:29.633623+0000","last_clean":"2024-06-13T06:53:29.633623+0000","last_became_active":"2024-06-13T06:50:16.502086+0000","last_became_peered":"2024-06-13T06:50:16.502086+0000","last_unstale":"2024-06-13T06:53:29.633623+0000","last_undegraded":"2024-06-13T06:53:29.633623+0000","last_fullsized":"2024-06-13T06:53:29.633623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:27:49.564812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:41.607532+0000","last_change":"2024-06-13T06:50:29.731694+0000","last_active":"2024-06-13T06:52:41.607532+0000","last_peered":"2024-06-13T06:52:41.607532+0000","last_clean":"2024-06-13T06:52:41.607532+0000","last_became_active":"2024-06-13T06:50:29.731144+0000","last_became_peered":"2024-06-13T06:50:29.731144+0000","last_unstale":"2024-06-13T06:52:41.607532+0000","last_undegraded":"2024-06-13T06:52:41.607532+0000","last_fullsized":"2024-06-13T06:52:41.607532+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:44:10.717991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623878+0000","last_change":"2024-06-13T06:50:16.479841+0000","last_active":"2024-06-13T06:53:29.623878+0000","last_peered":"2024-06-13T06:53:29.623878+0000","last_clean":"2024-06-13T06:53:29.623878+0000","last_became_active":"2024-06-13T06:50:16.479525+0000","last_became_peered":"2024-06-13T06:50:16.479525+0000","last_unstale":"2024-06-13T06:53:29.623878+0000","last_undegraded":"2024-06-13T06:53:29.623878+0000","last_fullsized":"2024-06-13T06:53:29.623878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:30:47.760768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'6","reported_seq":264,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:55:20.324967+0000","last_change":"2024-06-13T06:50:11.862864+0000","last_active":"2024-06-13T06:55:20.324967+0000","last_peered":"2024-06-13T06:55:20.324967+0000","last_clean":"2024-06-13T06:55:20.324967+0000","last_became_active":"2024-06-13T06:50:11.861771+0000","last_became_peered":"2024-06-13T06:50:11.861771+0000","last_unstale":"2024-06-13T06:55:20.324967+0000","last_undegraded":"2024-06-13T06:55:20.324967+0000","last_fullsized":"2024-06-13T06:55:20.324967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:36:36.802924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.033518+0000","last_change":"2024-06-13T06:51:01.255734+0000","last_active":"2024-06-13T06:52:36.033518+0000","last_peered":"2024-06-13T06:52:36.033518+0000","last_clean":"2024-06-13T06:52:36.033518+0000","last_became_active":"2024-06-13T06:51:01.255169+0000","last_became_peered":"2024-06-13T06:51:01.255169+0000","last_unstale":"2024-06-13T06:52:36.033518+0000","last_undegraded":"2024-06-13T06:52:36.033518+0000","last_fullsized":"2024-06-13T06:52:36.033518+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:40:31.306920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.636741+0000","last_change":"2024-06-13T06:50:16.493422+0000","last_active":"2024-06-13T06:53:29.636741+0000","last_peered":"2024-06-13T06:53:29.636741+0000","last_clean":"2024-06-13T06:53:29.636741+0000","last_became_active":"2024-06-13T06:50:16.493008+0000","last_became_peered":"2024-06-13T06:50:16.493008+0000","last_unstale":"2024-06-13T06:53:29.636741+0000","last_undegraded":"2024-06-13T06:53:29.636741+0000","last_fullsized":"2024-06-13T06:53:29.636741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:54.291205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.082967+0000","last_change":"2024-06-13T06:50:11.856827+0000","last_active":"2024-06-13T06:52:17.082967+0000","last_peered":"2024-06-13T06:52:17.082967+0000","last_clean":"2024-06-13T06:52:17.082967+0000","last_became_active":"2024-06-13T06:50:11.856443+0000","last_became_peered":"2024-06-13T06:50:11.856443+0000","last_unstale":"2024-06-13T06:52:17.082967+0000","last_undegraded":"2024-06-13T06:52:17.082967+0000","last_fullsized":"2024-06-13T06:52:17.082967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:51:20.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":366,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.037470+0000","last_change":"2024-06-13T06:48:37.800620+0000","last_active":"2024-06-13T06:53:11.037470+0000","last_peered":"2024-06-13T06:53:11.037470+0000","last_clean":"2024-06-13T06:53:11.037470+0000","last_became_active":"2024-06-13T06:48:37.800074+0000","last_became_peered":"2024-06-13T06:48:37.800074+0000","last_unstale":"2024-06-13T06:53:11.037470+0000","last_undegraded":"2024-06-13T06:53:11.037470+0000","last_fullsized":"2024-06-13T06:53:11.037470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:44:44.177963+0000","last_clean_scrub_stamp":"2024-06-13T06:44:44.177963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:27:05.260945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.250077+0000","last_change":"2024-06-13T06:50:29.743388+0000","last_active":"2024-06-13T06:52:40.250077+0000","last_peered":"2024-06-13T06:52:40.250077+0000","last_clean":"2024-06-13T06:52:40.250077+0000","last_became_active":"2024-06-13T06:50:29.742965+0000","last_became_peered":"2024-06-13T06:50:29.742965+0000","last_unstale":"2024-06-13T06:52:40.250077+0000","last_undegraded":"2024-06-13T06:52:40.250077+0000","last_fullsized":"2024-06-13T06:52:40.250077+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:36:32.657599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631450+0000","last_change":"2024-06-13T06:50:16.481213+0000","last_active":"2024-06-13T06:53:29.631450+0000","last_peered":"2024-06-13T06:53:29.631450+0000","last_clean":"2024-06-13T06:53:29.631450+0000","last_became_active":"2024-06-13T06:50:16.480795+0000","last_became_peered":"2024-06-13T06:50:16.480795+0000","last_unstale":"2024-06-13T06:53:29.631450+0000","last_undegraded":"2024-06-13T06:53:29.631450+0000","last_fullsized":"2024-06-13T06:53:29.631450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:41:07.970596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:11.030897+0000","last_change":"2024-06-13T06:50:11.840870+0000","last_active":"2024-06-13T06:52:11.030897+0000","last_peered":"2024-06-13T06:52:11.030897+0000","last_clean":"2024-06-13T06:52:11.030897+0000","last_became_active":"2024-06-13T06:50:11.840419+0000","last_became_peered":"2024-06-13T06:50:11.840419+0000","last_unstale":"2024-06-13T06:52:11.030897+0000","last_undegraded":"2024-06-13T06:52:11.030897+0000","last_fullsized":"2024-06-13T06:52:11.030897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:57:34.527146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:36.108536+0000","last_change":"2024-06-13T06:51:01.257244+0000","last_active":"2024-06-13T06:52:36.108536+0000","last_peered":"2024-06-13T06:52:36.108536+0000","last_clean":"2024-06-13T06:52:36.108536+0000","last_became_active":"2024-06-13T06:51:01.256180+0000","last_became_peered":"2024-06-13T06:51:01.256180+0000","last_unstale":"2024-06-13T06:52:36.108536+0000","last_undegraded":"2024-06-13T06:52:36.108536+0000","last_fullsized":"2024-06-13T06:52:36.108536+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:26:28.018927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:17.955053+0000","last_change":"2024-06-13T06:50:11.854330+0000","last_active":"2024-06-13T06:52:17.955053+0000","last_peered":"2024-06-13T06:52:17.955053+0000","last_clean":"2024-06-13T06:52:17.955053+0000","last_became_active":"2024-06-13T06:50:11.853987+0000","last_became_peered":"2024-06-13T06:50:11.853987+0000","last_unstale":"2024-06-13T06:52:17.955053+0000","last_undegraded":"2024-06-13T06:52:17.955053+0000","last_fullsized":"2024-06-13T06:52:17.955053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:24:48.672935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628081+0000","last_change":"2024-06-13T06:50:16.485638+0000","last_active":"2024-06-13T06:53:29.628081+0000","last_peered":"2024-06-13T06:53:29.628081+0000","last_clean":"2024-06-13T06:53:29.628081+0000","last_became_active":"2024-06-13T06:50:16.485216+0000","last_became_peered":"2024-06-13T06:50:16.485216+0000","last_unstale":"2024-06-13T06:53:29.628081+0000","last_undegraded":"2024-06-13T06:53:29.628081+0000","last_fullsized":"2024-06-13T06:53:29.628081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:51:16.077178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.259402+0000","last_change":"2024-06-13T06:50:29.723935+0000","last_active":"2024-06-13T06:52:30.259402+0000","last_peered":"2024-06-13T06:52:30.259402+0000","last_clean":"2024-06-13T06:52:30.259402+0000","last_became_active":"2024-06-13T06:50:29.723443+0000","last_became_peered":"2024-06-13T06:50:29.723443+0000","last_unstale":"2024-06-13T06:52:30.259402+0000","last_undegraded":"2024-06-13T06:52:30.259402+0000","last_fullsized":"2024-06-13T06:52:30.259402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:08:57.489551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'13","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:54:33.486327+0000","last_change":"2024-06-13T06:50:11.865166+0000","last_active":"2024-06-13T06:54:33.486327+0000","last_peered":"2024-06-13T06:54:33.486327+0000","last_clean":"2024-06-13T06:54:33.486327+0000","last_became_active":"2024-06-13T06:50:11.864718+0000","last_became_peered":"2024-06-13T06:50:11.864718+0000","last_unstale":"2024-06-13T06:54:33.486327+0000","last_undegraded":"2024-06-13T06:54:33.486327+0000","last_fullsized":"2024-06-13T06:54:33.486327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:10:51.113709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"98'5","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.633370+0000","last_change":"2024-06-13T06:50:16.493068+0000","last_active":"2024-06-13T06:53:29.633370+0000","last_peered":"2024-06-13T06:53:29.633370+0000","last_clean":"2024-06-13T06:53:29.633370+0000","last_became_active":"2024-06-13T06:50:16.492483+0000","last_became_peered":"2024-06-13T06:50:16.492483+0000","last_unstale":"2024-06-13T06:53:29.633370+0000","last_undegraded":"2024-06-13T06:53:29.633370+0000","last_fullsized":"2024-06-13T06:53:29.633370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:45:22.160414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:37.766412+0000","last_change":"2024-06-13T06:50:29.715554+0000","last_active":"2024-06-13T06:52:37.766412+0000","last_peered":"2024-06-13T06:52:37.766412+0000","last_clean":"2024-06-13T06:52:37.766412+0000","last_became_active":"2024-06-13T06:50:29.715040+0000","last_became_peered":"2024-06-13T06:50:29.715040+0000","last_unstale":"2024-06-13T06:52:37.766412+0000","last_undegraded":"2024-06-13T06:52:37.766412+0000","last_fullsized":"2024-06-13T06:52:37.766412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:23:52.397722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:22.083460+0000","last_change":"2024-06-13T06:50:12.349150+0000","last_active":"2024-06-13T06:52:22.083460+0000","last_peered":"2024-06-13T06:52:22.083460+0000","last_clean":"2024-06-13T06:52:22.083460+0000","last_became_active":"2024-06-13T06:50:12.348784+0000","last_became_peered":"2024-06-13T06:50:12.348784+0000","last_unstale":"2024-06-13T06:52:22.083460+0000","last_undegraded":"2024-06-13T06:52:22.083460+0000","last_fullsized":"2024-06-13T06:52:22.083460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:03:11.041536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.635347+0000","last_change":"2024-06-13T06:50:16.505060+0000","last_active":"2024-06-13T06:53:29.635347+0000","last_peered":"2024-06-13T06:53:29.635347+0000","last_clean":"2024-06-13T06:53:29.635347+0000","last_became_active":"2024-06-13T06:50:16.504318+0000","last_became_peered":"2024-06-13T06:50:16.504318+0000","last_unstale":"2024-06-13T06:53:29.635347+0000","last_undegraded":"2024-06-13T06:53:29.635347+0000","last_fullsized":"2024-06-13T06:53:29.635347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T14:01:45.854607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086435+0000","last_change":"2024-06-13T06:51:01.256069+0000","last_active":"2024-06-13T06:52:47.086435+0000","last_peered":"2024-06-13T06:52:47.086435+0000","last_clean":"2024-06-13T06:52:47.086435+0000","last_became_active":"2024-06-13T06:51:01.255024+0000","last_became_peered":"2024-06-13T06:51:01.255024+0000","last_unstale":"2024-06-13T06:52:47.086435+0000","last_undegraded":"2024-06-13T06:52:47.086435+0000","last_fullsized":"2024-06-13T06:52:47.086435+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:54:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:24.165534+0000","last_change":"2024-06-13T06:50:11.853461+0000","last_active":"2024-06-13T06:52:24.165534+0000","last_peered":"2024-06-13T06:52:24.165534+0000","last_clean":"2024-06-13T06:52:24.165534+0000","last_became_active":"2024-06-13T06:50:11.853136+0000","last_became_peered":"2024-06-13T06:50:11.853136+0000","last_unstale":"2024-06-13T06:52:24.165534+0000","last_undegraded":"2024-06-13T06:52:24.165534+0000","last_fullsized":"2024-06-13T06:52:24.165534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:15:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.620463+0000","last_change":"2024-06-13T06:50:16.478374+0000","last_active":"2024-06-13T06:53:29.620463+0000","last_peered":"2024-06-13T06:53:29.620463+0000","last_clean":"2024-06-13T06:53:29.620463+0000","last_became_active":"2024-06-13T06:50:16.477984+0000","last_became_peered":"2024-06-13T06:50:16.477984+0000","last_unstale":"2024-06-13T06:53:29.620463+0000","last_undegraded":"2024-06-13T06:53:29.620463+0000","last_fullsized":"2024-06-13T06:53:29.620463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:52:49.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:48.469183+0000","last_change":"2024-06-13T06:50:29.746140+0000","last_active":"2024-06-13T06:52:48.469183+0000","last_peered":"2024-06-13T06:52:48.469183+0000","last_clean":"2024-06-13T06:52:48.469183+0000","last_became_active":"2024-06-13T06:50:29.744721+0000","last_became_peered":"2024-06-13T06:50:29.744721+0000","last_unstale":"2024-06-13T06:52:48.469183+0000","last_undegraded":"2024-06-13T06:52:48.469183+0000","last_fullsized":"2024-06-13T06:52:48.469183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:36:27.622703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:25.258413+0000","last_change":"2024-06-13T06:50:11.854139+0000","last_active":"2024-06-13T06:52:25.258413+0000","last_peered":"2024-06-13T06:52:25.258413+0000","last_clean":"2024-06-13T06:52:25.258413+0000","last_became_active":"2024-06-13T06:50:11.853668+0000","last_became_peered":"2024-06-13T06:50:11.853668+0000","last_unstale":"2024-06-13T06:52:25.258413+0000","last_undegraded":"2024-06-13T06:52:25.258413+0000","last_fullsized":"2024-06-13T06:52:25.258413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:54:30.776451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.625157+0000","last_change":"2024-06-13T06:50:16.479840+0000","last_active":"2024-06-13T06:53:29.625157+0000","last_peered":"2024-06-13T06:53:29.625157+0000","last_clean":"2024-06-13T06:53:29.625157+0000","last_became_active":"2024-06-13T06:50:16.479404+0000","last_became_peered":"2024-06-13T06:50:16.479404+0000","last_unstale":"2024-06-13T06:53:29.625157+0000","last_undegraded":"2024-06-13T06:53:29.625157+0000","last_fullsized":"2024-06-13T06:53:29.625157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:55:48.267761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":27,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086403+0000","last_change":"2024-06-13T06:50:29.740517+0000","last_active":"2024-06-13T06:52:47.086403+0000","last_peered":"2024-06-13T06:52:47.086403+0000","last_clean":"2024-06-13T06:52:47.086403+0000","last_became_active":"2024-06-13T06:50:29.739910+0000","last_became_peered":"2024-06-13T06:50:29.739910+0000","last_unstale":"2024-06-13T06:52:47.086403+0000","last_undegraded":"2024-06-13T06:52:47.086403+0000","last_fullsized":"2024-06-13T06:52:47.086403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:24:19.125105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:11.112337+0000","last_change":"2024-06-13T06:51:01.256598+0000","last_active":"2024-06-13T06:53:11.112337+0000","last_peered":"2024-06-13T06:53:11.112337+0000","last_clean":"2024-06-13T06:53:11.112337+0000","last_became_active":"2024-06-13T06:51:01.255344+0000","last_became_peered":"2024-06-13T06:51:01.255344+0000","last_unstale":"2024-06-13T06:53:11.112337+0000","last_undegraded":"2024-06-13T06:53:11.112337+0000","last_fullsized":"2024-06-13T06:53:11.112337+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:01:47.892644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.639239+0000","last_change":"2024-06-13T06:50:16.505369+0000","last_active":"2024-06-13T06:53:29.639239+0000","last_peered":"2024-06-13T06:53:29.639239+0000","last_clean":"2024-06-13T06:53:29.639239+0000","last_became_active":"2024-06-13T06:50:16.504724+0000","last_became_peered":"2024-06-13T06:50:16.504724+0000","last_unstale":"2024-06-13T06:53:29.639239+0000","last_undegraded":"2024-06-13T06:53:29.639239+0000","last_fullsized":"2024-06-13T06:53:29.639239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:09:38.092069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:47.086424+0000","last_change":"2024-06-13T06:50:29.722380+0000","last_active":"2024-06-13T06:52:47.086424+0000","last_peered":"2024-06-13T06:52:47.086424+0000","last_clean":"2024-06-13T06:52:47.086424+0000","last_became_active":"2024-06-13T06:50:29.721880+0000","last_became_peered":"2024-06-13T06:50:29.721880+0000","last_unstale":"2024-06-13T06:52:47.086424+0000","last_undegraded":"2024-06-13T06:52:47.086424+0000","last_fullsized":"2024-06-13T06:52:47.086424+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:09:44.153139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.955951+0000","last_change":"2024-06-13T06:50:11.842417+0000","last_active":"2024-06-13T06:52:27.955951+0000","last_peered":"2024-06-13T06:52:27.955951+0000","last_clean":"2024-06-13T06:52:27.955951+0000","last_became_active":"2024-06-13T06:50:11.841503+0000","last_became_peered":"2024-06-13T06:50:11.841503+0000","last_unstale":"2024-06-13T06:52:27.955951+0000","last_undegraded":"2024-06-13T06:52:27.955951+0000","last_fullsized":"2024-06-13T06:52:27.955951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:11:41.282488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"98'5","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631970+0000","last_change":"2024-06-13T06:50:16.481310+0000","last_active":"2024-06-13T06:53:29.631970+0000","last_peered":"2024-06-13T06:53:29.631970+0000","last_clean":"2024-06-13T06:53:29.631970+0000","last_became_active":"2024-06-13T06:50:16.480983+0000","last_became_peered":"2024-06-13T06:50:16.480983+0000","last_unstale":"2024-06-13T06:53:29.631970+0000","last_undegraded":"2024-06-13T06:53:29.631970+0000","last_fullsized":"2024-06-13T06:53:29.631970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:25:22.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:39.167112+0000","last_change":"2024-06-13T06:50:29.731252+0000","last_active":"2024-06-13T06:52:39.167112+0000","last_peered":"2024-06-13T06:52:39.167112+0000","last_clean":"2024-06-13T06:52:39.167112+0000","last_became_active":"2024-06-13T06:50:29.730851+0000","last_became_peered":"2024-06-13T06:50:29.730851+0000","last_unstale":"2024-06-13T06:52:39.167112+0000","last_undegraded":"2024-06-13T06:52:39.167112+0000","last_fullsized":"2024-06-13T06:52:39.167112+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:33:02.774613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:27.765245+0000","last_change":"2024-06-13T06:50:12.348969+0000","last_active":"2024-06-13T06:52:27.765245+0000","last_peered":"2024-06-13T06:52:27.765245+0000","last_clean":"2024-06-13T06:52:27.765245+0000","last_became_active":"2024-06-13T06:50:12.348507+0000","last_became_peered":"2024-06-13T06:50:12.348507+0000","last_unstale":"2024-06-13T06:52:27.765245+0000","last_undegraded":"2024-06-13T06:52:27.765245+0000","last_fullsized":"2024-06-13T06:52:27.765245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T11:06:44.262679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"98'7","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632700+0000","last_change":"2024-06-13T06:50:16.482341+0000","last_active":"2024-06-13T06:53:29.632700+0000","last_peered":"2024-06-13T06:53:29.632700+0000","last_clean":"2024-06-13T06:53:29.632700+0000","last_became_active":"2024-06-13T06:50:16.481508+0000","last_became_peered":"2024-06-13T06:50:16.481508+0000","last_unstale":"2024-06-13T06:53:29.632700+0000","last_undegraded":"2024-06-13T06:53:29.632700+0000","last_fullsized":"2024-06-13T06:53:29.632700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:29.886862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:44.822664+0000","last_change":"2024-06-13T06:51:01.257334+0000","last_active":"2024-06-13T06:52:44.822664+0000","last_peered":"2024-06-13T06:52:44.822664+0000","last_clean":"2024-06-13T06:52:44.822664+0000","last_became_active":"2024-06-13T06:51:01.255952+0000","last_became_peered":"2024-06-13T06:51:01.255952+0000","last_unstale":"2024-06-13T06:52:44.822664+0000","last_undegraded":"2024-06-13T06:52:44.822664+0000","last_fullsized":"2024-06-13T06:52:44.822664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:32:12.414560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:30.248910+0000","last_change":"2024-06-13T06:50:11.858081+0000","last_active":"2024-06-13T06:52:30.248910+0000","last_peered":"2024-06-13T06:52:30.248910+0000","last_clean":"2024-06-13T06:52:30.248910+0000","last_became_active":"2024-06-13T06:50:11.857684+0000","last_became_peered":"2024-06-13T06:50:11.857684+0000","last_unstale":"2024-06-13T06:52:30.248910+0000","last_undegraded":"2024-06-13T06:52:30.248910+0000","last_fullsized":"2024-06-13T06:52:30.248910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:32:13.152223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"98'6","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629774+0000","last_change":"2024-06-13T06:50:16.495322+0000","last_active":"2024-06-13T06:53:29.629774+0000","last_peered":"2024-06-13T06:53:29.629774+0000","last_clean":"2024-06-13T06:53:29.629774+0000","last_became_active":"2024-06-13T06:50:16.495021+0000","last_became_peered":"2024-06-13T06:50:16.495021+0000","last_unstale":"2024-06-13T06:53:29.629774+0000","last_undegraded":"2024-06-13T06:53:29.629774+0000","last_fullsized":"2024-06-13T06:53:29.629774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:17:48.288420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:40.249974+0000","last_change":"2024-06-13T06:50:29.718734+0000","last_active":"2024-06-13T06:52:40.249974+0000","last_peered":"2024-06-13T06:52:40.249974+0000","last_clean":"2024-06-13T06:52:40.249974+0000","last_became_active":"2024-06-13T06:50:29.717916+0000","last_became_peered":"2024-06-13T06:50:29.717916+0000","last_unstale":"2024-06-13T06:52:40.249974+0000","last_undegraded":"2024-06-13T06:52:40.249974+0000","last_fullsized":"2024-06-13T06:52:40.249974+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:28.678060+0000","last_clean_scrub_stamp":"2024-06-13T06:50:28.678060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:56:14.259095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:52:26.605224+0000","last_change":"2024-06-13T06:50:11.867057+0000","last_active":"2024-06-13T06:52:26.605224+0000","last_peered":"2024-06-13T06:52:26.605224+0000","last_clean":"2024-06-13T06:52:26.605224+0000","last_became_active":"2024-06-13T06:50:11.866651+0000","last_became_peered":"2024-06-13T06:50:11.866651+0000","last_unstale":"2024-06-13T06:52:26.605224+0000","last_undegraded":"2024-06-13T06:52:26.605224+0000","last_fullsized":"2024-06-13T06:52:26.605224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:10.821383+0000","last_clean_scrub_stamp":"2024-06-13T06:50:10.821383+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T09:39:33.573285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"98'9","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.621192+0000","last_change":"2024-06-13T06:50:16.483669+0000","last_active":"2024-06-13T06:53:29.621192+0000","last_peered":"2024-06-13T06:53:29.621192+0000","last_clean":"2024-06-13T06:53:29.621192+0000","last_became_active":"2024-06-13T06:50:16.482669+0000","last_became_peered":"2024-06-13T06:50:16.482669+0000","last_unstale":"2024-06-13T06:53:29.621192+0000","last_undegraded":"2024-06-13T06:53:29.621192+0000","last_fullsized":"2024-06-13T06:53:29.621192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:43:36.667513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"98'9","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.623559+0000","last_change":"2024-06-13T06:50:16.479011+0000","last_active":"2024-06-13T06:53:29.623559+0000","last_peered":"2024-06-13T06:53:29.623559+0000","last_clean":"2024-06-13T06:53:29.623559+0000","last_became_active":"2024-06-13T06:50:16.478605+0000","last_became_peered":"2024-06-13T06:50:16.478605+0000","last_unstale":"2024-06-13T06:53:29.623559+0000","last_undegraded":"2024-06-13T06:53:29.623559+0000","last_fullsized":"2024-06-13T06:53:29.623559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T08:44:09.182323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"98'12","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.624643+0000","last_change":"2024-06-13T06:50:16.480486+0000","last_active":"2024-06-13T06:53:29.624643+0000","last_peered":"2024-06-13T06:53:29.624643+0000","last_clean":"2024-06-13T06:53:29.624643+0000","last_became_active":"2024-06-13T06:50:16.480103+0000","last_became_peered":"2024-06-13T06:50:16.480103+0000","last_unstale":"2024-06-13T06:53:29.624643+0000","last_undegraded":"2024-06-13T06:53:29.624643+0000","last_fullsized":"2024-06-13T06:53:29.624643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:26:39.721347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629702+0000","last_change":"2024-06-13T06:50:16.500156+0000","last_active":"2024-06-13T06:53:29.629702+0000","last_peered":"2024-06-13T06:53:29.629702+0000","last_clean":"2024-06-13T06:53:29.629702+0000","last_became_active":"2024-06-13T06:50:16.499794+0000","last_became_peered":"2024-06-13T06:50:16.499794+0000","last_unstale":"2024-06-13T06:53:29.629702+0000","last_undegraded":"2024-06-13T06:53:29.629702+0000","last_fullsized":"2024-06-13T06:53:29.629702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T13:15:39.154907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"98'7","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.638325+0000","last_change":"2024-06-13T06:50:16.495229+0000","last_active":"2024-06-13T06:53:29.638325+0000","last_peered":"2024-06-13T06:53:29.638325+0000","last_clean":"2024-06-13T06:53:29.638325+0000","last_became_active":"2024-06-13T06:50:16.485897+0000","last_became_peered":"2024-06-13T06:50:16.485897+0000","last_unstale":"2024-06-13T06:53:29.638325+0000","last_undegraded":"2024-06-13T06:53:29.638325+0000","last_fullsized":"2024-06-13T06:53:29.638325+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T18:04:26.504704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"98'4","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.631155+0000","last_change":"2024-06-13T06:50:16.480772+0000","last_active":"2024-06-13T06:53:29.631155+0000","last_peered":"2024-06-13T06:53:29.631155+0000","last_clean":"2024-06-13T06:53:29.631155+0000","last_became_active":"2024-06-13T06:50:16.480327+0000","last_became_peered":"2024-06-13T06:50:16.480327+0000","last_unstale":"2024-06-13T06:53:29.631155+0000","last_undegraded":"2024-06-13T06:53:29.631155+0000","last_fullsized":"2024-06-13T06:53:29.631155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:22:21.950462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.617420+0000","last_change":"2024-06-13T06:51:01.257598+0000","last_active":"2024-06-13T06:53:29.617420+0000","last_peered":"2024-06-13T06:53:29.617420+0000","last_clean":"2024-06-13T06:53:29.617420+0000","last_became_active":"2024-06-13T06:51:01.256652+0000","last_became_peered":"2024-06-13T06:51:01.256652+0000","last_unstale":"2024-06-13T06:53:29.617420+0000","last_undegraded":"2024-06-13T06:53:29.617420+0000","last_fullsized":"2024-06-13T06:53:29.617420+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T16:07:07.897154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"98'15","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.630989+0000","last_change":"2024-06-13T06:50:16.477619+0000","last_active":"2024-06-13T06:53:29.630989+0000","last_peered":"2024-06-13T06:53:29.630989+0000","last_clean":"2024-06-13T06:53:29.630989+0000","last_became_active":"2024-06-13T06:50:16.477237+0000","last_became_peered":"2024-06-13T06:50:16.477237+0000","last_unstale":"2024-06-13T06:53:29.630989+0000","last_undegraded":"2024-06-13T06:53:29.630989+0000","last_fullsized":"2024-06-13T06:53:29.630989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T17:35:10.336538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"98'2","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.629961+0000","last_change":"2024-06-13T06:50:16.493910+0000","last_active":"2024-06-13T06:53:29.629961+0000","last_peered":"2024-06-13T06:53:29.629961+0000","last_clean":"2024-06-13T06:53:29.629961+0000","last_became_active":"2024-06-13T06:50:16.493058+0000","last_became_peered":"2024-06-13T06:50:16.493058+0000","last_unstale":"2024-06-13T06:53:29.629961+0000","last_undegraded":"2024-06-13T06:53:29.629961+0000","last_fullsized":"2024-06-13T06:53:29.629961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:43:30.774262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"98'6","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640037+0000","last_change":"2024-06-13T06:50:16.478392+0000","last_active":"2024-06-13T06:53:29.640037+0000","last_peered":"2024-06-13T06:53:29.640037+0000","last_clean":"2024-06-13T06:53:29.640037+0000","last_became_active":"2024-06-13T06:50:16.478060+0000","last_became_peered":"2024-06-13T06:50:16.478060+0000","last_unstale":"2024-06-13T06:53:29.640037+0000","last_undegraded":"2024-06-13T06:53:29.640037+0000","last_fullsized":"2024-06-13T06:53:29.640037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:09:15.734445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632710+0000","last_change":"2024-06-13T06:50:16.489112+0000","last_active":"2024-06-13T06:53:29.632710+0000","last_peered":"2024-06-13T06:53:29.632710+0000","last_clean":"2024-06-13T06:53:29.632710+0000","last_became_active":"2024-06-13T06:50:16.488736+0000","last_became_peered":"2024-06-13T06:50:16.488736+0000","last_unstale":"2024-06-13T06:53:29.632710+0000","last_undegraded":"2024-06-13T06:53:29.632710+0000","last_fullsized":"2024-06-13T06:53:29.632710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T07:50:53.629146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"98'10","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.640563+0000","last_change":"2024-06-13T06:50:16.499884+0000","last_active":"2024-06-13T06:53:29.640563+0000","last_peered":"2024-06-13T06:53:29.640563+0000","last_clean":"2024-06-13T06:53:29.640563+0000","last_became_active":"2024-06-13T06:50:16.499573+0000","last_became_peered":"2024-06-13T06:50:16.499573+0000","last_unstale":"2024-06-13T06:53:29.640563+0000","last_undegraded":"2024-06-13T06:53:29.640563+0000","last_fullsized":"2024-06-13T06:53:29.640563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T10:01:41.703761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"98'11","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.632622+0000","last_change":"2024-06-13T06:50:16.488215+0000","last_active":"2024-06-13T06:53:29.632622+0000","last_peered":"2024-06-13T06:53:29.632622+0000","last_clean":"2024-06-13T06:53:29.632622+0000","last_became_active":"2024-06-13T06:50:16.487825+0000","last_became_peered":"2024-06-13T06:50:16.487825+0000","last_unstale":"2024-06-13T06:53:29.632622+0000","last_undegraded":"2024-06-13T06:53:29.632622+0000","last_fullsized":"2024-06-13T06:53:29.632622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T15:25:52.247153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"98'8","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-13T06:53:29.628373+0000","last_change":"2024-06-13T06:50:16.477640+0000","last_active":"2024-06-13T06:53:29.628373+0000","last_peered":"2024-06-13T06:53:29.628373+0000","last_clean":"2024-06-13T06:53:29.628373+0000","last_became_active":"2024-06-13T06:50:16.477209+0000","last_became_peered":"2024-06-13T06:50:16.477209+0000","last_unstale":"2024-06-13T06:53:29.628373+0000","last_undegraded":"2024-06-13T06:53:29.628373+0000","last_fullsized":"2024-06-13T06:53:29.628373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-13T06:50:15.452700+0000","last_clean_scrub_stamp":"2024-06-13T06:50:15.452700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-14T12:56:31.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284256,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73374,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1345,"num_read_kb":751,"num_write":185,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":220122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351055,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553947,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612693,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32380,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712772,"statfs":{"total":95995035648,"available":95961878528,"internally_reserved":0,"allocated":1232896,"data_stored":1215075,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776217,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":554183,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554103,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299493,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150458,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463019,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549359,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":618196,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738509,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550747,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1171456,"data_stored":619546,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149820,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":549553,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":554846,"data_compressed":26574,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724659,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145209,"data_compressed":62721,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34507,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36147,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-13T06:55:26.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:55:26.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:26 smithi189 ceph-mon[36616]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:26.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:26 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:26.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:26 smithi175 ceph-mon[36886]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:26.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:26 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:26.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:26 smithi007 ceph-mon[32077]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:26.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:26 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:55:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:27.518 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.519 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:27.519 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.519 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:27.519 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:27.519 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:27 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:27 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:55:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-13T06:55:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:55:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:55:27.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 -- 172.21.15.7:0/4002197417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 msgr2=0x7f6cd0108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 --2- 172.21.15.7:0/4002197417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd0108f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009990 tx=0x7f6cbc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 -- 172.21.15.7:0/4002197417 shutdown_connections 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 --2- 172.21.15.7:0/4002197417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 --2- 172.21.15.7:0/4002197417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 0x7f6cd010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 --2- 172.21.15.7:0/4002197417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd0108f40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 -- 172.21.15.7:0/4002197417 >> 172.21.15.7:0/4002197417 conn(0x7f6cd0075c60 msgr2=0x7f6cd01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 -- 172.21.15.7:0/4002197417 shutdown_connections 2024-06-13T06:55:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.179+0000 7f6cd6000700 1 -- 172.21.15.7:0/4002197417 wait complete. 2024-06-13T06:55:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 Processor -- start 2024-06-13T06:55:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 -- start start 2024-06-13T06:55:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 0x7f6cd01a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0118e50 con 0x7f6cd0109480 2024-06-13T06:55:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0118cd0 con 0x7f6cd010eac0 2024-06-13T06:55:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.181+0000 7f6cd6000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0118fd0 con 0x7f6cd0108ac0 2024-06-13T06:55:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccf400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39500/0 (socket says 172.21.15.7:39500) 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 -- 172.21.15.7:0/3684163205 learned_addr learned my addr 172.21.15.7:0/3684163205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccf400700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd019f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48924/0 (socket says 172.21.15.7:48924) 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccfe00700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 0x7f6cd01a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 -- 172.21.15.7:0/3684163205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 msgr2=0x7f6cd019f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd019f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 -- 172.21.15.7:0/3684163205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 msgr2=0x7f6cd01a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 0x7f6cd01a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.182+0000 7f6ccea00700 1 -- 172.21.15.7:0/3684163205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd01a4210 con 0x7f6cd0109480 2024-06-13T06:55:29.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.183+0000 7f6ccea00700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.183+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc027070 con 0x7f6cd0109480 2024-06-13T06:55:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.183+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd01a44a0 con 0x7f6cd0109480 2024-06-13T06:55:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.183+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cbc003d70 con 0x7f6cd0109480 2024-06-13T06:55:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.183+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01abd80 con 0x7f6cd0109480 2024-06-13T06:55:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.184+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc031060 con 0x7f6cd0109480 2024-06-13T06:55:29.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.184+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd0069180 con 0x7f6cd0109480 2024-06-13T06:55:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.187+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6cbc02a070 con 0x7f6cd0109480 2024-06-13T06:55:29.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.188+0000 7f6cc7e00700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 0x7f6cb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:29.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.188+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6cbc0d04a0 con 0x7f6cd0109480 2024-06-13T06:55:29.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.188+0000 7f6ccf400700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 0x7f6cb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.189+0000 7f6ccf400700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 0x7f6cb40a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009680 tx=0x7f6cc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:29.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.191+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6cbc013070 con 0x7f6cd0109480 2024-06-13T06:55:29.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.477+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cd002dca0 con 0x7f6cd0109480 2024-06-13T06:55:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.479+0000 7f6cc7e00700 1 -- 172.21.15.7:0/3684163205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11868 (secure 0 0 0) 0x7f6cbc09d1b0 con 0x7f6cd0109480 2024-06-13T06:55:29.479 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:29.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:55:17.483515+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T06:55:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 msgr2=0x7f6cb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 0x7f6cb40a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009680 tx=0x7f6cc0009260 comp rx=0 tx=0).stop 2024-06-13T06:55:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 msgr2=0x7f6cd019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc00bab0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 shutdown_connections 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6cb40a2c20 0x7f6cb40a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd010eac0 0x7f6cd01a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0108ac0 0x7f6cd019f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 --2- 172.21.15.7:0/3684163205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cd0109480 0x7f6cd019f740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.482+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 >> 172.21.15.7:0/3684163205 conn(0x7f6cd0075c60 msgr2=0x7f6cd0076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.483+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 shutdown_connections 2024-06-13T06:55:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:29.483+0000 7f6cd6000700 1 -- 172.21.15.7:0/3684163205 wait complete. 2024-06-13T06:55:29.484 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-06-13T06:55:29.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:29 smithi007 ceph-mon[32077]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:29 smithi189 ceph-mon[36616]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:29 smithi175 ceph-mon[36886]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:30.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T06:55:30.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3684163205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:30.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3684163205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:30.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3684163205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:55:30.835 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:31 smithi189 ceph-mon[36616]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-13T06:55:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:31 smithi175 ceph-mon[36886]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-13T06:55:31.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:31 smithi007 ceph-mon[32077]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 -- 172.21.15.7:0/2246102805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57241046b0 msgr2=0x7f5724104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 --2- 172.21.15.7:0/2246102805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57241046b0 0x7f5724104b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 -- 172.21.15.7:0/2246102805 shutdown_connections 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 --2- 172.21.15.7:0/2246102805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5724105070 0x7f5724111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 --2- 172.21.15.7:0/2246102805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 --2- 172.21.15.7:0/2246102805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57241046b0 0x7f5724104b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.158+0000 7f572c800700 1 -- 172.21.15.7:0/2246102805 >> 172.21.15.7:0/2246102805 conn(0x7f57241001a0 msgr2=0x7f57241025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.159+0000 7f572c800700 1 -- 172.21.15.7:0/2246102805 shutdown_connections 2024-06-13T06:55:33.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.159+0000 7f572c800700 1 -- 172.21.15.7:0/2246102805 wait complete. 2024-06-13T06:55:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 Processor -- start 2024-06-13T06:55:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 -- start start 2024-06-13T06:55:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57241046b0 0x7f572419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5724114a50 con 0x7f5724105070 2024-06-13T06:55:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57241148d0 con 0x7f572410a6b0 2024-06-13T06:55:33.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.160+0000 7f572c800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5724114bd0 con 0x7f57241046b0 2024-06-13T06:55:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33016/0 (socket says 172.21.15.7:33016) 2024-06-13T06:55:33.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 -- 172.21.15.7:0/539721289 learned_addr learned my addr 172.21.15.7:0/539721289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729e00700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57241046b0 0x7f572419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f572a800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 -- 172.21.15.7:0/539721289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57241046b0 msgr2=0x7f572419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57241046b0 0x7f572419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 -- 172.21.15.7:0/539721289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 msgr2=0x7f572419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f5729400700 1 -- 172.21.15.7:0/539721289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572419fe50 con 0x7f5724105070 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.161+0000 7f572a800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f5729400700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5714005910 tx=0x7f571400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714027070 con 0x7f5724105070 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5714004650 con 0x7f5724105070 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a00e0 con 0x7f5724105070 2024-06-13T06:55:33.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400f460 con 0x7f5724105070 2024-06-13T06:55:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.162+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a79c0 con 0x7f5724105070 2024-06-13T06:55:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.164+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5724069180 con 0x7f5724105070 2024-06-13T06:55:33.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.166+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f571402a070 con 0x7f5724105070 2024-06-13T06:55:33.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.168+0000 7f571ea00700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 0x7f570c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:33.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.168+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f57140d0650 con 0x7f5724105070 2024-06-13T06:55:33.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.168+0000 7f5729e00700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 0x7f570c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:33.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.169+0000 7f5729e00700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 0x7f570c0a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5718009be0 tx=0x7f5718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:33.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.171+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5714031070 con 0x7f5724105070 2024-06-13T06:55:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.405+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f572402dca0 con 0x7f5724105070 2024-06-13T06:55:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.408+0000 7f571ea00700 1 -- 172.21.15.7:0/539721289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f5714013070 con 0x7f5724105070 2024-06-13T06:55:33.409 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:33.409 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T06:55:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 msgr2=0x7f570c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 0x7f570c0a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5718009be0 tx=0x7f5718009260 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 msgr2=0x7f572419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5714005910 tx=0x7f571400bab0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 shutdown_connections 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f570c0a2ca0 0x7f570c0a5160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f572410a6b0 0x7f572419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57241046b0 0x7f572419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 --2- 172.21.15.7:0/539721289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5724105070 0x7f572419b380 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.411+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 >> 172.21.15.7:0/539721289 conn(0x7f57241001a0 msgr2=0x7f57241006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.412+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 shutdown_connections 2024-06-13T06:55:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:33.412+0000 7f572c800700 1 -- 172.21.15.7:0/539721289 wait complete. 2024-06-13T06:55:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:33 smithi189 ceph-mon[36616]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T06:55:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:33 smithi175 ceph-mon[36886]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T06:55:33.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:33 smithi007 ceph-mon[32077]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-13T06:55:34.313 DEBUG:tasks.fwd_scrub:Yielding 2024-06-13T06:55:34.313 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-13T06:55:34.324 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:55:34.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-13T06:55:34.325 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-13T06:55:34.325 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T06:55:34.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T06:55:34.332 INFO:teuthology.run_tasks:Running task exec... 2024-06-13T06:55:34.369 INFO:teuthology.task.exec:Executing custom commands... 2024-06-13T06:55:34.369 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T06:55:34.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-06-13T06:55:34.460 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:34.482 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:34 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/539721289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:34.495 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem Size Used Avail Use% Mounted on 2024-06-13T06:55:34.495 INFO:teuthology.orchestra.run.smithi007.stdout:0@b9295806-294f-11ef-bc9c-c7b262605968.cephfs=/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 1.1T 372M 1.1T 1% /tmp/cephfs 2024-06-13T06:55:34.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-06-13T06:55:34.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:34 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/539721289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:34.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:34 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/539721289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T06:55:34.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-06-13T06:55:34.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-06-13T06:55:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:35.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:35 smithi007 ceph-mon[32077]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:35 smithi189 ceph-mon[36616]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:35 smithi175 ceph-mon[36886]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-13T06:55:37.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-06-13T06:55:37.232 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-06-13T06:55:37.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-06-13T06:55:37.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-13T06:55:37.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:37 smithi007 ceph-mon[32077]: pgmap v496: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 data 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:37.456 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-06-13T06:55:37.457 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:37.457 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:37.457 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T06:55:37.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-06-13T06:55:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:37 smithi189 ceph-mon[36616]: pgmap v496: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:37 smithi175 ceph-mon[36886]: pgmap v496: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-13T06:55:37.645 INFO:teuthology.orchestra.run.smithi007.stdout:[Service] 2024-06-13T06:55:37.645 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-06-13T06:55:37.645 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-06-13T06:55:37.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-06-13T06:55:37.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 -- 172.21.15.7:0/4004519138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0102740 msgr2=0x7fe3d0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/4004519138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0102740 0x7fe3d0102b20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009990 tx=0x7fe3c4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 -- 172.21.15.7:0/4004519138 shutdown_connections 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/4004519138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0107840 0x7fe3d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/4004519138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0102740 0x7fe3d0102b20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/4004519138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 -- 172.21.15.7:0/4004519138 >> 172.21.15.7:0/4004519138 conn(0x7fe3d00795f0 msgr2=0x7fe3d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 -- 172.21.15.7:0/4004519138 shutdown_connections 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.700+0000 7fe3d7a00700 1 -- 172.21.15.7:0/4004519138 wait complete. 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.701+0000 7fe3d7a00700 1 Processor -- start 2024-06-13T06:55:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.701+0000 7fe3d7a00700 1 -- start start 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 0x7fe3d0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 0x7fe3d0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0112760 con 0x7fe3d0102740 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01125e0 con 0x7fe3d01030f0 2024-06-13T06:55:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d7a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01128e0 con 0x7fe3d0107840 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47818/0 (socket says 172.21.15.7:47818) 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3cfe00700 1 -- 172.21.15.7:0/510780633 learned_addr learned my addr 172.21.15.7:0/510780633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 0x7fe3d0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.702+0000 7fe3d5000700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 0x7fe3d0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 -- 172.21.15.7:0/510780633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 msgr2=0x7fe3d0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 -- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 msgr2=0x7fe3d0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 0x7fe3d0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 -- 172.21.15.7:0/510780633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d01a70f0 con 0x7fe3d0107840 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5000700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 0x7fe3d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.703+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 0x7fe3d0075210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800efc0 tx=0x7fe3c800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.704+0000 7fe3cd600700 1 -- 172.21.15.7:0/510780633 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80092e0 con 0x7fe3d0107840 2024-06-13T06:55:37.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.704+0000 7fe3cd600700 1 -- 172.21.15.7:0/510780633 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c8004bb0 con 0x7fe3d0107840 2024-06-13T06:55:37.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.704+0000 7fe3cd600700 1 -- 172.21.15.7:0/510780633 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004280 con 0x7fe3d0107840 2024-06-13T06:55:37.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.704+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d01a7380 con 0x7fe3d0107840 2024-06-13T06:55:37.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.704+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a7870 con 0x7fe3d0107840 2024-06-13T06:55:37.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.707+0000 7fe3cd600700 1 -- 172.21.15.7:0/510780633 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe3c801b030 con 0x7fe3d0107840 2024-06-13T06:55:37.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 Processor -- start 2024-06-13T06:55:37.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3cd600700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 0x7fe3b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 -- start start 2024-06-13T06:55:37.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3cd600700 1 -- 172.21.15.7:0/510780633 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe3c80cee90 con 0x7fe3d0107840 2024-06-13T06:55:37.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b3aeb74a90 con 0x55b3aec65630 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b3aebb7b80 con 0x55b3aec70380 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3aeb72970 con 0x55b3aebe1ec0 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.709+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 0x7fe3b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3d5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33062/0 (socket says 172.21.15.7:33062) 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3cfe00700 1 -- 172.21.15.7:0/4144056735 learned_addr learned my addr 172.21.15.7:0/4144056735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3d5a00700 1 -- 172.21.15.7:0/4144056735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 msgr2=0x55b3aec650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.710+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3d5a00700 1 -- 172.21.15.7:0/4144056735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 msgr2=0x55b3aec65ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3d5a00700 1 -- 172.21.15.7:0/4144056735 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3aeb60010 con 0x55b3aec70380 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec72770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cc630 tx=0x7fe3c809b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150424a331f298f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3c2000700 1 -- 172.21.15.7:0/4144056735 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c809bb20 con 0x55b3aec70380 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3c2000700 1 -- 172.21.15.7:0/4144056735 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c809b420 con 0x55b3aec70380 2024-06-13T06:55:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.711+0000 7fe3c2000700 1 -- 172.21.15.7:0/4144056735 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a3dd0 con 0x55b3aec70380 2024-06-13T06:55:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/4144056735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 msgr2=0x55b3aec72770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 0x7fe3b80a5160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b3aec629a0 tx=0x7fe3bc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec72770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cc630 tx=0x7fe3c809b2c0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/4144056735 shutdown_connections 2024-06-13T06:55:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec72770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/4144056735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec65ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.712+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/4144056735 >> 172.21.15.7:0/4144056735 conn(0x55b3aeb921f0 msgr2=0x55b3aec61530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.713+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/4144056735 shutdown_connections 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.713+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/4144056735 wait complete. 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.713+0000 7fe3e23c6b80 1 -- start start 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.713+0000 7fe3e23c6b80 1 Processor -- start 2024-06-13T06:55:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aebe1ec0 0x55b3aec6fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 0x55b3aec6ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b3aeb53fe0 con 0x55b3aec70380 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b3aebb3b40 con 0x55b3aebe1ec0 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3aec73040 con 0x55b3aec65630 2024-06-13T06:55:37.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3e23c6b80 10 client.0 _ll_register_callbacks cb 0x55b3aeb517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 0x55b3aec6ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3d5a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 0x55b3aec6ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52370/0 (socket says 172.21.15.7:52370) 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.714+0000 7fe3cfe00700 1 -- 172.21.15.7:0/284571410 learned_addr learned my addr 172.21.15.7:0/284571410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5000700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aebe1ec0 0x55b3aec6fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 -- 172.21.15.7:0/284571410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 msgr2=0x55b3aec6ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 0x55b3aec6ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 -- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aebe1ec0 msgr2=0x55b3aec6fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aebe1ec0 0x55b3aec6fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 -- 172.21.15.7:0/284571410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3aeb06950 con 0x55b3aec70380 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3d5a00700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec6d360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3c809f990 tx=0x7fe3c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e25041247cba66 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.715+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80b8020 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe3c80b4150 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c809b420 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3c80b4370 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3e23c6b80 10 client.15102 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3aec85fa0 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c809b420 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe3c8097200 con 0x55b3aec70380 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.716+0000 7fe3e23c6b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3e23c6b80 10 client.15102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3e23c6b80 20 client.15102 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aec88470 0x55b3aec8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3e23c6b80 4 client.15102 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3aebc5080 con 0x55b3aec88470 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.717+0000 7fe3d5000700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aec88470 0x55b3aec8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.718+0000 7fe3d5000700 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aec88470 0x55b3aec8a860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa517a5ced64929 server_cookie=fdb8ae34d88b50b6 in_seq=0 out_seq=0 2024-06-13T06:55:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.718+0000 7fe3b6a00700 10 client.15102 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3b6a00700 1 -- 172.21.15.7:0/284571410 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b3aebc5080 con 0x55b3aec88470 2024-06-13T06:55:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3b6a00700 10 client.15102 handle_command_reply: tid=0 2024-06-13T06:55:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 client.15102 shutdown 2024-06-13T06:55:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 msgr2=0x55b3aec6d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec6d360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3c809f990 tx=0x7fe3c800cf60 comp rx=0 tx=0).stop 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 shutdown_connections 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aec88470 0x55b3aec8a860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec6d360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aebe1ec0 0x55b3aec6fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/284571410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aec65630 0x55b3aec6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 >> 172.21.15.7:0/284571410 conn(0x55b3aeb921f0 msgr2=0x55b3aec607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3b6a00700 0 client.15102 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.719+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 shutdown_connections 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.720+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/284571410 wait complete. 2024-06-13T06:55:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.720+0000 7fe3e23c6b80 20 client.15102 trim_cache size 0 max 16384 2024-06-13T06:55:37.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 Processor -- start 2024-06-13T06:55:37.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 -- start start 2024-06-13T06:55:37.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec60640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ae8f0 con 0x55b3aec65630 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ac180 con 0x55b3aec70380 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.743+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01aa000 con 0x55b3aebe1ec0 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec60640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52380/0 (socket says 172.21.15.7:52380) 2024-06-13T06:55:37.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 -- 172.21.15.7:0/825689597 learned_addr learned my addr 172.21.15.7:0/825689597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 -- 172.21.15.7:0/825689597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 msgr2=0x55b3aec60640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec60640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 -- 172.21.15.7:0/825689597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 msgr2=0x55b3aec6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec6bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3d5000700 1 -- 172.21.15.7:0/825689597 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3aec6cbf0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.744+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:37.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.745+0000 7fe3d5000700 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009990 tx=0x7fe3c4005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c84c98db1885924 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.745+0000 7fe3c1600700 1 -- 172.21.15.7:0/825689597 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4005940 con 0x55b3aebe1ec0 2024-06-13T06:55:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.745+0000 7fe3c1600700 1 -- 172.21.15.7:0/825689597 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c400bcd0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.745+0000 7fe3c1600700 1 -- 172.21.15.7:0/825689597 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4003e50 con 0x55b3aebe1ec0 2024-06-13T06:55:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.746+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/825689597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 msgr2=0x55b3aec616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.746+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009990 tx=0x7fe3c4005800 comp rx=0 tx=0).stop 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/825689597 shutdown_connections 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec70380 0x55b3aec60640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec616a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/825689597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec65630 0x55b3aec6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/825689597 >> 172.21.15.7:0/825689597 conn(0x55b3aeb921f0 msgr2=0x55b3aec5ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/825689597 shutdown_connections 2024-06-13T06:55:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/825689597 wait complete. 2024-06-13T06:55:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.747+0000 7fe3e23c6b80 1 -- start start 2024-06-13T06:55:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 Processor -- start 2024-06-13T06:55:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec65630 0x55b3aec699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01aa640 con 0x55b3aec70380 2024-06-13T06:55:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.748+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01abab0 con 0x55b3aec65630 2024-06-13T06:55:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3e23c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01aad10 con 0x55b3aebe1ec0 2024-06-13T06:55:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3e23c6b80 10 client.0 _ll_register_callbacks cb 0x55b3aeb517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52396/0 (socket says 172.21.15.7:52396) 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 -- 172.21.15.7:0/421434068 learned_addr learned my addr 172.21.15.7:0/421434068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 -- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec65630 msgr2=0x55b3aec699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec65630 0x55b3aec699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 -- 172.21.15.7:0/421434068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 msgr2=0x55b3aec69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.749+0000 7fe3d5000700 1 -- 172.21.15.7:0/421434068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3aec6a610 con 0x55b3aebe1ec0 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.750+0000 7fe3d5000700 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c4005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513ee511ab67857e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.750+0000 7fe3b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.750+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4003a50 con 0x55b3aebe1ec0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.750+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe3c4033040 con 0x55b3aebe1ec0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.750+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c400bcd0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.751+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3c4003bf0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.751+0000 7fe3e23c6b80 10 client.25231 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.751+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3aecf2cb0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.751+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c400bcd0 con 0x55b3aebe1ec0 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe3c400f990 con 0x55b3aebe1ec0 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 20 client.25231 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aecf31e0 0x55b3aecf5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:37.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 4 client.25231 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T06:55:37.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.752+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3aebbf600 con 0x55b3aecf31e0 2024-06-13T06:55:37.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.753+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aecf31e0 0x55b3aecf5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:37.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.754+0000 7fe3cfe00700 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aecf31e0 0x55b3aecf5600 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a524dba39c552c server_cookie=a22314e651d86ffc in_seq=0 out_seq=0 2024-06-13T06:55:37.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.754+0000 7fe3b6a00700 10 client.25231 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:37.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.755+0000 7fe3b6a00700 1 -- 172.21.15.7:0/421434068 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3aebbf600 con 0x55b3aecf31e0 2024-06-13T06:55:37.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.755+0000 7fe3b6a00700 10 client.25231 handle_command_reply: tid=0 2024-06-13T06:55:37.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.755+0000 7fe3e23c6b80 1 client.25231 shutdown 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 msgr2=0x55b3aec8dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c4005c90 comp rx=0 tx=0).stop 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 shutdown_connections 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b3aecf31e0 0x55b3aecf5600 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3aec70380 0x55b3aec69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3aebe1ec0 0x55b3aec8dac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 --2- 172.21.15.7:0/421434068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3aec65630 0x55b3aec699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 >> 172.21.15.7:0/421434068 conn(0x55b3aeb921f0 msgr2=0x55b3aec607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.756+0000 7fe3b6a00700 0 client.25231 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.757+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 shutdown_connections 2024-06-13T06:55:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.757+0000 7fe3e23c6b80 1 -- 172.21.15.7:0/421434068 wait complete. 2024-06-13T06:55:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.757+0000 7fe3e23c6b80 20 client.25231 trim_cache size 0 max 16384 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e14bfda4-1a07-43c1-ac31-427d81b97871", 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.758+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 msgr2=0x7fe3b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.758+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 0x7fe3b80a5160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b3aec629a0 tx=0x7fe3bc0095c0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.758+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 msgr2=0x7fe3d0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.758+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 0x7fe3d0075210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800efc0 tx=0x7fe3c800c3a0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 shutdown_connections 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0107840 0x7fe3d0075210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d0102740 0x7fe3d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe3b80a2ca0 0x7fe3b80a5160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 --2- 172.21.15.7:0/510780633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3d01030f0 0x7fe3d0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.759+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 >> 172.21.15.7:0/510780633 conn(0x7fe3d00795f0 msgr2=0x7fe3d01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.760+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 shutdown_connections 2024-06-13T06:55:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:37.760+0000 7fe3d7a00700 1 -- 172.21.15.7:0/510780633 wait complete. 2024-06-13T06:55:37.894 INFO:teuthology.orchestra.run.smithi007.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-06-13T06:55:38.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:55:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:55:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:39 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T06:55:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:39 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:39 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:55:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:39 smithi189 ceph-mon[36616]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:39 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:39.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:39 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T06:55:39.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:39 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:39.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:39 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:55:39.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:39 smithi007 ceph-mon[32077]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:39.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:39 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:39 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T06:55:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:39 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:39 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:55:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:39 smithi175 ceph-mon[36886]: pgmap v497: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-13T06:55:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:39 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:55:41.375 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:41 smithi007 ceph-mon[32077]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-13T06:55:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:41 smithi189 ceph-mon[36616]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-13T06:55:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:41 smithi175 ceph-mon[36886]: pgmap v498: 129 pgs: 129 active+clean; 656 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-13T06:55:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 -- 172.21.15.7:0/3725563030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4104f80 msgr2=0x7f38e4111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 --2- 172.21.15.7:0/3725563030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4104f80 0x7f38e4111440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009fc0 tx=0x7f38dc00b330 comp rx=0 tx=0).stop 2024-06-13T06:55:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 -- 172.21.15.7:0/3725563030 shutdown_connections 2024-06-13T06:55:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 --2- 172.21.15.7:0/3725563030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4104f80 0x7f38e4111440 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 --2- 172.21.15.7:0/3725563030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e410a670 0x7f38e410aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 --2- 172.21.15.7:0/3725563030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e4104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 -- 172.21.15.7:0/3725563030 >> 172.21.15.7:0/3725563030 conn(0x7f38e4100070 msgr2=0x7f38e4102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 -- 172.21.15.7:0/3725563030 shutdown_connections 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.710+0000 7f38e9800700 1 -- 172.21.15.7:0/3725563030 wait complete. 2024-06-13T06:55:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 Processor -- start 2024-06-13T06:55:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 -- start start 2024-06-13T06:55:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 0x7f38e41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 0x7f38e419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41148e0 con 0x7f38e410a670 2024-06-13T06:55:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4114760 con 0x7f38e4104f80 2024-06-13T06:55:41.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.712+0000 7f38e9800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4114a60 con 0x7f38e41045c0 2024-06-13T06:55:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e8e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 0x7f38e419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 0x7f38e41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 0x7f38e41a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49490/0 (socket says 172.21.15.7:49490) 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e8e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 0x7f38e419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48846/0 (socket says 172.21.15.7:48846) 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3400700 1 -- 172.21.15.7:0/3427838460 learned_addr learned my addr 172.21.15.7:0/3427838460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 -- 172.21.15.7:0/3427838460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 msgr2=0x7f38e41a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 0x7f38e41a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 -- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 msgr2=0x7f38e419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 0x7f38e419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.713+0000 7f38e3e00700 1 -- 172.21.15.7:0/3427838460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e419b220 con 0x7f38e41045c0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e3e00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e41a08b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38d400eb90 tx=0x7f38d400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e0c00700 1 -- 172.21.15.7:0/3427838460 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d400cc90 con 0x7f38e41045c0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e419b510 con 0x7f38e41045c0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e0c00700 1 -- 172.21.15.7:0/3427838460 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d400ce30 con 0x7f38e41045c0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e0c00700 1 -- 172.21.15.7:0/3427838460 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4010430 con 0x7f38e41045c0 2024-06-13T06:55:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.714+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e41a7780 con 0x7f38e41045c0 2024-06-13T06:55:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.718+0000 7f38e0c00700 1 -- 172.21.15.7:0/3427838460 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f38d4003680 con 0x7f38e41045c0 2024-06-13T06:55:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38f5818b80 1 Processor -- start 2024-06-13T06:55:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38e0c00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 0x7f38c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38f5818b80 1 -- start start 2024-06-13T06:55:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38e0c00700 1 -- 172.21.15.7:0/3427838460 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f38d40cf620 con 0x7f38e41045c0 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38e8e00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 0x7f38c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.719+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38f5818b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577fd4eaa10 con 0x5577fd5dd960 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38f5818b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5577fd52db00 con 0x5577fd557e40 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38f5818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577fd4e88f0 con 0x5577fd5e6570 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48862/0 (socket says 172.21.15.7:48862) 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 -- 172.21.15.7:0/3877604067 learned_addr learned my addr 172.21.15.7:0/3877604067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3e00700 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 -- 172.21.15.7:0/3877604067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 msgr2=0x5577fd5e8960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e8960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 -- 172.21.15.7:0/3877604067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 msgr2=0x5577fd5dd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.720+0000 7f38e3400700 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5dd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38e3400700 1 -- 172.21.15.7:0/3877604067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577fd4d5f90 con 0x5577fd5dd960 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38e3e00700 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38e3400700 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009e40 tx=0x7f38d0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9d1485c4e94549 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38e8e00700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 0x7f38c80a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009550 tx=0x7f38dc009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38d9600700 1 -- 172.21.15.7:0/3877604067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0004a30 con 0x5577fd5dd960 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38d9600700 1 -- 172.21.15.7:0/3877604067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d0005500 con 0x5577fd5dd960 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.721+0000 7f38d9600700 1 -- 172.21.15.7:0/3877604067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00199f0 con 0x5577fd5dd960 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.722+0000 7f38f5818b80 1 -- 172.21.15.7:0/3877604067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 msgr2=0x5577fd5ddde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.722+0000 7f38f5818b80 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009e40 tx=0x7f38d0011720 comp rx=0 tx=0).stop 2024-06-13T06:55:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.722+0000 7f38f5818b80 1 -- 172.21.15.7:0/3877604067 shutdown_connections 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.722+0000 7f38f5818b80 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.722+0000 7f38f5818b80 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.723+0000 7f38f5818b80 1 --2- 172.21.15.7:0/3877604067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5dd960 0x5577fd5ddde0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.723+0000 7f38f5818b80 1 -- 172.21.15.7:0/3877604067 >> 172.21.15.7:0/3877604067 conn(0x5577fd508170 msgr2=0x5577fd5d7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.723+0000 7f38f5818b80 1 -- 172.21.15.7:0/3877604067 shutdown_connections 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.723+0000 7f38f5818b80 1 -- 172.21.15.7:0/3877604067 wait complete. 2024-06-13T06:55:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.723+0000 7f38f5818b80 1 -- start start 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 Processor -- start 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577fd4c9f60 con 0x5577fd557e40 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5577fd529ac0 con 0x5577fd5dd960 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577fd5e91e0 con 0x5577fd5e6570 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38f5818b80 10 client.0 _ll_register_callbacks cb 0x5577fd4c7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38e3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.724+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48864/0 (socket says 172.21.15.7:48864) 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3e00700 1 -- 172.21.15.7:0/2364527578 learned_addr learned my addr 172.21.15.7:0/2364527578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e8e00700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 -- 172.21.15.7:0/2364527578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 msgr2=0x5577fd5e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 -- 172.21.15.7:0/2364527578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 msgr2=0x5577fd5e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 -- 172.21.15.7:0/2364527578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577fd47c8d0 con 0x5577fd5dd960 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e8e00700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38e3400700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5e2fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f38d0011a70 tx=0x7f38d0004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ac40d35ef97c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.725+0000 7f38ce000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:55:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0028070 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f38d0004bf0 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d0005500 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38d0034050 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38f5818b80 10 client.25244 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.726+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577fd5f38f0 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d0005500 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f38d0024430 con 0x5577fd5dd960 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 10 client.25244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 20 client.25244 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd5f5e20 0x5577fd5f8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 4 client.25244 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577fd53b000 con 0x5577fd5f5e20 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.727+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd5f5e20 0x5577fd5f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.728+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd5f5e20 0x5577fd5f8210 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8621849feccf60 server_cookie=2f92eb22eecb9537 in_seq=0 out_seq=0 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.728+0000 7f38ce000700 10 client.25244 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.728+0000 7f38ce000700 1 -- 172.21.15.7:0/2364527578 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5577fd53b000 con 0x5577fd5f5e20 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.728+0000 7f38ce000700 10 client.25244 handle_command_reply: tid=0 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 client.25244 shutdown 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 msgr2=0x5577fd5e2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5e2fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f38d0011a70 tx=0x7f38d0004ec0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 shutdown_connections 2024-06-13T06:55:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd5f5e20 0x5577fd5f8210 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd5e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2364527578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5e2fd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38ce000700 0 client.25244 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 >> 172.21.15.7:0/2364527578 conn(0x5577fd508170 msgr2=0x5577fd5d7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 shutdown_connections 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 1 -- 172.21.15.7:0/2364527578 wait complete. 2024-06-13T06:55:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.729+0000 7f38f5818b80 20 client.25244 trim_cache size 0 max 16384 2024-06-13T06:55:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 Processor -- start 2024-06-13T06:55:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 -- start start 2024-06-13T06:55:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 0x5577fd5e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5e6570 0x5577fd5d7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41b2a70 con 0x5577fd5e6570 2024-06-13T06:55:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41b0300 con 0x5577fd557e40 2024-06-13T06:55:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.756+0000 7f38f5818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41ae150 con 0x5577fd5dd960 2024-06-13T06:55:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49510/0 (socket says 172.21.15.7:49510) 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 -- 172.21.15.7:0/4037483211 learned_addr learned my addr 172.21.15.7:0/4037483211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3400700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 0x5577fd5e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e8e00700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5e6570 0x5577fd5d7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 -- 172.21.15.7:0/4037483211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 msgr2=0x5577fd5e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 0x5577fd5e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 -- 172.21.15.7:0/4037483211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5e6570 msgr2=0x5577fd5d7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5e6570 0x5577fd5d7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3e00700 1 -- 172.21.15.7:0/4037483211 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577fd5e3eb0 con 0x5577fd557e40 2024-06-13T06:55:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.757+0000 7f38e3400700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 0x5577fd5e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:55:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.758+0000 7f38e3e00700 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f38d40ccd20 tx=0x7f38d4097d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d99e0dc4c9907b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.758+0000 7f38d8c00700 1 -- 172.21.15.7:0/4037483211 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d40b3020 con 0x5577fd557e40 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.758+0000 7f38d8c00700 1 -- 172.21.15.7:0/4037483211 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d409fe90 con 0x5577fd557e40 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.758+0000 7f38d8c00700 1 -- 172.21.15.7:0/4037483211 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a42c0 con 0x5577fd557e40 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 -- 172.21.15.7:0/4037483211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 msgr2=0x5577fd5e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f38d40ccd20 tx=0x7f38d4097d80 comp rx=0 tx=0).stop 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 -- 172.21.15.7:0/4037483211 shutdown_connections 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd5e6570 0x5577fd5d7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd557e40 0x5577fd5e2f90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 --2- 172.21.15.7:0/4037483211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5dd960 0x5577fd5e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 -- 172.21.15.7:0/4037483211 >> 172.21.15.7:0/4037483211 conn(0x5577fd508170 msgr2=0x5577fd5d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 -- 172.21.15.7:0/4037483211 shutdown_connections 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.759+0000 7f38f5818b80 1 -- 172.21.15.7:0/4037483211 wait complete. 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.760+0000 7f38f5818b80 1 -- start start 2024-06-13T06:55:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.760+0000 7f38f5818b80 1 Processor -- start 2024-06-13T06:55:41.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd6588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41ae7c0 con 0x5577fd557e40 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41afc30 con 0x5577fd5dd960 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38e41aee90 con 0x5577fd5e6570 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38f5818b80 10 client.0 _ll_register_callbacks cb 0x5577fd4c7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48890/0 (socket says 172.21.15.7:48890) 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e3e00700 1 -- 172.21.15.7:0/2610169891 learned_addr learned my addr 172.21.15.7:0/2610169891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e8e00700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd6588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e3400700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.761+0000 7f38e3e00700 1 -- 172.21.15.7:0/2610169891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 msgr2=0x5577fd6588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd6588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38e3e00700 1 -- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 msgr2=0x5577fd5ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38e3e00700 1 -- 172.21.15.7:0/2610169891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577fd658e60 con 0x5577fd557e40 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38e3e00700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fd0 tx=0x7f38d40a47d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557596c860278db7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38ce000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d407f060 con 0x5577fd557e40 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f38d409bd40 con 0x5577fd557e40 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d409fe90 con 0x5577fd557e40 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.762+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f38d40b3040 con 0x5577fd557e40 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 10 client.15126 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577fd660220 con 0x5577fd557e40 2024-06-13T06:55:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d409fe90 con 0x5577fd557e40 2024-06-13T06:55:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f38d40b4ce0 con 0x5577fd557e40 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 10 client.15126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 20 client.15126 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd660800 0x5577fd662c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 4 client.15126 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.763+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577fd535580 con 0x5577fd660800 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38e3400700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd660800 0x5577fd662c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38e3400700 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd660800 0x5577fd662c20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90629e9e33af1351 server_cookie=c433f6234dfc68dc in_seq=0 out_seq=0 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38ce000700 10 client.15126 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38ce000700 1 -- 172.21.15.7:0/2610169891 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5577fd535580 con 0x5577fd660800 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38ce000700 10 client.15126 handle_command_reply: tid=0 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38f5818b80 1 client.15126 shutdown 2024-06-13T06:55:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 msgr2=0x5577fd5e9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.764+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fd0 tx=0x7f38d40a47d0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 shutdown_connections 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5577fd660800 0x5577fd662c20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577fd5e6570 0x5577fd6588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577fd557e40 0x5577fd5e9bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 --2- 172.21.15.7:0/2610169891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5577fd5dd960 0x5577fd5ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 >> 172.21.15.7:0/2610169891 conn(0x5577fd508170 msgr2=0x5577fd5e2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38ce000700 0 client.15126 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 shutdown_connections 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 1 -- 172.21.15.7:0/2610169891 wait complete. 2024-06-13T06:55:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.765+0000 7f38f5818b80 20 client.15126 trim_cache size 0 max 16384 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "e14bfda4-1a07-43c1-ac31-427d81b97871": { 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e14bfda4-1a07-43c1-ac31-427d81b97871", 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:55:41.768 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 msgr2=0x7f38c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 0x7f38c80a5160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009550 tx=0x7f38dc009f10 comp rx=0 tx=0).stop 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 msgr2=0x7f38e41a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e41a08b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38d400eb90 tx=0x7f38d400eea0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 shutdown_connections 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e410a670 0x7f38e419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38e41045c0 0x7f38e41a08b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f38c80a2ca0 0x7f38c80a5160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 --2- 172.21.15.7:0/3427838460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38e4104f80 0x7f38e41a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 >> 172.21.15.7:0/3427838460 conn(0x7f38e4100070 msgr2=0x7f38e41015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 shutdown_connections 2024-06-13T06:55:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:55:41.766+0000 7f38e9800700 1 -- 172.21.15.7:0/3427838460 wait complete. 2024-06-13T06:55:42.792 INFO:tasks.cephfs.filesystem:scrub status for tag:e14bfda4-1a07-43c1-ac31-427d81b97871 - {'path': '/', 'tag': 'e14bfda4-1a07-43c1-ac31-427d81b97871', 'options': 'recursive,force'} 2024-06-13T06:55:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:43 smithi189 ceph-mon[36616]: pgmap v499: 129 pgs: 129 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-06-13T06:55:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:43 smithi175 ceph-mon[36886]: pgmap v499: 129 pgs: 129 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-06-13T06:55:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:43 smithi007 ceph-mon[32077]: pgmap v499: 129 pgs: 129 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-06-13T06:55:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:45 smithi189 ceph-mon[36616]: pgmap v500: 129 pgs: 129 active+clean; 19 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-06-13T06:55:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:45 smithi175 ceph-mon[36886]: pgmap v500: 129 pgs: 129 active+clean; 19 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-06-13T06:55:45.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:45 smithi007 ceph-mon[32077]: pgmap v500: 129 pgs: 129 active+clean; 19 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.578+0000 7f2129800640 1 -- 172.21.15.7:0/4286201703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21241022d0 msgr2=0x7f2124108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.578+0000 7f2129800640 1 --2- 172.21.15.7:0/4286201703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21241022d0 0x7f2124108b60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c02a3f0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 -- 172.21.15.7:0/4286201703 shutdown_connections 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 --2- 172.21.15.7:0/4286201703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21241022d0 0x7f2124108b60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 --2- 172.21.15.7:0/4286201703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2124101910 0x7f2124101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 --2- 172.21.15.7:0/4286201703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 -- 172.21.15.7:0/4286201703 >> 172.21.15.7:0/4286201703 conn(0x7f21240fbc90 msgr2=0x7f21240fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 -- 172.21.15.7:0/4286201703 shutdown_connections 2024-06-13T06:55:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.579+0000 7f2129800640 1 -- 172.21.15.7:0/4286201703 wait complete. 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.580+0000 7f2129800640 1 Processor -- start 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.580+0000 7f2129800640 1 -- start start 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.580+0000 7f2129800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.580+0000 7f2129800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2124101910 0x7f21241980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2129800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2129800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f212410c3f0 con 0x7f2124100720 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2129800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f212410c270 con 0x7f21241022d0 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2129800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f212410c570 con 0x7f2124101910 2024-06-13T06:55:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48908/0 (socket says 172.21.15.7:48908) 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49540/0 (socket says 172.21.15.7:49540) 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2123e00640 1 -- 172.21.15.7:0/1596380381 learned_addr learned my addr 172.21.15.7:0/1596380381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2123400640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2124101910 0x7f21241980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 -- 172.21.15.7:0/1596380381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2124101910 msgr2=0x7f21241980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2124101910 0x7f21241980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 -- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 msgr2=0x7f2124197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.581+0000 7f2128e00640 1 -- 172.21.15.7:0/1596380381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212419f750 con 0x7f21241022d0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2128e00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f211c02a910 tx=0x7f211c032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2120c00640 1 -- 172.21.15.7:0/1596380381 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c03f030 con 0x7f21241022d0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212419f9e0 con 0x7f21241022d0 2024-06-13T06:55:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2120c00640 1 -- 172.21.15.7:0/1596380381 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c032e50 con 0x7f21241022d0 2024-06-13T06:55:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2120c00640 1 -- 172.21.15.7:0/1596380381 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c035070 con 0x7f21241022d0 2024-06-13T06:55:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.582+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212419fea0 con 0x7f21241022d0 2024-06-13T06:55:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 Processor -- start 2024-06-13T06:55:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f2120c00640 1 -- 172.21.15.7:0/1596380381 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f211c042030 con 0x7f21241022d0 2024-06-13T06:55:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 -- start start 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e71db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1dcd420 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1d74c50 con 0x55f7b1e69dc0 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f212dbbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1dc93e0 con 0x55f7b1e6f920 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.586+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48922/0 (socket says 172.21.15.7:48922) 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123e00640 1 -- 172.21.15.7:0/3267327478 learned_addr learned my addr 172.21.15.7:0/3267327478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e71db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 -- 172.21.15.7:0/3267327478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 msgr2=0x55f7b1e71db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e71db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 -- 172.21.15.7:0/3267327478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 msgr2=0x55f7b1e6f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 -- 172.21.15.7:0/3267327478 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b1ae1210 con 0x55f7b1e69dc0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123e00640 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2120c00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 0x7f21080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2123400640 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6a1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f40 tx=0x7f210c009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85fc89f8f5ed6a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.587+0000 7f2120c00640 1 -- 172.21.15.7:0/1596380381 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f211c0ef620 con 0x7f21241022d0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f2123e00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 0x7f21080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f2119600640 1 -- 172.21.15.7:0/3267327478 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c004050 con 0x55f7b1e69dc0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f2119600640 1 -- 172.21.15.7:0/3267327478 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005500 con 0x55f7b1e69dc0 2024-06-13T06:55:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f2119600640 1 -- 172.21.15.7:0/3267327478 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c0035f0 con 0x55f7b1e69dc0 2024-06-13T06:55:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 -- 172.21.15.7:0/3267327478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 msgr2=0x55f7b1e6a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6a1e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f40 tx=0x7f210c009940 comp rx=0 tx=0).stop 2024-06-13T06:55:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f2123e00640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 0x7f21080a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f7b1e6abb0 tx=0x7f2114009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 -- 172.21.15.7:0/3267327478 shutdown_connections 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e71db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 --2- 172.21.15.7:0/3267327478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6a1e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.588+0000 7f212dbbe740 1 -- 172.21.15.7:0/3267327478 >> 172.21.15.7:0/3267327478 conn(0x55f7b1e31fb0 msgr2=0x55f7b1e65840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.589+0000 7f212dbbe740 1 -- 172.21.15.7:0/3267327478 shutdown_connections 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.589+0000 7f212dbbe740 1 -- 172.21.15.7:0/3267327478 wait complete. 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.589+0000 7f212dbbe740 1 -- start start 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.589+0000 7f212dbbe740 1 Processor -- start 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e7fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1e728d0 con 0x55f7b1e69dc0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1d880b0 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1e72a50 con 0x55f7b1e6f920 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f212dbbe740 10 client.0 _ll_register_callbacks cb 0x55f7b1d73f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48928/0 (socket says 172.21.15.7:48928) 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 -- 172.21.15.7:0/2100940552 learned_addr learned my addr 172.21.15.7:0/2100940552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123400640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2128e00640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e7fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 -- 172.21.15.7:0/2100940552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 msgr2=0x55f7b1e7fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e7fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 -- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 msgr2=0x55f7b1e78b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e78b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.590+0000 7f2123e00640 1 -- 172.21.15.7:0/2100940552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b1bad620 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f2123e00640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2114011650 tx=0x7f2114004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c46fb263f30851 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f2112000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114003b80 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2114003d20 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c005500 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2114004b20 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f212dbbe740 10 client.15138 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.591+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7b1e82fb0 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c005500 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2114026070 con 0x55f7b1e69dc0 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 10 client.15138 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 10 client.15138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 20 client.15138 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1e846f0 0x55f7b1e86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 4 client.15138 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b1ddab50 con 0x55f7b1e846f0 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.592+0000 7f2123400640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1e846f0 0x55f7b1e86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.593+0000 7f2123400640 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1e846f0 0x55f7b1e86b90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df045d3db3174b5 server_cookie=84470bbf178e7570 in_seq=0 out_seq=0 2024-06-13T06:55:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.593+0000 7f2112000640 10 client.15138 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f2112000640 1 -- 172.21.15.7:0/2100940552 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f7b1ddab50 con 0x55f7b1e846f0 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f2112000640 10 client.15138 handle_command_reply: tid=0 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 client.15138 shutdown 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 msgr2=0x55f7b1e785f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2114011650 tx=0x7f2114004ab0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 shutdown_connections 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1e846f0 0x55f7b1e86b90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e6f920 0x55f7b1e7fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e78b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2100940552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e785f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f2112000640 0 client.15138 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.594+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 >> 172.21.15.7:0/2100940552 conn(0x55f7b1e31fb0 msgr2=0x55f7b1e649a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.595+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 shutdown_connections 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.595+0000 7f212dbbe740 1 -- 172.21.15.7:0/2100940552 wait complete. 2024-06-13T06:55:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.595+0000 7f212dbbe740 20 client.15138 trim_cache size 0 max 16384 2024-06-13T06:55:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 Processor -- start 2024-06-13T06:55:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 -- start start 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e68e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21241aa940 con 0x55f7b1e6f920 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a8210 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f212dbbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a6060 con 0x55f7b1e69dc0 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f2128e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f2123e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e68e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f2128e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48942/0 (socket says 172.21.15.7:48942) 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f2128e00640 1 -- 172.21.15.7:0/2567016741 learned_addr learned my addr 172.21.15.7:0/2567016741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.612+0000 7f2123400640 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 -- 172.21.15.7:0/2567016741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 msgr2=0x55f7b1e68e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e68e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 -- 172.21.15.7:0/2567016741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 msgr2=0x55f7b1e6f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 -- 172.21.15.7:0/2567016741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b1e664f0 con 0x55f7b1e6f920 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2123e00640 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e68e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2128e00640 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211c03b6d0 tx=0x7f211c0bb7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7a81c44bd223b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2118c00640 1 -- 172.21.15.7:0/2567016741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c049040 con 0x55f7b1e6f920 2024-06-13T06:55:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2118c00640 1 -- 172.21.15.7:0/2567016741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c0a6040 con 0x55f7b1e6f920 2024-06-13T06:55:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.613+0000 7f2118c00640 1 -- 172.21.15.7:0/2567016741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0b82c0 con 0x55f7b1e6f920 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 -- 172.21.15.7:0/2567016741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 msgr2=0x55f7b1e77c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f211c03b6d0 tx=0x7f211c0bb7f0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 -- 172.21.15.7:0/2567016741 shutdown_connections 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1e77c10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2567016741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e68e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 -- 172.21.15.7:0/2567016741 >> 172.21.15.7:0/2567016741 conn(0x55f7b1e31fb0 msgr2=0x55f7b1e68700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 -- 172.21.15.7:0/2567016741 shutdown_connections 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.614+0000 7f212dbbe740 1 -- 172.21.15.7:0/2567016741 wait complete. 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 -- start start 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 Processor -- start 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1ef1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a66d0 con 0x55f7b1e6f920 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a7b40 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a6da0 con 0x55f7b1e69dc0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.615+0000 7f212dbbe740 10 client.0 _ll_register_callbacks cb 0x7f2114009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2128e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1ef1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49600/0 (socket says 172.21.15.7:49600) 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 -- 172.21.15.7:0/2902733503 learned_addr learned my addr 172.21.15.7:0/2902733503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123e00640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 -- 172.21.15.7:0/2902733503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 msgr2=0x55f7b1e6e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 -- 172.21.15.7:0/2902733503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 msgr2=0x55f7b1ef1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:55:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1ef1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 -- 172.21.15.7:0/2902733503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b1ef2190 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2128e00640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1ef1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2123400640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f40 tx=0x7f210c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcdc348e86c83d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.616+0000 7f2112000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c034670 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f210c01e460 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c0a6040 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.620 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:47 smithi175 ceph-mon[36886]: pgmap v501: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.1 MiB/s wr, 33 op/s 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f210c034810 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f212dbbe740 10 client.25268 fetch_fsmap learned FSMap version 26 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f7b1ef32d0 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c0a6040 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.617+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f210c01ec60 con 0x55f7b1e6a7b0 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 10 client.25268 fetch_fsmap finished waiting for FSMap version 26 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 20 client.25268 populate_metadata read hostname 'smithi007' 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1ef39b0 0x55f7b1ef5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 4 client.25268 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7b1dd4ea0 con 0x55f7b1ef39b0 2024-06-13T06:55:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f2123e00640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1ef39b0 0x55f7b1ef5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:55:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f2123e00640 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1ef39b0 0x55f7b1ef5dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e82eac08ff96236 server_cookie=d81c3edcec58b10c in_seq=0 out_seq=0 2024-06-13T06:55:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:47.618+0000 7f2112000640 10 client.25268 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:47 smithi007 ceph-mon[32077]: pgmap v501: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.1 MiB/s wr, 33 op/s 2024-06-13T06:55:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:47 smithi189 ceph-mon[36616]: pgmap v501: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.1 MiB/s wr, 33 op/s 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f2112000640 1 -- 172.21.15.7:0/2902733503 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7b1dd4ea0 con 0x55f7b1ef39b0 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f2112000640 10 client.25268 handle_command_reply: tid=0 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 client.25268 shutdown 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 msgr2=0x55f7b1e6ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f40 tx=0x7f210c009f70 comp rx=0 tx=0).stop 2024-06-13T06:55:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 shutdown_connections 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f7b1ef39b0 0x55f7b1ef5dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7b1e6f920 0x55f7b1ef1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7b1e6a7b0 0x55f7b1e6ebf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f2112000640 0 client.25268 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 --2- 172.21.15.7:0/2902733503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f7b1e69dc0 0x55f7b1e6e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.109+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 >> 172.21.15.7:0/2902733503 conn(0x55f7b1e31fb0 msgr2=0x55f7b1e68f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 shutdown_connections 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f212dbbe740 1 -- 172.21.15.7:0/2902733503 wait complete. 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f212dbbe740 20 client.25268 trim_cache size 0 max 16384 2024-06-13T06:55:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:55:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:55:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:55:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 msgr2=0x7f21080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 0x7f21080a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f7b1e6abb0 tx=0x7f2114009500 comp rx=0 tx=0).stop 2024-06-13T06:55:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 msgr2=0x7f212419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.110+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f211c02a910 tx=0x7f211c032c60 comp rx=0 tx=0).stop 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 shutdown_connections 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f21080a2b80 0x7f21080a5040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21241022d0 0x7f212419f0e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2124101910 0x7f21241980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 --2- 172.21.15.7:0/1596380381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2124100720 0x7f2124197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 >> 172.21.15.7:0/1596380381 conn(0x7f21240fbc90 msgr2=0x7f2124069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 shutdown_connections 2024-06-13T06:55:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:55:48.111+0000 7f2129800640 1 -- 172.21.15.7:0/1596380381 wait complete. 2024-06-13T06:55:49.058 INFO:teuthology.orchestra.run.smithi007.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-06-13T06:55:49.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 data 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:total 35 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 . 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T06:55:49.219 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 base 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Jun 13 06:55 global 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 log 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_commit_ts 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_dynshmem 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Jun 13 06:55 pg_hba.conf 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 13 06:55 pg_ident.conf 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 pg_logical 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 pg_multixact 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_notify 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_replslot 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_serial 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_snapshots 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_stat 2024-06-13T06:55:49.220 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_stat_tmp 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 pg_subtrans 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_tblspc 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_twophase 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 pg_wal 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 pg_xact 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 13 06:55 postgresql.auto.conf 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Jun 13 06:55 postgresql.conf 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 . 2024-06-13T06:55:49.221 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 1 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 13493 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 13494 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 . 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 .. 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 1247 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1247_vm 2024-06-13T06:55:49.222 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:55 1249 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1249_fsm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1249_vm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 06:55 1259 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1259_vm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T06:55:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T06:55:49.224 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2579 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T06:55:49.225 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T06:55:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606_vm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:55 2608 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2608_vm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2610 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T06:55:49.227 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2610_vm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T06:55:49.228 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 13 06:55 2619 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2619_fsm 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2619_vm 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T06:55:49.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:55 2658 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 2659 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2662 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2663 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2664 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2665 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2666 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2667 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T06:55:49.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:55 2673 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:55 2674 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2678 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2679 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T06:55:49.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2696 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2703 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2704 2024-06-13T06:55:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T06:55:49.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T06:55:49.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T06:55:49.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T06:55:49.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3455 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T06:55:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T06:55:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T06:55:49.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T06:55:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T06:55:49.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T06:55:49.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T06:55:49.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T06:55:49.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T06:55:49.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T06:55:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 . 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 .. 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T06:55:49.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 1247 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1247_vm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:55 1249 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1249_fsm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1249_vm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 06:55 1259 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1259_vm 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T06:55:49.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T06:55:49.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T06:55:49.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2579 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T06:55:49.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606_vm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:55 2608 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T06:55:49.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2608_vm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2610 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2610_vm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T06:55:49.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 13 06:55 2619 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2619_fsm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2619_vm 2024-06-13T06:55:49.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:55 2658 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 2659 2024-06-13T06:55:49.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2662 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2663 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2664 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2665 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2666 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2667 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:55 2673 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:55 2674 2024-06-13T06:55:49.254 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2678 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2679 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T06:55:49.255 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2696 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2703 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2704 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T06:55:49.256 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T06:55:49.257 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T06:55:49.258 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T06:55:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3455 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T06:55:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T06:55:49.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T06:55:49.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T06:55:49.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T06:55:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T06:55:49.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T06:55:49.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T06:55:49.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T06:55:49.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 . 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 .. 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 1247 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T06:55:49.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1247_vm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:55 1249 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1249_fsm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1249_vm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 06:55 1259 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1259_vm 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T06:55:49.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T06:55:49.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T06:55:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2579 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T06:55:49.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606_vm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T06:55:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:55 2608 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2608_vm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2610 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2610_vm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T06:55:49.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T06:55:49.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 13 06:55 2619 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2619_fsm 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2619_vm 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:55 2658 2024-06-13T06:55:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 2659 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2662 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2663 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2664 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2665 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2666 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2667 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:55 2673 2024-06-13T06:55:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:55 2674 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2678 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2679 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T06:55:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2696 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2703 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2704 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T06:55:49.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T06:55:49.281 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T06:55:49.282 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T06:55:49.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3455 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T06:55:49.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T06:55:49.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T06:55:49.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T06:55:49.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T06:55:49.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T06:55:49.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T06:55:49.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:total 537 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Jun 13 06:55 . 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1213 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1213_fsm 2024-06-13T06:55:49.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1213_vm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1214 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1214_fsm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1214_vm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 1232 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 1233 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1260 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1260_fsm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1260_vm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1261 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1261_fsm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1261_vm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1262 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1262_fsm 2024-06-13T06:55:49.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1262_vm 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2396 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2396_fsm 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2396_vm 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2397 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2671 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2672 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2676 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2677 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2694 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2695 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2697 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2698 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2846 2024-06-13T06:55:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2847 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2964 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2965 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2966 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2967 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3592 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3593 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4060 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4061 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4175 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4176 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4177 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4178 2024-06-13T06:55:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4181 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4182 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4183 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4184 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4185 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4186 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6000 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6001 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6002 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6100 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6114 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6115 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 pg_control 2024-06-13T06:55:49.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.296 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 . 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 mappings 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 13 06:55 replorigin_checkpoint 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 snapshots 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.297 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 .. 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 .. 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 . 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 members 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 offsets 2024-06-13T06:55:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 .. 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 0000 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 .. 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 0000 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.299 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.301 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 0000 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.302 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:total 16384 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 . 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 06:55 000000010000000000000001 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 archive_status 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T06:55:49.303 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 .. 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Jun 13 06:55 .. 2024-06-13T06:55:49.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 0000 2024-06-13T06:55:49.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-06-13T06:55:49.432 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:49 smithi007 ceph-mon[32077]: pgmap v502: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.1 MiB/s wr, 32 op/s 2024-06-13T06:55:49.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-06-13T06:55:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:49 smithi175 ceph-mon[36886]: pgmap v502: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.1 MiB/s wr, 32 op/s 2024-06-13T06:55:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:dropping old tables... 2024-06-13T06:55:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-06-13T06:55:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-06-13T06:55:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-06-13T06:55:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-06-13T06:55:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:creating tables... 2024-06-13T06:55:49.694 INFO:teuthology.orchestra.run.smithi007.stderr:generating data (client-side)... 2024-06-13T06:55:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:49 smithi189 ceph-mon[36616]: pgmap v502: 129 pgs: 129 active+clean; 25 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.1 MiB/s wr, 32 op/s 2024-06-13T06:55:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.12 s, remaining 3.74 s) 2024-06-13T06:55:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.45 s, remaining 6.72 s) 2024-06-13T06:55:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.75 s, remaining 7.23 s) 2024-06-13T06:55:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.95 s, remaining 6.63 s) 2024-06-13T06:55:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.28 s, remaining 6.89 s) 2024-06-13T06:55:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.60 s, remaining 6.93 s) 2024-06-13T06:55:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.70 s, remaining 6.06 s) 2024-06-13T06:55:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:51 smithi175 ceph-mon[36886]: pgmap v503: 129 pgs: 129 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.4 MiB/s wr, 88 op/s 2024-06-13T06:55:51.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:51 smithi007 ceph-mon[32077]: pgmap v503: 129 pgs: 129 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.4 MiB/s wr, 88 op/s 2024-06-13T06:55:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:51 smithi189 ceph-mon[36616]: pgmap v503: 129 pgs: 129 active+clean; 28 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 2.4 MiB/s wr, 88 op/s 2024-06-13T06:55:51.884 INFO:teuthology.orchestra.run.smithi007.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.15 s, remaining 6.45 s) 2024-06-13T06:55:51.985 INFO:teuthology.orchestra.run.smithi007.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.25 s, remaining 5.75 s) 2024-06-13T06:55:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.61 s, remaining 5.75 s) 2024-06-13T06:55:52.829 INFO:teuthology.orchestra.run.smithi007.stderr:1100000 of 3200000 tuples (34%) done (elapsed 3.09 s, remaining 5.91 s) 2024-06-13T06:55:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.20 s, remaining 5.34 s) 2024-06-13T06:55:53.322 INFO:teuthology.orchestra.run.smithi007.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.59 s, remaining 5.24 s) 2024-06-13T06:55:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:53 smithi175 ceph-mon[36886]: pgmap v504: 129 pgs: 129 active+clean; 70 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 9.0 MiB/s wr, 293 op/s 2024-06-13T06:55:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:53 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:55:53.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:53 smithi007 ceph-mon[32077]: pgmap v504: 129 pgs: 129 active+clean; 70 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 9.0 MiB/s wr, 293 op/s 2024-06-13T06:55:53.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:53 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:55:53.705 INFO:teuthology.orchestra.run.smithi007.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.97 s, remaining 5.11 s) 2024-06-13T06:55:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:1500000 of 3200000 tuples (46%) done (elapsed 4.09 s, remaining 4.63 s) 2024-06-13T06:55:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:53 smithi189 ceph-mon[36616]: pgmap v504: 129 pgs: 129 active+clean; 70 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 9.0 MiB/s wr, 293 op/s 2024-06-13T06:55:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:53 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:55:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.48 s, remaining 4.48 s) 2024-06-13T06:55:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.66 s, remaining 4.11 s) 2024-06-13T06:55:54.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:54 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.98 s, remaining 3.87 s) 2024-06-13T06:55:54.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:54 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:54.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:54 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:55.137 INFO:teuthology.orchestra.run.smithi007.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.40 s, remaining 3.70 s) 2024-06-13T06:55:55.342 INFO:teuthology.orchestra.run.smithi007.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.61 s, remaining 3.37 s) 2024-06-13T06:55:55.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:55 smithi007 ceph-mon[32077]: pgmap v505: 129 pgs: 129 active+clean; 120 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 16 MiB/s wr, 356 op/s 2024-06-13T06:55:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.99 s, remaining 3.14 s) 2024-06-13T06:55:55.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:55 smithi189 ceph-mon[36616]: pgmap v505: 129 pgs: 129 active+clean; 120 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 16 MiB/s wr, 356 op/s 2024-06-13T06:55:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:55 smithi175 ceph-mon[36886]: pgmap v505: 129 pgs: 129 active+clean; 120 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 16 MiB/s wr, 356 op/s 2024-06-13T06:55:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2200000 of 3200000 tuples (68%) done (elapsed 6.36 s, remaining 2.89 s) 2024-06-13T06:55:56.285 INFO:teuthology.orchestra.run.smithi007.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.55 s, remaining 2.56 s) 2024-06-13T06:55:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.93 s, remaining 2.31 s) 2024-06-13T06:55:57.027 INFO:teuthology.orchestra.run.smithi007.stderr:2500000 of 3200000 tuples (78%) done (elapsed 7.29 s, remaining 2.04 s) 2024-06-13T06:55:57.216 INFO:teuthology.orchestra.run.smithi007.stderr:2600000 of 3200000 tuples (81%) done (elapsed 7.48 s, remaining 1.73 s) 2024-06-13T06:55:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.86 s, remaining 1.46 s) 2024-06-13T06:55:57.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:57 smithi007 ceph-mon[32077]: pgmap v506: 129 pgs: 129 active+clean; 180 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 26 MiB/s wr, 358 op/s 2024-06-13T06:55:57.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:57 smithi175 ceph-mon[36886]: pgmap v506: 129 pgs: 129 active+clean; 180 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 26 MiB/s wr, 358 op/s 2024-06-13T06:55:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:57 smithi189 ceph-mon[36616]: pgmap v506: 129 pgs: 129 active+clean; 180 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 26 MiB/s wr, 358 op/s 2024-06-13T06:55:57.939 INFO:teuthology.orchestra.run.smithi007.stderr:2800000 of 3200000 tuples (87%) done (elapsed 8.21 s, remaining 1.17 s) 2024-06-13T06:55:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2900000 of 3200000 tuples (90%) done (elapsed 8.40 s, remaining 0.87 s) 2024-06-13T06:55:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:3000000 of 3200000 tuples (93%) done (elapsed 8.75 s, remaining 0.58 s) 2024-06-13T06:55:58.835 INFO:teuthology.orchestra.run.smithi007.stderr:3100000 of 3200000 tuples (96%) done (elapsed 9.10 s, remaining 0.29 s) 2024-06-13T06:55:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:3200000 of 3200000 tuples (100%) done (elapsed 9.21 s, remaining 0.00 s) 2024-06-13T06:55:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:vacuuming... 2024-06-13T06:55:59.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:59 smithi007 ceph-mon[32077]: pgmap v507: 129 pgs: 129 active+clean; 236 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 34 MiB/s wr, 357 op/s 2024-06-13T06:55:59.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:55:59 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:59 smithi189 ceph-mon[36616]: pgmap v507: 129 pgs: 129 active+clean; 236 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 34 MiB/s wr, 357 op/s 2024-06-13T06:55:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:55:59 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:55:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:59 smithi175 ceph-mon[36886]: pgmap v507: 129 pgs: 129 active+clean; 236 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 34 MiB/s wr, 357 op/s 2024-06-13T06:55:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:55:59 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:creating primary keys... 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.259+0000 7f790a000640 1 -- 172.21.15.7:0/169054624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 msgr2=0x7f7904102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.259+0000 7f790a000640 1 --2- 172.21.15.7:0/169054624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f7904102f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009860 tx=0x7f78f002a340 comp rx=0 tx=0).stop 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 -- 172.21.15.7:0/169054624 shutdown_connections 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 --2- 172.21.15.7:0/169054624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7904103470 0x7f7904109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 --2- 172.21.15.7:0/169054624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f7904102f30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 --2- 172.21.15.7:0/169054624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79041018c0 0x7f7904101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 -- 172.21.15.7:0/169054624 >> 172.21.15.7:0/169054624 conn(0x7f79040fce30 msgr2=0x7f79040ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 -- 172.21.15.7:0/169054624 shutdown_connections 2024-06-13T06:56:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.260+0000 7f790a000640 1 -- 172.21.15.7:0/169054624 wait complete. 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 Processor -- start 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 -- start start 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79041018c0 0x7f7904197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904103470 0x7f790419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f790410d4b0 con 0x7f7904102ab0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f790410d330 con 0x7f79041018c0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f790a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f790410d630 con 0x7f7904103470 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f7903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.261+0000 7f7908c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79041018c0 0x7f7904197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49416/0 (socket says 172.21.15.7:49416) 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 -- 172.21.15.7:0/2415638976 learned_addr learned my addr 172.21.15.7:0/2415638976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7909600640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904103470 0x7f790419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 -- 172.21.15.7:0/2415638976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904103470 msgr2=0x7f790419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904103470 0x7f790419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 -- 172.21.15.7:0/2415638976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79041018c0 msgr2=0x7f7904197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79041018c0 0x7f7904197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 -- 172.21.15.7:0/2415638976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790419f7c0 con 0x7f7904102ab0 2024-06-13T06:56:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.262+0000 7f7903e00640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005940 tx=0x7f78f0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.263+0000 7f7901600640 1 -- 172.21.15.7:0/2415638976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f003f020 con 0x7f7904102ab0 2024-06-13T06:56:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.263+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790419fa50 con 0x7f7904102ab0 2024-06-13T06:56:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.263+0000 7f7901600640 1 -- 172.21.15.7:0/2415638976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0030850 con 0x7f7904102ab0 2024-06-13T06:56:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.263+0000 7f7901600640 1 -- 172.21.15.7:0/2415638976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0037040 con 0x7f7904102ab0 2024-06-13T06:56:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.263+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790419ff90 con 0x7f7904102ab0 2024-06-13T06:56:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.268+0000 7f790e413740 1 Processor -- start 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f7901600640 1 -- 172.21.15.7:0/2415638976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f78f0042020 con 0x7f7904102ab0 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 -- start start 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x5568336816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556833681d30 0x5568336841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568335db420 con 0x55683367cbc0 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556833582c50 con 0x556833681d30 2024-06-13T06:56:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.269+0000 7f790e413740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5568335d73e0 con 0x55683367c1d0 2024-06-13T06:56:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7908c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x5568336816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45112/0 (socket says 172.21.15.7:45112) 2024-06-13T06:56:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 -- 172.21.15.7:0/3466333061 learned_addr learned my addr 172.21.15.7:0/3466333061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7903e00640 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556833681d30 0x5568336841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 -- 172.21.15.7:0/3466333061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556833681d30 msgr2=0x5568336841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556833681d30 0x5568336841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 -- 172.21.15.7:0/3466333061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 msgr2=0x5568336816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x5568336816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 -- 172.21.15.7:0/3466333061 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568332ef210 con 0x55683367c1d0 2024-06-13T06:56:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.270+0000 7f7909600640 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0039f0 tx=0x7f78fc003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebb675ae81231ba server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f7901600640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 0x7f78ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f78f5c00640 1 -- 172.21.15.7:0/3466333061 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc01a470 con 0x55683367c1d0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f78f5c00640 1 -- 172.21.15.7:0/3466333061 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc015070 con 0x55683367c1d0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f78f5c00640 1 -- 172.21.15.7:0/3466333061 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0108a0 con 0x55683367c1d0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f7901600640 1 -- 172.21.15.7:0/2415638976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f78f00eee80 con 0x7f7904102ab0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.271+0000 7f7908c00640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 0x7f78ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 -- 172.21.15.7:0/3466333061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 msgr2=0x55683367c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0039f0 tx=0x7f78fc003eb0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 -- 172.21.15.7:0/3466333061 shutdown_connections 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556833681d30 0x5568336841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x5568336816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 --2- 172.21.15.7:0/3466333061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367c1d0 0x55683367c5f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 -- 172.21.15.7:0/3466333061 >> 172.21.15.7:0/3466333061 conn(0x55683363ffb0 msgr2=0x556833673840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f7908c00640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 0x7f78ec0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55683367cfc0 tx=0x7f78f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 -- 172.21.15.7:0/3466333061 shutdown_connections 2024-06-13T06:56:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.272+0000 7f790e413740 1 -- 172.21.15.7:0/3466333061 wait complete. 2024-06-13T06:56:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.273+0000 7f790e413740 1 -- start start 2024-06-13T06:56:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.273+0000 7f790e413740 1 Processor -- start 2024-06-13T06:56:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833686560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367cbc0 0x556833686aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556833684ce0 con 0x556833681d30 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5568335960b0 con 0x55683367c1d0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556833684e60 con 0x55683367cbc0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f790e413740 10 client.0 _ll_register_callbacks cb 0x556833581f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.274+0000 7f7909600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7908c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833686560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7903e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367cbc0 0x556833686aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49424/0 (socket says 172.21.15.7:49424) 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7908c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833686560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52546/0 (socket says 172.21.15.7:52546) 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 -- 172.21.15.7:0/794095388 learned_addr learned my addr 172.21.15.7:0/794095388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 -- 172.21.15.7:0/794095388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367cbc0 msgr2=0x556833686aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367cbc0 0x556833686aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 -- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 msgr2=0x556833686560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833686560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.275+0000 7f7909600640 1 -- 172.21.15.7:0/794095388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568333bb620 con 0x556833681d30 2024-06-13T06:56:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f7909600640 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f78fc006010 tx=0x7f78fc004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eec024395628990 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f78ea000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:56:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02a060 con 0x556833681d30 2024-06-13T06:56:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc015070 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f78fc011650 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78fc011850 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f790e413740 10 client.15156 fetch_fsmap learned FSMap version 28 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556833690ea0 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.276+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc015070 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f78fc01a590 con 0x556833681d30 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 10 client.15156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 20 client.15156 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556833692300 0x5568336946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 4 client.15156 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.277+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568335e8b50 con 0x556833692300 2024-06-13T06:56:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.278+0000 7f7908c00640 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556833692300 0x5568336946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.278+0000 7f7908c00640 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556833692300 0x5568336946f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec67fe838d7c396 server_cookie=810a70eea4d10059 in_seq=0 out_seq=0 2024-06-13T06:56:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.278+0000 7f78ea000640 10 client.15156 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.279+0000 7f78ea000640 1 -- 172.21.15.7:0/794095388 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568335e8b50 con 0x556833692300 2024-06-13T06:56:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.279+0000 7f78ea000640 10 client.15156 handle_command_reply: tid=0 2024-06-13T06:56:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.279+0000 7f790e413740 1 client.15156 shutdown 2024-06-13T06:56:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.279+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 msgr2=0x55683368dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.279+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f78fc006010 tx=0x7f78fc004970 comp rx=0 tx=0).stop 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 shutdown_connections 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556833692300 0x5568336946f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556833681d30 0x55683368dae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55683367cbc0 0x556833686aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 --2- 172.21.15.7:0/794095388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833686560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 >> 172.21.15.7:0/794095388 conn(0x55683363ffb0 msgr2=0x5568336729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f78ea000640 0 client.15156 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 shutdown_connections 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 1 -- 172.21.15.7:0/794095388 wait complete. 2024-06-13T06:56:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.280+0000 7f790e413740 20 client.15156 trim_cache size 0 max 16384 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 Processor -- start 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 -- start start 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367cbc0 0x55683368daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x556833694850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79041aa9f0 con 0x55683367c1d0 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79041a82c0 con 0x55683367cbc0 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f790e413740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79041a6110 con 0x556833681d30 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f7908c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f7908c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49430/0 (socket says 172.21.15.7:49430) 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f7908c00640 1 -- 172.21.15.7:0/1509427660 learned_addr learned my addr 172.21.15.7:0/1509427660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f7909600640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x556833694850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.297+0000 7f7903e00640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367cbc0 0x55683368daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 -- 172.21.15.7:0/1509427660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367cbc0 msgr2=0x55683368daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367cbc0 0x55683368daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 -- 172.21.15.7:0/1509427660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 msgr2=0x556833694310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 -- 172.21.15.7:0/1509427660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568336744f0 con 0x556833681d30 2024-06-13T06:56:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7908c00640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f7909600640 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x556833694850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000f80 tx=0x7f78fc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b39cfbdf405ab18 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f78f5200640 1 -- 172.21.15.7:0/1509427660 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02a040 con 0x556833681d30 2024-06-13T06:56:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f78f5200640 1 -- 172.21.15.7:0/1509427660 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc015070 con 0x556833681d30 2024-06-13T06:56:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.298+0000 7f78f5200640 1 -- 172.21.15.7:0/1509427660 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc011c10 con 0x556833681d30 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 -- 172.21.15.7:0/1509427660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 msgr2=0x556833694850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x556833694850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000f80 tx=0x7f78fc005f90 comp rx=0 tx=0).stop 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 -- 172.21.15.7:0/1509427660 shutdown_connections 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x556833694850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367cbc0 0x55683368daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 --2- 172.21.15.7:0/1509427660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367c1d0 0x556833694310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 -- 172.21.15.7:0/1509427660 >> 172.21.15.7:0/1509427660 conn(0x55683363ffb0 msgr2=0x55683367dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.299+0000 7f790e413740 1 -- 172.21.15.7:0/1509427660 shutdown_connections 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 -- 172.21.15.7:0/1509427660 wait complete. 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 -- start start 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 Processor -- start 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833689c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x55683368d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79041a6780 con 0x55683367cbc0 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.300+0000 7f790e413740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79041a7bf0 con 0x55683367c1d0 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f790e413740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79041a6e50 con 0x556833681d30 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f790e413740 10 client.0 _ll_register_callbacks cb 0x7f78f80097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49434/0 (socket says 172.21.15.7:49434) 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 -- 172.21.15.7:0/113250829 learned_addr learned my addr 172.21.15.7:0/113250829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7908c00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833689c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7909600640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x55683368d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 -- 172.21.15.7:0/113250829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 msgr2=0x55683368d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x55683368d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 -- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 msgr2=0x556833689c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833689c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 -- 172.21.15.7:0/113250829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683368d7e0 con 0x55683367cbc0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7908c00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833689c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f7903e00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78f00b7d70 tx=0x7f78f0009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7489f9604aba2bc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.301+0000 7f78ea000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0049040 con 0x55683367cbc0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f78f00c3b80 con 0x55683367cbc0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc015070 con 0x55683367cbc0 2024-06-13T06:56:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78f00bad00 con 0x55683367cbc0 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 10 client.15162 fetch_fsmap learned FSMap version 28 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568336fba20 con 0x55683367cbc0 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc015070 con 0x55683367cbc0 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f78f003a030 con 0x55683367cbc0 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 10 client.15162 fetch_fsmap finished waiting for FSMap version 28 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 10 client.15162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 20 client.15162 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5568336fc460 0x5568336fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 4 client.15162 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.302+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568335e2ea0 con 0x5568336fc460 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.303+0000 7f7908c00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5568336fc460 0x5568336fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.303+0000 7f7908c00640 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5568336fc460 0x5568336fe880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcd0aee29deabab server_cookie=691c03e13816f619 in_seq=0 out_seq=0 2024-06-13T06:56:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.303+0000 7f78ea000640 10 client.15162 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f78ea000640 1 -- 172.21.15.7:0/113250829 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568335e2ea0 con 0x5568336fc460 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f78ea000640 10 client.15162 handle_command_reply: tid=0 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 client.15162 shutdown 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 msgr2=0x55683368a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78f00b7d70 tx=0x7f78f0009550 comp rx=0 tx=0).stop 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 shutdown_connections 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5568336fc460 0x5568336fe880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556833681d30 0x55683368d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55683367cbc0 0x55683368a1d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f78ea000640 0 client.15162 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 --2- 172.21.15.7:0/113250829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55683367c1d0 0x556833689c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 >> 172.21.15.7:0/113250829 conn(0x55683363ffb0 msgr2=0x556833694310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 shutdown_connections 2024-06-13T06:56:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 1 -- 172.21.15.7:0/113250829 wait complete. 2024-06-13T06:56:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.464+0000 7f790e413740 20 client.15162 trim_cache size 0 max 16384 2024-06-13T06:56:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:56:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:56:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:56:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 msgr2=0x7f78ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 0x7f78ec0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55683367cfc0 tx=0x7f78f8009260 comp rx=0 tx=0).stop 2024-06-13T06:56:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 msgr2=0x7f79041980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f78f0005940 tx=0x7f78f0033d90 comp rx=0 tx=0).stop 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 shutdown_connections 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f78ec0a2b80 0x7f78ec0a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904103470 0x7f790419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7904102ab0 0x7f79041980e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 --2- 172.21.15.7:0/2415638976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79041018c0 0x7f7904197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 >> 172.21.15.7:0/2415638976 conn(0x7f79040fce30 msgr2=0x7f7904105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.465+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 shutdown_connections 2024-06-13T06:56:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:01.466+0000 7f790a000640 1 -- 172.21.15.7:0/2415638976 wait complete. 2024-06-13T06:56:01.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:01 smithi007 ceph-mon[32077]: pgmap v508: 129 pgs: 129 active+clean; 264 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 39 MiB/s wr, 357 op/s 2024-06-13T06:56:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:01 smithi189 ceph-mon[36616]: pgmap v508: 129 pgs: 129 active+clean; 264 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 39 MiB/s wr, 357 op/s 2024-06-13T06:56:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:01 smithi175 ceph-mon[36886]: pgmap v508: 129 pgs: 129 active+clean; 264 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 39 MiB/s wr, 357 op/s 2024-06-13T06:56:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:done in 12.97 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 9.38 s, vacuum 1.05 s, primary keys 2.51 s). 2024-06-13T06:56:02.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-06-13T06:56:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:starting vacuum...end. 2024-06-13T06:56:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:03 smithi189 ceph-mon[36616]: pgmap v509: 129 pgs: 129 active+clean; 345 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 52 MiB/s wr, 318 op/s 2024-06-13T06:56:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:03 smithi175 ceph-mon[36886]: pgmap v509: 129 pgs: 129 active+clean; 345 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 52 MiB/s wr, 318 op/s 2024-06-13T06:56:03.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:03 smithi007 ceph-mon[32077]: pgmap v509: 129 pgs: 129 active+clean; 345 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 52 MiB/s wr, 318 op/s 2024-06-13T06:56:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:05 smithi189 ceph-mon[36616]: pgmap v510: 129 pgs: 129 active+clean; 421 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 57 MiB/s wr, 144 op/s 2024-06-13T06:56:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:05 smithi175 ceph-mon[36886]: pgmap v510: 129 pgs: 129 active+clean; 421 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 57 MiB/s wr, 144 op/s 2024-06-13T06:56:05.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:05 smithi007 ceph-mon[32077]: pgmap v510: 129 pgs: 129 active+clean; 421 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 57 MiB/s wr, 144 op/s 2024-06-13T06:56:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:06 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:06 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:06.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:06 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:56:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:07 smithi189 ceph-mon[36616]: pgmap v511: 129 pgs: 129 active+clean; 457 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 54 MiB/s wr, 102 op/s 2024-06-13T06:56:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:07 smithi175 ceph-mon[36886]: pgmap v511: 129 pgs: 129 active+clean; 457 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 54 MiB/s wr, 102 op/s 2024-06-13T06:56:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 5.0 s, 2437.6 tps, lat 39.275 ms stddev 45.892 2024-06-13T06:56:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:07 smithi007 ceph-mon[32077]: pgmap v511: 129 pgs: 129 active+clean; 457 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 54 MiB/s wr, 102 op/s 2024-06-13T06:56:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:09 smithi189 ceph-mon[36616]: pgmap v512: 129 pgs: 129 active+clean; 521 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 50 MiB/s wr, 160 op/s 2024-06-13T06:56:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:09 smithi175 ceph-mon[36886]: pgmap v512: 129 pgs: 129 active+clean; 521 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 50 MiB/s wr, 160 op/s 2024-06-13T06:56:09.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:09 smithi007 ceph-mon[32077]: pgmap v512: 129 pgs: 129 active+clean; 521 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 50 MiB/s wr, 160 op/s 2024-06-13T06:56:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:11 smithi189 ceph-mon[36616]: pgmap v513: 129 pgs: 129 active+clean; 654 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 41 MiB/s wr, 203 op/s 2024-06-13T06:56:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:11 smithi175 ceph-mon[36886]: pgmap v513: 129 pgs: 129 active+clean; 654 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 41 MiB/s wr, 203 op/s 2024-06-13T06:56:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:11 smithi007 ceph-mon[32077]: pgmap v513: 129 pgs: 129 active+clean; 654 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 41 MiB/s wr, 203 op/s 2024-06-13T06:56:12.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:56:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 10.0 s, 2010.7 tps, lat 48.088 ms stddev 51.549 2024-06-13T06:56:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:13 smithi189 ceph-mon[36616]: pgmap v514: 129 pgs: 129 active+clean; 807 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 37 MiB/s wr, 262 op/s 2024-06-13T06:56:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:13 smithi175 ceph-mon[36886]: pgmap v514: 129 pgs: 129 active+clean; 807 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 37 MiB/s wr, 262 op/s 2024-06-13T06:56:13.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:13 smithi007 ceph-mon[32077]: pgmap v514: 129 pgs: 129 active+clean; 807 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 37 MiB/s wr, 262 op/s 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.670+0000 7f8926c00640 1 -- 172.21.15.7:0/2948265731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89201035b0 msgr2=0x7f8920109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.670+0000 7f8926c00640 1 --2- 172.21.15.7:0/2948265731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89201035b0 0x7f8920109e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8918009ca0 tx=0x7f891802a8c0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.671+0000 7f8926c00640 1 -- 172.21.15.7:0/2948265731 shutdown_connections 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.672+0000 7f8926c00640 1 --2- 172.21.15.7:0/2948265731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89201035b0 0x7f8920109e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.672+0000 7f8926c00640 1 --2- 172.21.15.7:0/2948265731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920102bf0 0x7f8920103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.672+0000 7f8926c00640 1 --2- 172.21.15.7:0/2948265731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8920101a00 0x7f8920101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.672+0000 7f8926c00640 1 -- 172.21.15.7:0/2948265731 >> 172.21.15.7:0/2948265731 conn(0x7f89200fcf90 msgr2=0x7f89200ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.675+0000 7f8926c00640 1 -- 172.21.15.7:0/2948265731 shutdown_connections 2024-06-13T06:56:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.675+0000 7f8926c00640 1 -- 172.21.15.7:0/2948265731 wait complete. 2024-06-13T06:56:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.677+0000 7f8926c00640 1 Processor -- start 2024-06-13T06:56:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.678+0000 7f8926c00640 1 -- start start 2024-06-13T06:56:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.678+0000 7f8926c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920101a00 0x7f8920110d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.678+0000 7f8926c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f8926c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 0x7f89201162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f8926c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f892010b510 con 0x7f8920101a00 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f8926c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f892010b390 con 0x7f8920102bf0 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f8926c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f892010b690 con 0x7f89201035b0 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49816/0 (socket says 172.21.15.7:49816) 2024-06-13T06:56:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.679+0000 7f891f400640 1 -- 172.21.15.7:0/547072173 learned_addr learned my addr 172.21.15.7:0/547072173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 -- 172.21.15.7:0/547072173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 msgr2=0x7f89201162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891fe00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920101a00 0x7f8920110d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f8924c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 0x7f89201162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 0x7f89201162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 -- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920101a00 msgr2=0x7f8920110d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920101a00 0x7f8920110d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 -- 172.21.15.7:0/547072173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8920116920 con 0x7f8920102bf0 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.680+0000 7f891f400640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f890c00b4a0 tx=0x7f890c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f8924c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 0x7f89201162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f891cc00640 1 -- 172.21.15.7:0/547072173 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c014070 con 0x7f8920102bf0 2024-06-13T06:56:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f891cc00640 1 -- 172.21.15.7:0/547072173 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c010af0 con 0x7f8920102bf0 2024-06-13T06:56:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f891cc00640 1 -- 172.21.15.7:0/547072173 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c017cc0 con 0x7f8920102bf0 2024-06-13T06:56:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8920116c10 con 0x7f8920102bf0 2024-06-13T06:56:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.681+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89201aefd0 con 0x7f8920102bf0 2024-06-13T06:56:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.690+0000 7f89299ae740 1 Processor -- start 2024-06-13T06:56:14.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.691+0000 7f891cc00640 1 -- 172.21.15.7:0/547072173 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f890c017070 con 0x7f8920102bf0 2024-06-13T06:56:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.691+0000 7f89299ae740 1 -- start start 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2941c00 0x55f6e2942020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 0x55f6e29470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6e28a5420 con 0x55f6e29425f0 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6e284cc50 con 0x55f6e2947760 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f89299ae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6e28a13e0 con 0x55f6e2941c00 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49818/0 (socket says 172.21.15.7:49818) 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.692+0000 7f891f400640 1 -- 172.21.15.7:0/1121781724 learned_addr learned my addr 172.21.15.7:0/1121781724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.693+0000 7f891f400640 1 -- 172.21.15.7:0/1121781724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2941c00 msgr2=0x55f6e2942020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.693+0000 7f891fe00640 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 0x55f6e29470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.693+0000 7f891f400640 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2941c00 0x55f6e2942020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.693+0000 7f891f400640 1 -- 172.21.15.7:0/1121781724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 msgr2=0x55f6e29470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.694+0000 7f891f400640 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 0x55f6e29470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.694+0000 7f891f400640 1 -- 172.21.15.7:0/1121781724 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e25b9210 con 0x55f6e2947760 2024-06-13T06:56:14.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.694+0000 7f891fe00640 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 0x55f6e29470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.695+0000 7f891f400640 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f890c0d6480 tx=0x7f890c0d6940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c870a86d96c4163e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.698+0000 7f8915200640 1 -- 172.21.15.7:0/1121781724 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0e1070 con 0x55f6e2947760 2024-06-13T06:56:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.698+0000 7f8915200640 1 -- 172.21.15.7:0/1121781724 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c0ceaa0 con 0x55f6e2947760 2024-06-13T06:56:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.698+0000 7f8915200640 1 -- 172.21.15.7:0/1121781724 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0dcaa0 con 0x55f6e2947760 2024-06-13T06:56:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.701+0000 7f89299ae740 1 -- 172.21.15.7:0/1121781724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 msgr2=0x55f6e2949bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.701+0000 7f89299ae740 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f890c0d6480 tx=0x7f890c0d6940 comp rx=0 tx=0).stop 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.701+0000 7f891cc00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 0x7f89040a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.701+0000 7f891cc00640 1 -- 172.21.15.7:0/547072173 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f890c0ce110 con 0x7f8920102bf0 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f89299ae740 1 -- 172.21.15.7:0/1121781724 shutdown_connections 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f89299ae740 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e2947760 0x55f6e2949bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f89299ae740 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e29425f0 0x55f6e29470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f89299ae740 1 --2- 172.21.15.7:0/1121781724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2941c00 0x55f6e2942020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f89299ae740 1 -- 172.21.15.7:0/1121781724 >> 172.21.15.7:0/1121781724 conn(0x55f6e2909fb0 msgr2=0x55f6e293d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.702+0000 7f8924c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 0x7f89040a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.703+0000 7f89299ae740 1 -- 172.21.15.7:0/1121781724 shutdown_connections 2024-06-13T06:56:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.703+0000 7f89299ae740 1 -- 172.21.15.7:0/1121781724 wait complete. 2024-06-13T06:56:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.704+0000 7f89299ae740 1 -- start start 2024-06-13T06:56:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.704+0000 7f8924c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 0x7f89040a5260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f6e2942e50 tx=0x7f8918009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.704+0000 7f89299ae740 1 Processor -- start 2024-06-13T06:56:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.705+0000 7f89299ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.705+0000 7f89299ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e294c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.705+0000 7f89299ae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2947760 0x55f6e2953540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.705+0000 7f891fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f89299ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f6e294a710 con 0x55f6e2941c00 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f89299ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6e28600b0 con 0x55f6e29425f0 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f89299ae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6e294a890 con 0x55f6e2947760 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e294c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42158/0 (socket says 172.21.15.7:42158) 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891fe00640 1 -- 172.21.15.7:0/3187529734 learned_addr learned my addr 172.21.15.7:0/3187529734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f89299ae740 10 client.0 _ll_register_callbacks cb 0x55f6e284bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f8924c00640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2947760 0x55f6e2953540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891fe00640 1 -- 172.21.15.7:0/3187529734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2947760 msgr2=0x55f6e2953540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891fe00640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2947760 0x55f6e2953540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.706+0000 7f891fe00640 1 -- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 msgr2=0x55f6e294c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.707+0000 7f891fe00640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e294c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.707+0000 7f891fe00640 1 -- 172.21.15.7:0/3187529734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e2685620 con 0x55f6e2941c00 2024-06-13T06:56:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.707+0000 7f891f400640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e294c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.708+0000 7f891fe00640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8910011690 tx=0x7f8910011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fbc1c8a6f35765 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.708+0000 7f8909600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:56:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.709+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910004220 con 0x55f6e2941c00 2024-06-13T06:56:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.709+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f89100043c0 con 0x55f6e2941c00 2024-06-13T06:56:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.709+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c0ceaa0 con 0x55f6e2941c00 2024-06-13T06:56:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.710+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8910011e90 con 0x55f6e2941c00 2024-06-13T06:56:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.710+0000 7f89299ae740 10 client.15165 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.710+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6e29568f0 con 0x55f6e2941c00 2024-06-13T06:56:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.710+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c0ceaa0 con 0x55f6e2941c00 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f89100047c0 con 0x55f6e2941c00 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 10 client.15165 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 10 client.15165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 20 client.15165 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e2957ff0 0x55f6e295a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 4 client.15165 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.711+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6e28b2b50 con 0x55f6e2957ff0 2024-06-13T06:56:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.712+0000 7f891f400640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e2957ff0 0x55f6e295a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.713+0000 7f891f400640 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e2957ff0 0x55f6e295a4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f95520f8a2ca6c server_cookie=52687cbd4a0a6268 in_seq=0 out_seq=0 2024-06-13T06:56:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.713+0000 7f8909600640 10 client.15165 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.714+0000 7f8909600640 1 -- 172.21.15.7:0/3187529734 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f6e28b2b50 con 0x55f6e2957ff0 2024-06-13T06:56:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.714+0000 7f8909600640 10 client.15165 handle_command_reply: tid=0 2024-06-13T06:56:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.714+0000 7f89299ae740 1 client.15165 shutdown 2024-06-13T06:56:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 msgr2=0x55f6e294bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8910011690 tx=0x7f8910011b50 comp rx=0 tx=0).stop 2024-06-13T06:56:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 shutdown_connections 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e2957ff0 0x55f6e295a4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e2947760 0x55f6e2953540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e294c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 --2- 172.21.15.7:0/3187529734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2941c00 0x55f6e294bfc0 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8910011690 tx=0x7f8910011b50 comp rx=0 tx=0).stop 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 >> 172.21.15.7:0/3187529734 conn(0x55f6e2909fb0 msgr2=0x55f6e293c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.715+0000 7f8909600640 0 client.15165 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.716+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 shutdown_connections 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.716+0000 7f89299ae740 1 -- 172.21.15.7:0/3187529734 wait complete. 2024-06-13T06:56:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.716+0000 7f89299ae740 20 client.15165 trim_cache size 0 max 16384 2024-06-13T06:56:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.748+0000 7f89299ae740 1 Processor -- start 2024-06-13T06:56:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.748+0000 7f89299ae740 1 -- start start 2024-06-13T06:56:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.748+0000 7f89299ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e2942e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.748+0000 7f89299ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e2953500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f89299ae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 0x55f6e29405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f89299ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89201bbd50 con 0x55f6e2947760 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f89299ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89201b95f0 con 0x55f6e29425f0 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f89299ae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89201b7440 con 0x55f6e29440c0 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f8924c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 0x55f6e29405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f8924c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 0x55f6e29405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56910/0 (socket says 172.21.15.7:56910) 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.749+0000 7f8924c00640 1 -- 172.21.15.7:0/200972861 learned_addr learned my addr 172.21.15.7:0/200972861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891f400640 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e2953500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e2942e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 -- 172.21.15.7:0/200972861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 msgr2=0x55f6e29405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 0x55f6e29405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 -- 172.21.15.7:0/200972861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 msgr2=0x55f6e2953500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e2953500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.750+0000 7f891fe00640 1 -- 172.21.15.7:0/200972861 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e293e330 con 0x55f6e29425f0 2024-06-13T06:56:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.751+0000 7f891fe00640 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e2942e80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55f6e293e500 tx=0x7f8910003660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340801632306c849 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.751+0000 7f890be00640 1 -- 172.21.15.7:0/200972861 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910028040 con 0x55f6e29425f0 2024-06-13T06:56:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.752+0000 7f890be00640 1 -- 172.21.15.7:0/200972861 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8910011e90 con 0x55f6e29425f0 2024-06-13T06:56:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.752+0000 7f890be00640 1 -- 172.21.15.7:0/200972861 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910004a00 con 0x55f6e29425f0 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- 172.21.15.7:0/200972861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 msgr2=0x55f6e2942e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e2942e80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55f6e293e500 tx=0x7f8910003660 comp rx=0 tx=0).stop 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- 172.21.15.7:0/200972861 shutdown_connections 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29440c0 0x55f6e29405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e2953500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 --2- 172.21.15.7:0/200972861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29425f0 0x55f6e2942e80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- 172.21.15.7:0/200972861 >> 172.21.15.7:0/200972861 conn(0x55f6e2909fb0 msgr2=0x55f6e29439b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- 172.21.15.7:0/200972861 shutdown_connections 2024-06-13T06:56:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- 172.21.15.7:0/200972861 wait complete. 2024-06-13T06:56:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.753+0000 7f89299ae740 1 -- start start 2024-06-13T06:56:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.754+0000 7f89299ae740 1 Processor -- start 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 0x55f6e2950170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e29c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89201b7ab0 con 0x55f6e2947760 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89201b8f20 con 0x55f6e29440c0 2024-06-13T06:56:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89201b8180 con 0x55f6e29425f0 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.755+0000 7f89299ae740 10 client.0 _ll_register_callbacks cb 0x7f8910009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 0x55f6e2950170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49846/0 (socket says 172.21.15.7:49846) 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 -- 172.21.15.7:0/1883669555 learned_addr learned my addr 172.21.15.7:0/1883669555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 -- 172.21.15.7:0/1883669555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 msgr2=0x55f6e2950170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 0x55f6e2950170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 -- 172.21.15.7:0/1883669555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 msgr2=0x55f6e29c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e29c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.756+0000 7f891f400640 1 -- 172.21.15.7:0/1883669555 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e2953780 con 0x55f6e29440c0 2024-06-13T06:56:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f891f400640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f890c096a90 tx=0x7f890c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8411c87346b9e24 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f8909600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f891fe00640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 0x55f6e2950170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c01e040 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8910011e90 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f890c098930 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.757+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f890c0968d0 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.758+0000 7f89299ae740 10 client.25304 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.758+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6e29c2340 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.758+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8910011e90 con 0x55f6e29440c0 2024-06-13T06:56:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f890c096ca0 con 0x55f6e29440c0 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 10 client.25304 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 10 client.25304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 20 client.25304 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e29c2770 0x55f6e29c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 4 client.25304 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.759+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6e28acea0 con 0x55f6e29c2770 2024-06-13T06:56:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.760+0000 7f891fe00640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e29c2770 0x55f6e29c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.761+0000 7f891fe00640 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e29c2770 0x55f6e29c4b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94eba729a2aaf2cb server_cookie=98b868da82f98d8 in_seq=0 out_seq=0 2024-06-13T06:56:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.762+0000 7f8909600640 10 client.25304 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.790+0000 7f8909600640 1 -- 172.21.15.7:0/1883669555 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f6e28acea0 con 0x55f6e29c2770 2024-06-13T06:56:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.790+0000 7f8909600640 10 client.25304 handle_command_reply: tid=0 2024-06-13T06:56:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.791+0000 7f89299ae740 1 client.25304 shutdown 2024-06-13T06:56:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.793+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 msgr2=0x55f6e29506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.793+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f890c096a90 tx=0x7f890c0046c0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 shutdown_connections 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6e29c2770 0x55f6e29c4b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6e2947760 0x55f6e29c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6e29440c0 0x55f6e29506b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 --2- 172.21.15.7:0/1883669555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6e29425f0 0x55f6e2950170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f8909600640 0 client.25304 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 >> 172.21.15.7:0/1883669555 conn(0x55f6e2909fb0 msgr2=0x55f6e29405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 shutdown_connections 2024-06-13T06:56:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.794+0000 7f89299ae740 1 -- 172.21.15.7:0/1883669555 wait complete. 2024-06-13T06:56:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.795+0000 7f89299ae740 20 client.25304 trim_cache size 0 max 16384 2024-06-13T06:56:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:56:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:56:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 msgr2=0x7f89040a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 0x7f89040a5260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f6e2942e50 tx=0x7f8918009800 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 msgr2=0x7f8920111250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f890c00b4a0 tx=0x7f890c00b960 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 shutdown_connections 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89040a2da0 0x7f89040a5260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89201035b0 0x7f89201162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8920102bf0 0x7f8920111250 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 --2- 172.21.15.7:0/547072173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8920101a00 0x7f8920110d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 >> 172.21.15.7:0/547072173 conn(0x7f89200fcf90 msgr2=0x7f8920105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 shutdown_connections 2024-06-13T06:56:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:14.797+0000 7f8926c00640 1 -- 172.21.15.7:0/547072173 wait complete. 2024-06-13T06:56:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:15 smithi189 ceph-mon[36616]: pgmap v515: 129 pgs: 129 active+clean; 898 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 25 MiB/s wr, 414 op/s 2024-06-13T06:56:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:15 smithi175 ceph-mon[36886]: pgmap v515: 129 pgs: 129 active+clean; 898 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 25 MiB/s wr, 414 op/s 2024-06-13T06:56:15.875 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:15 smithi007 ceph-mon[32077]: pgmap v515: 129 pgs: 129 active+clean; 898 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 25 MiB/s wr, 414 op/s 2024-06-13T06:56:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.901+0000 7f0df7000700 1 -- 172.21.15.7:0/1330307337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 msgr2=0x7f0df0109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.901+0000 7f0def400700 1 -- 172.21.15.7:0/1330307337 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0004230 con 0x7f0df010cb80 2024-06-13T06:56:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.901+0000 7f0df7000700 1 --2- 172.21.15.7:0/1330307337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df0109490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009990 tx=0x7f0de0009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.902+0000 7f0df7000700 1 -- 172.21.15.7:0/1330307337 shutdown_connections 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.902+0000 7f0df7000700 1 --2- 172.21.15.7:0/1330307337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df0117520 0x7f0df0119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.902+0000 7f0df7000700 1 --2- 172.21.15.7:0/1330307337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df0109490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.902+0000 7f0df7000700 1 --2- 172.21.15.7:0/1330307337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df01099d0 0x7f0df0109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.902+0000 7f0df7000700 1 -- 172.21.15.7:0/1330307337 >> 172.21.15.7:0/1330307337 conn(0x7f0df006d9e0 msgr2=0x7f0df006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.903+0000 7f0df7000700 1 -- 172.21.15.7:0/1330307337 shutdown_connections 2024-06-13T06:56:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.905+0000 7f0df7000700 1 -- 172.21.15.7:0/1330307337 wait complete. 2024-06-13T06:56:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.906+0000 7f0df7000700 1 Processor -- start 2024-06-13T06:56:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.907+0000 7f0df7000700 1 -- start start 2024-06-13T06:56:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.908+0000 7f0df7000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.908+0000 7f0df7000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.908+0000 7f0df7000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 0x7f0df01a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0df7000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df011d250 con 0x7f0df0117520 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0de7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0defe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0de7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49868/0 (socket says 172.21.15.7:49868) 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0defe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56934/0 (socket says 172.21.15.7:56934) 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0de7400700 1 -- 172.21.15.7:0/3705452297 learned_addr learned my addr 172.21.15.7:0/3705452297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0df5000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 0x7f0df01a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0df011d0d0 con 0x7f0df010cb80 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.909+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0df011d3d0 con 0x7f0df01099d0 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 -- 172.21.15.7:0/3705452297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 msgr2=0x7f0df01a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 -- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 msgr2=0x7f0df01a7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 0x7f0df01a7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 -- 172.21.15.7:0/3705452297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df01a83d0 con 0x7f0df010cb80 2024-06-13T06:56:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0defe00700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0df5000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 0x7f0df01a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0de7400700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d950 tx=0x7f0dd800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.910+0000 7f0ded600700 1 -- 172.21.15.7:0/3705452297 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8016070 con 0x7f0df010cb80 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.911+0000 7f0ded600700 1 -- 172.21.15.7:0/3705452297 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd8009b40 con 0x7f0df010cb80 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.911+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df01a86c0 con 0x7f0df010cb80 2024-06-13T06:56:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.911+0000 7f0ded600700 1 -- 172.21.15.7:0/3705452297 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800f620 con 0x7f0df010cb80 2024-06-13T06:56:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.911+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df01aff40 con 0x7f0df010cb80 2024-06-13T06:56:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.920+0000 7f0ded600700 1 -- 172.21.15.7:0/3705452297 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0dd8019070 con 0x7f0df010cb80 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.922+0000 7f0ded600700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 0x7f0dd40a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.923+0000 7f0ded600700 1 -- 172.21.15.7:0/3705452297 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0dd80cf320 con 0x7f0df010cb80 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.923+0000 7f0defe00700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 0x7f0dd40a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.923+0000 7f0e01992b80 1 Processor -- start 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.923+0000 7f0e01992b80 1 -- start start 2024-06-13T06:56:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f394cb0 0x55ea9f38cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f38d760 0x55ea9f3ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f29ea10 con 0x55ea9f38d760 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f2e1b00 con 0x55ea9f394cb0 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0e01992b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f29c8f0 con 0x55ea9f395090 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0de7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0de7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56938/0 (socket says 172.21.15.7:56938) 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.924+0000 7f0de7400700 1 -- 172.21.15.7:0/241406159 learned_addr learned my addr 172.21.15.7:0/241406159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.925+0000 7f0de7400700 1 -- 172.21.15.7:0/241406159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f394cb0 msgr2=0x55ea9f38cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.925+0000 7f0de7400700 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f394cb0 0x55ea9f38cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.925+0000 7f0de7400700 1 -- 172.21.15.7:0/241406159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f38d760 msgr2=0x55ea9f3ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.925+0000 7f0defe00700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 0x7f0dd40a5120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005cb0 tx=0x7f0de001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.925+0000 7f0df5000700 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f38d760 0x55ea9f3ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de7400700 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f38d760 0x55ea9f3ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de7400700 1 -- 172.21.15.7:0/241406159 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9f289f90 con 0x55ea9f395090 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de7400700 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80a4020 tx=0x7f0dd809b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237b0dcf39a39287 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de5600700 1 -- 172.21.15.7:0/241406159 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8020070 con 0x55ea9f395090 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de5600700 1 -- 172.21.15.7:0/241406159 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd809fc90 con 0x55ea9f395090 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.926+0000 7f0de5600700 1 -- 172.21.15.7:0/241406159 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80862b0 con 0x55ea9f395090 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.927+0000 7f0e01992b80 1 -- 172.21.15.7:0/241406159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 msgr2=0x55ea9f38d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.927+0000 7f0e01992b80 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80a4020 tx=0x7f0dd809b2c0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.928+0000 7f0e01992b80 1 -- 172.21.15.7:0/241406159 shutdown_connections 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.928+0000 7f0e01992b80 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f38d760 0x55ea9f3ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.928+0000 7f0e01992b80 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f394cb0 0x55ea9f38cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.928+0000 7f0e01992b80 1 --2- 172.21.15.7:0/241406159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f38d220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.928+0000 7f0e01992b80 1 -- 172.21.15.7:0/241406159 >> 172.21.15.7:0/241406159 conn(0x55ea9f2bc170 msgr2=0x55ea9f38aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:16.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.929+0000 7f0e01992b80 1 -- 172.21.15.7:0/241406159 shutdown_connections 2024-06-13T06:56:16.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.929+0000 7f0e01992b80 1 -- 172.21.15.7:0/241406159 wait complete. 2024-06-13T06:56:16.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.931+0000 7f0e01992b80 1 -- start start 2024-06-13T06:56:16.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.931+0000 7f0e01992b80 1 Processor -- start 2024-06-13T06:56:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 0x55ea9f399b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0defe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0de7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.933+0000 7f0de7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56948/0 (socket says 172.21.15.7:56948) 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0defe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49876/0 (socket says 172.21.15.7:49876) 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0de7400700 1 -- 172.21.15.7:0/2136761693 learned_addr learned my addr 172.21.15.7:0/2136761693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f27df60 con 0x55ea9f395090 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f2ddac0 con 0x55ea9f38d760 2024-06-13T06:56:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ea9f3987e0 con 0x55ea9f394cb0 2024-06-13T06:56:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0e01992b80 10 client.0 _ll_register_callbacks cb 0x55ea9f277380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.934+0000 7f0df5000700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 0x55ea9f399b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0de7400700 1 -- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 msgr2=0x55ea9f39e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0de7400700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0de7400700 1 -- 172.21.15.7:0/2136761693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 msgr2=0x55ea9f399b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0de7400700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 0x55ea9f399b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0de7400700 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9f2308d0 con 0x55ea9f394cb0 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.936+0000 7f0df5000700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 0x55ea9f399b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.937+0000 7f0defe00700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0de7400700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dd809b870 tx=0x7f0dd8098200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8abbe83e45bd6b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0dde000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8020070 con 0x55ea9f394cb0 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd809fc90 con 0x55ea9f394cb0 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0dd80862c0 con 0x55ea9f394cb0 2024-06-13T06:56:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.938+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0dd80cf170 con 0x55ea9f394cb0 2024-06-13T06:56:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.939+0000 7f0e01992b80 10 client.25306 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.939+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea9f3a72b0 con 0x55ea9f394cb0 2024-06-13T06:56:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.939+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd809fc90 con 0x55ea9f394cb0 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f0dd80a40c0 con 0x55ea9f394cb0 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 10 client.25306 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 20 client.25306 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f3a9780 0x55ea9f3abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 4 client.25306 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.941+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea9f2ef000 con 0x55ea9f3a9780 2024-06-13T06:56:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.942+0000 7f0defe00700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f3a9780 0x55ea9f3abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.943+0000 7f0defe00700 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f3a9780 0x55ea9f3abba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983102a3128693c5 server_cookie=d44daaa99ccb7bc1 in_seq=0 out_seq=0 2024-06-13T06:56:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.944+0000 7f0dde000700 10 client.25306 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.944+0000 7f0dde000700 1 -- 172.21.15.7:0/2136761693 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea9f2ef000 con 0x55ea9f3a9780 2024-06-13T06:56:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.944+0000 7f0dde000700 10 client.25306 handle_command_reply: tid=0 2024-06-13T06:56:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 client.25306 shutdown 2024-06-13T06:56:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 msgr2=0x55ea9f39ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dd809b870 tx=0x7f0dd8098200 comp rx=0 tx=0).stop 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 shutdown_connections 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f3a9780 0x55ea9f3abba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f395090 0x55ea9f399b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f39e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 --2- 172.21.15.7:0/2136761693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f394cb0 0x55ea9f39ecb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 >> 172.21.15.7:0/2136761693 conn(0x55ea9f2bc170 msgr2=0x55ea9f384790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.945+0000 7f0dde000700 0 client.25306 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.946+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 shutdown_connections 2024-06-13T06:56:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.946+0000 7f0e01992b80 1 -- 172.21.15.7:0/2136761693 wait complete. 2024-06-13T06:56:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.946+0000 7f0e01992b80 20 client.25306 trim_cache size 0 max 16384 2024-06-13T06:56:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 Processor -- start 2024-06-13T06:56:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 -- start start 2024-06-13T06:56:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f385130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f39e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b6ea0 con 0x55ea9f394cb0 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b4700 con 0x55ea9f38d760 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.988+0000 7f0e01992b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b2580 con 0x55ea9f395090 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 -- 172.21.15.7:0/4275975429 learned_addr learned my addr 172.21.15.7:0/4275975429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 -- 172.21.15.7:0/4275975429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 msgr2=0x55ea9f39e330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f39e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.989+0000 7f0defe00700 1 -- 172.21.15.7:0/4275975429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 msgr2=0x55ea9f385130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.990+0000 7f0de7400700 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f385130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.990+0000 7f0defe00700 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f385130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.990+0000 7f0defe00700 1 -- 172.21.15.7:0/4275975429 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9f38d590 con 0x55ea9f38d760 2024-06-13T06:56:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.990+0000 7f0de7400700 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f385130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.991+0000 7f0defe00700 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098e0 tx=0x7f0de001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c5ba118161166c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.991+0000 7f0de4c00700 1 -- 172.21.15.7:0/4275975429 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de000fdd0 con 0x55ea9f38d760 2024-06-13T06:56:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.992+0000 7f0de4c00700 1 -- 172.21.15.7:0/4275975429 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea9f2ef000 con 0x55ea9f38d760 2024-06-13T06:56:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.992+0000 7f0de4c00700 1 -- 172.21.15.7:0/4275975429 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0023840 con 0x55ea9f38d760 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 -- 172.21.15.7:0/4275975429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 msgr2=0x55ea9f397ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098e0 tx=0x7f0de001e000 comp rx=0 tx=0).stop 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 -- 172.21.15.7:0/4275975429 shutdown_connections 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f395090 0x55ea9f39e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f38d760 0x55ea9f397ec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 --2- 172.21.15.7:0/4275975429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f385130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.993+0000 7f0e01992b80 1 -- 172.21.15.7:0/4275975429 >> 172.21.15.7:0/4275975429 conn(0x55ea9f2bc170 msgr2=0x55ea9f38ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.994+0000 7f0e01992b80 1 -- 172.21.15.7:0/4275975429 shutdown_connections 2024-06-13T06:56:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.994+0000 7f0e01992b80 1 -- 172.21.15.7:0/4275975429 wait complete. 2024-06-13T06:56:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.995+0000 7f0e01992b80 1 -- start start 2024-06-13T06:56:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.995+0000 7f0e01992b80 1 Processor -- start 2024-06-13T06:56:16.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.998+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0e01992b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f3a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0defe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0defe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56970/0 (socket says 172.21.15.7:56970) 2024-06-13T06:56:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0defe00700 1 -- 172.21.15.7:0/3565524140 learned_addr learned my addr 172.21.15.7:0/3565524140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f395090 0x55ea9f3a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:16.999+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b2bc0 con 0x55ea9f394cb0 2024-06-13T06:56:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.000+0000 7f0de7400700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f3a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.000+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b4030 con 0x55ea9f395090 2024-06-13T06:56:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.000+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01b3290 con 0x55ea9f38d760 2024-06-13T06:56:16.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.000+0000 7f0df5000700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f395090 0x55ea9f3a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.000+0000 7f0e01992b80 10 client.0 _ll_register_callbacks cb 0x55ea9f277380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 -- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f395090 msgr2=0x55ea9f3a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f395090 0x55ea9f3a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 -- 172.21.15.7:0/3565524140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 msgr2=0x55ea9f3a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f3a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9f434b30 con 0x55ea9f38d760 2024-06-13T06:56:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0de7400700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f3a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0defe00700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005c40 tx=0x7f0de000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997e3dcff824060 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.001+0000 7f0dde000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.002+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de003e060 con 0x55ea9f38d760 2024-06-13T06:56:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.002+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0de0005460 con 0x55ea9f38d760 2024-06-13T06:56:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.002+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea9f2ef000 con 0x55ea9f38d760 2024-06-13T06:56:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.003+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0de0045050 con 0x55ea9f38d760 2024-06-13T06:56:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.003+0000 7f0e01992b80 10 client.25309 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.003+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea9f435d10 con 0x55ea9f38d760 2024-06-13T06:56:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.003+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea9f2ef000 con 0x55ea9f38d760 2024-06-13T06:56:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.004+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f0de0004a20 con 0x55ea9f38d760 2024-06-13T06:56:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 10 client.25309 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 20 client.25309 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f436310 0x55ea9f438730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 4 client.25309 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea9f2e9580 con 0x55ea9f436310 2024-06-13T06:56:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.005+0000 7f0de7400700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f436310 0x55ea9f438730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.007+0000 7f0de7400700 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f436310 0x55ea9f438730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24639179786df0f server_cookie=f7ab51a1424f0d5e in_seq=0 out_seq=0 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.007+0000 7f0dde000700 10 client.25309 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0dde000700 1 -- 172.21.15.7:0/3565524140 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea9f2e9580 con 0x55ea9f436310 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0dde000700 10 client.25309 handle_command_reply: tid=0 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 client.25309 shutdown 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 msgr2=0x55ea9f3a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005c40 tx=0x7f0de000b730 comp rx=0 tx=0).stop 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 shutdown_connections 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ea9f436310 0x55ea9f438730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ea9f395090 0x55ea9f3a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea9f38d760 0x55ea9f3a19f0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005c40 tx=0x7f0de000b730 comp rx=0 tx=0).stop 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 --2- 172.21.15.7:0/3565524140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea9f394cb0 0x55ea9f3a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.008+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 >> 172.21.15.7:0/3565524140 conn(0x55ea9f2bc170 msgr2=0x55ea9f384120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:17.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.009+0000 7f0dde000700 0 client.25309 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:17.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.009+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 shutdown_connections 2024-06-13T06:56:17.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.009+0000 7f0e01992b80 1 -- 172.21.15.7:0/3565524140 wait complete. 2024-06-13T06:56:17.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.010+0000 7f0e01992b80 20 client.25309 trim_cache size 0 max 16384 2024-06-13T06:56:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.011+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 msgr2=0x7f0dd40a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.011+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 0x7f0dd40a5120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0de0005cb0 tx=0x7f0de001a040 comp rx=0 tx=0).stop 2024-06-13T06:56:17.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.011+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 msgr2=0x7f0df01a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.011+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800d950 tx=0x7f0dd800dd10 comp rx=0 tx=0).stop 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 shutdown_connections 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df0117520 0x7f0df01a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0df01099d0 0x7f0df01a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0dd40a2c60 0x7f0dd40a5120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 --2- 172.21.15.7:0/3705452297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df010cb80 0x7f0df01a3900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 >> 172.21.15.7:0/3705452297 conn(0x7f0df006d9e0 msgr2=0x7f0df0118980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 shutdown_connections 2024-06-13T06:56:17.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:17.012+0000 7f0df7000700 1 -- 172.21.15.7:0/3705452297 wait complete. 2024-06-13T06:56:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:17 smithi189 ceph-mon[36616]: pgmap v516: 129 pgs: 129 active+clean; 905 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 461 op/s 2024-06-13T06:56:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:17 smithi175 ceph-mon[36886]: pgmap v516: 129 pgs: 129 active+clean; 905 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 461 op/s 2024-06-13T06:56:17.868 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:17 smithi007 ceph-mon[32077]: pgmap v516: 129 pgs: 129 active+clean; 905 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 461 op/s 2024-06-13T06:56:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 15.0 s, 1760.4 tps, lat 58.404 ms stddev 80.588 2024-06-13T06:56:17.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T06:56:17.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:56:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:19 smithi189 ceph-mon[36616]: pgmap v517: 129 pgs: 129 active+clean; 921 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 556 op/s 2024-06-13T06:56:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:19 smithi175 ceph-mon[36886]: pgmap v517: 129 pgs: 129 active+clean; 921 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 556 op/s 2024-06-13T06:56:19.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:19 smithi007 ceph-mon[32077]: pgmap v517: 129 pgs: 129 active+clean; 921 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 556 op/s 2024-06-13T06:56:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.036+0000 7fb86fc00700 1 -- 172.21.15.7:0/1115527623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 msgr2=0x7fb868115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.036+0000 7fb86fc00700 1 --2- 172.21.15.7:0/1115527623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb868115a50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb860009f90 tx=0x7fb86000b330 comp rx=0 tx=0).stop 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.037+0000 7fb86fc00700 1 -- 172.21.15.7:0/1115527623 shutdown_connections 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.037+0000 7fb86fc00700 1 --2- 172.21.15.7:0/1115527623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb868115a50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.037+0000 7fb86fc00700 1 --2- 172.21.15.7:0/1115527623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8680746c0 0x7fb868074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.037+0000 7fb86fc00700 1 --2- 172.21.15.7:0/1115527623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 0x7fb868113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.037+0000 7fb86fc00700 1 -- 172.21.15.7:0/1115527623 >> 172.21.15.7:0/1115527623 conn(0x7fb868079640 msgr2=0x7fb868079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.038+0000 7fb86fc00700 1 -- 172.21.15.7:0/1115527623 shutdown_connections 2024-06-13T06:56:21.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.038+0000 7fb86fc00700 1 -- 172.21.15.7:0/1115527623 wait complete. 2024-06-13T06:56:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.040+0000 7fb86fc00700 1 Processor -- start 2024-06-13T06:56:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.040+0000 7fb86fc00700 1 -- start start 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8680746c0 0x7fb868112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 0x7fb868113040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb868118df0 con 0x7fb8680746c0 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb868118c70 con 0x7fb868074fe0 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.041+0000 7fb86fc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb868118f70 con 0x7fb868113660 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35748/0 (socket says 172.21.15.7:35748) 2024-06-13T06:56:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 -- 172.21.15.7:0/3565455787 learned_addr learned my addr 172.21.15.7:0/3565455787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb867e00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 0x7fb868113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 -- 172.21.15.7:0/3565455787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 msgr2=0x7fb868113040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 0x7fb868113040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 -- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8680746c0 msgr2=0x7fb868112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8680746c0 0x7fb868112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.042+0000 7fb86dc00700 1 -- 172.21.15.7:0/3565455787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86810c660 con 0x7fb868113660 2024-06-13T06:56:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.043+0000 7fb86dc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb860009550 tx=0x7fb86000cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.043+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860021070 con 0x7fb868113660 2024-06-13T06:56:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.043+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb860007ac0 con 0x7fb868113660 2024-06-13T06:56:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.043+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8600052a0 con 0x7fb868113660 2024-06-13T06:56:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.044+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86810c8f0 con 0x7fb868113660 2024-06-13T06:56:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.044+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb868110800 con 0x7fb868113660 2024-06-13T06:56:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.048+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb868075de0 con 0x7fb868113660 2024-06-13T06:56:21.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.051+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb860024070 con 0x7fb868113660 2024-06-13T06:56:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.053+0000 7fb865600700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 0x7fb84c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.054+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb86000f020 con 0x7fb868113660 2024-06-13T06:56:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.055+0000 7fb86d200700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 0x7fb84c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.056+0000 7fb86d200700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 0x7fb84c0a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb858005d90 tx=0x7fb858005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:21.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.057+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb86002b020 con 0x7fb868113660 2024-06-13T06:56:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.568+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb868110af0 con 0x7fb868113660 2024-06-13T06:56:21.573 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:56:21.573 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:56:05.053870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.569+0000 7fb865600700 1 -- 172.21.15.7:0/3565455787 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11872 (secure 0 0 0) 0x7fb86009d4d0 con 0x7fb868113660 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 msgr2=0x7fb84c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 0x7fb84c0a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb858005d90 tx=0x7fb858005ce0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 msgr2=0x7fb86810bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb860009550 tx=0x7fb86000cfa0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 shutdown_connections 2024-06-13T06:56:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.574+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb868113660 0x7fb86810bda0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8680746c0 0x7fb868112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb84c0a2ca0 0x7fb84c0a5160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 --2- 172.21.15.7:0/3565455787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb868074fe0 0x7fb868113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 >> 172.21.15.7:0/3565455787 conn(0x7fb868079640 msgr2=0x7fb868104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 shutdown_connections 2024-06-13T06:56:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:21.575+0000 7fb86fc00700 1 -- 172.21.15.7:0/3565455787 wait complete. 2024-06-13T06:56:21.576 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-06-13T06:56:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:21 smithi189 ceph-mon[36616]: pgmap v518: 129 pgs: 129 active+clean; 947 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 608 op/s 2024-06-13T06:56:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:21 smithi175 ceph-mon[36886]: pgmap v518: 129 pgs: 129 active+clean; 947 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 608 op/s 2024-06-13T06:56:21.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:21 smithi007 ceph-mon[32077]: pgmap v518: 129 pgs: 129 active+clean; 947 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 608 op/s 2024-06-13T06:56:22.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T06:56:22.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:22 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3565455787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:56:22.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:22 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3565455787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:56:22.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:22 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3565455787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:56:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 20.0 s, 2082.8 tps, lat 47.524 ms stddev 52.001 2024-06-13T06:56:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:23.556 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:23 smithi007 ceph-mon[32077]: pgmap v519: 129 pgs: 129 active+clean; 965 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 787 op/s 2024-06-13T06:56:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:23 smithi189 ceph-mon[36616]: pgmap v519: 129 pgs: 129 active+clean; 965 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 787 op/s 2024-06-13T06:56:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:23 smithi175 ceph-mon[36886]: pgmap v519: 129 pgs: 129 active+clean; 965 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 787 op/s 2024-06-13T06:56:25.805 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:25 smithi007 ceph-mon[32077]: pgmap v520: 129 pgs: 129 active+clean; 958 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 873 op/s 2024-06-13T06:56:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 -- 172.21.15.7:0/3473068285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f8075180 msgr2=0x7ff7f810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 --2- 172.21.15.7:0/3473068285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f8075180 0x7ff7f810e720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 -- 172.21.15.7:0/3473068285 shutdown_connections 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 --2- 172.21.15.7:0/3473068285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f810ec60 0x7ff7f8111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 --2- 172.21.15.7:0/3473068285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8074860 0x7ff7f8074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 --2- 172.21.15.7:0/3473068285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f8075180 0x7ff7f810e720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.804+0000 7ff7fd800700 1 -- 172.21.15.7:0/3473068285 >> 172.21.15.7:0/3473068285 conn(0x7ff7f80fdf10 msgr2=0x7ff7f8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.805+0000 7ff7fd800700 1 -- 172.21.15.7:0/3473068285 shutdown_connections 2024-06-13T06:56:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.805+0000 7ff7fd800700 1 -- 172.21.15.7:0/3473068285 wait complete. 2024-06-13T06:56:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.809+0000 7ff7fd800700 1 Processor -- start 2024-06-13T06:56:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.809+0000 7ff7fd800700 1 -- start start 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.810+0000 7ff7fd800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.810+0000 7ff7fd800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 0x7ff7f810b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.810+0000 7ff7fd800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 0x7ff7f810bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.810+0000 7ff7fd800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81149f0 con 0x7ff7f8075180 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.810+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34432/0 (socket says 172.21.15.7:34432) 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7f7e00700 1 -- 172.21.15.7:0/2632730169 learned_addr learned my addr 172.21.15.7:0/2632730169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8114870 con 0x7ff7f8074860 2024-06-13T06:56:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8114b70 con 0x7ff7f810ec60 2024-06-13T06:56:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 0x7ff7f810bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.811+0000 7ff7f7400700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 0x7ff7f810b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7fce00700 1 -- 172.21.15.7:0/2632730169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 msgr2=0x7ff7f810e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7fce00700 1 -- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 msgr2=0x7ff7f810b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 0x7ff7f810b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7fce00700 1 -- 172.21.15.7:0/2632730169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f810c400 con 0x7ff7f810ec60 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.812+0000 7ff7f7400700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 0x7ff7f810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.813+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 0x7ff7f810bb50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b390 tx=0x7ff7e400b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.815+0000 7ff7f4c00700 1 -- 172.21.15.7:0/2632730169 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4015020 con 0x7ff7f810ec60 2024-06-13T06:56:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.815+0000 7ff7f4c00700 1 -- 172.21.15.7:0/2632730169 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e400be50 con 0x7ff7f810ec60 2024-06-13T06:56:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.815+0000 7ff7f4c00700 1 -- 172.21.15.7:0/2632730169 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4010ce0 con 0x7ff7f810ec60 2024-06-13T06:56:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.815+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f8199fe0 con 0x7ff7f810ec60 2024-06-13T06:56:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.815+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f819a4d0 con 0x7ff7f810ec60 2024-06-13T06:56:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.824+0000 7ff7f4c00700 1 -- 172.21.15.7:0/2632730169 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff7e4018020 con 0x7ff7f810ec60 2024-06-13T06:56:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.825+0000 7ff7f4c00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 0x7ff7dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.826+0000 7ff7f4c00700 1 -- 172.21.15.7:0/2632730169 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff7e40ce910 con 0x7ff7f810ec60 2024-06-13T06:56:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.826+0000 7ff809763b80 1 Processor -- start 2024-06-13T06:56:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.826+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 0x7ff7dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.826+0000 7ff809763b80 1 -- start start 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7346d60 0x55aae7347140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aae7256a10 con 0x55aae7352530 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aae7299b00 con 0x55aae7346d60 2024-06-13T06:56:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff809763b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aae72548f0 con 0x55aae7347680 2024-06-13T06:56:25.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.827+0000 7ff7fce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.828+0000 7ff7fce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60820/0 (socket says 172.21.15.7:60820) 2024-06-13T06:56:25.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.828+0000 7ff7fce00700 1 -- 172.21.15.7:0/297071064 learned_addr learned my addr 172.21.15.7:0/297071064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.828+0000 7ff7f7400700 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.829+0000 7ff7fce00700 1 -- 172.21.15.7:0/297071064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 msgr2=0x55aae7351ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.829+0000 7ff7fce00700 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.829+0000 7ff7fce00700 1 -- 172.21.15.7:0/297071064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7346d60 msgr2=0x55aae7347140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.829+0000 7ff7fce00700 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7346d60 0x55aae7347140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.829+0000 7ff7fce00700 1 -- 172.21.15.7:0/297071064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae7241f90 con 0x55aae7352530 2024-06-13T06:56:25.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.830+0000 7ff7fce00700 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7e409adc0 tx=0x7ff7e409aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad76a397da39216 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.830+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 0x7ff7dc0a5230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c590 tx=0x7ff7f000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.830+0000 7ff7ed600700 1 -- 172.21.15.7:0/297071064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e409f1e0 con 0x55aae7352530 2024-06-13T06:56:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.831+0000 7ff7ed600700 1 -- 172.21.15.7:0/297071064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e40a3aa0 con 0x55aae7352530 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.831+0000 7ff7ed600700 1 -- 172.21.15.7:0/297071064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40a3870 con 0x55aae7352530 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 -- 172.21.15.7:0/297071064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 msgr2=0x55aae7354920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7e409adc0 tx=0x7ff7e409aa80 comp rx=0 tx=0).stop 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 -- 172.21.15.7:0/297071064 shutdown_connections 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7352530 0x55aae7354920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7346d60 0x55aae7347140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 --2- 172.21.15.7:0/297071064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.832+0000 7ff809763b80 1 -- 172.21.15.7:0/297071064 >> 172.21.15.7:0/297071064 conn(0x55aae7274170 msgr2=0x55aae7343280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.833+0000 7ff809763b80 1 -- 172.21.15.7:0/297071064 shutdown_connections 2024-06-13T06:56:25.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.833+0000 7ff809763b80 1 -- 172.21.15.7:0/297071064 wait complete. 2024-06-13T06:56:25.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.834+0000 7ff809763b80 1 -- start start 2024-06-13T06:56:25.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.835+0000 7ff809763b80 1 Processor -- start 2024-06-13T06:56:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:25 smithi189 ceph-mon[36616]: pgmap v520: 129 pgs: 129 active+clean; 958 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 873 op/s 2024-06-13T06:56:25.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.840+0000 7ff809763b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7351e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.841+0000 7ff809763b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae734f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.841+0000 7ff809763b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae734f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.841+0000 7ff809763b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aae7235f60 con 0x55aae7346d60 2024-06-13T06:56:25.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.841+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7351e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.841+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae734f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.842+0000 7ff7fce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae734f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.842+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae734f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2024-06-13T06:56:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.842+0000 7ff7f7400700 1 -- 172.21.15.7:0/537976277 learned_addr learned my addr 172.21.15.7:0/537976277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.842+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55aae7295ac0 con 0x55aae7352530 2024-06-13T06:56:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.842+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aae7354fc0 con 0x55aae7347680 2024-06-13T06:56:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.843+0000 7ff809763b80 10 client.0 _ll_register_callbacks cb 0x55aae722f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 -- 172.21.15.7:0/537976277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 msgr2=0x55aae734f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae734f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 -- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 msgr2=0x55aae7351e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7351e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae71e88d0 con 0x55aae7352530 2024-06-13T06:56:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7e2000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.844+0000 7ff7fce00700 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae734f650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7e409ab30 tx=0x7ff7e4009270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c261136bdd08dd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.845+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40a7020 con 0x55aae7352530 2024-06-13T06:56:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.845+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff7e409b090 con 0x55aae7352530 2024-06-13T06:56:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.846+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e40a3aa0 con 0x55aae7352530 2024-06-13T06:56:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.846+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7e40a7020 con 0x55aae7352530 2024-06-13T06:56:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.847+0000 7ff809763b80 10 client.25331 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.847+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aae735f810 con 0x55aae7352530 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.847+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e40a3aa0 con 0x55aae7352530 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.847+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ff7e409a280 con 0x55aae7352530 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 10 client.25331 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 10 client.25331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 20 client.25331 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae7361ce0 0x55aae73640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 4 client.25331 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.848+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae72a7000 con 0x55aae7361ce0 2024-06-13T06:56:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.850+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae7361ce0 0x55aae73640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.852+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae7361ce0 0x55aae73640d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6de9c86c6751ccb server_cookie=8eac0878c612bbde in_seq=0 out_seq=0 2024-06-13T06:56:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.852+0000 7ff7e2000700 10 client.25331 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:56:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.856+0000 7ff7e2000700 1 -- 172.21.15.7:0/537976277 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aae72a7000 con 0x55aae7361ce0 2024-06-13T06:56:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.856+0000 7ff7e2000700 10 client.25331 handle_command_reply: tid=0 2024-06-13T06:56:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.856+0000 7ff809763b80 1 client.25331 shutdown 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 msgr2=0x55aae734f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae734f650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7e409ab30 tx=0x7ff7e4009270 comp rx=0 tx=0).stop 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 shutdown_connections 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae7361ce0 0x55aae73640d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff7e2000700 0 client.25331 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae734f650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7351e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 --2- 172.21.15.7:0/537976277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae734f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.857+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 >> 172.21.15.7:0/537976277 conn(0x55aae7274170 msgr2=0x55aae7342e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.858+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 shutdown_connections 2024-06-13T06:56:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.858+0000 7ff809763b80 1 -- 172.21.15.7:0/537976277 wait complete. 2024-06-13T06:56:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.858+0000 7ff809763b80 20 client.25331 trim_cache size 0 max 16384 2024-06-13T06:56:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:25 smithi175 ceph-mon[36886]: pgmap v520: 129 pgs: 129 active+clean; 958 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 873 op/s 2024-06-13T06:56:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.890+0000 7ff809763b80 1 Processor -- start 2024-06-13T06:56:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.891+0000 7ff809763b80 1 -- start start 2024-06-13T06:56:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.891+0000 7ff809763b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.892+0000 7ff809763b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.893+0000 7ff809763b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae73670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.893+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.893+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60844/0 (socket says 172.21.15.7:60844) 2024-06-13T06:56:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.893+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.893+0000 7ff7f7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35798/0 (socket says 172.21.15.7:35798) 2024-06-13T06:56:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.894+0000 7ff7f7e00700 1 -- 172.21.15.7:0/2921144959 learned_addr learned my addr 172.21.15.7:0/2921144959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.894+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81bf7c0 con 0x55aae7346d60 2024-06-13T06:56:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.894+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81bd020 con 0x55aae7352530 2024-06-13T06:56:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.894+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81a0e00 con 0x55aae7347680 2024-06-13T06:56:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.894+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae73670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.895+0000 7ff7f7e00700 1 -- 172.21.15.7:0/2921144959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 msgr2=0x55aae7351950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.895+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.895+0000 7ff7f7e00700 1 -- 172.21.15.7:0/2921144959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 msgr2=0x55aae73670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.895+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae73670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.895+0000 7ff7f7e00700 1 -- 172.21.15.7:0/2921144959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae734d680 con 0x55aae7346d60 2024-06-13T06:56:25.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.896+0000 7ff7fce00700 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae73670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:25.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.896+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55aae7256a70 tx=0x7ff7f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badbfe9d16e4fee4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.897+0000 7ff7ecc00700 1 -- 172.21.15.7:0/2921144959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f001b040 con 0x55aae7346d60 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.897+0000 7ff7ecc00700 1 -- 172.21.15.7:0/2921144959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f002f070 con 0x55aae7346d60 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.897+0000 7ff7ecc00700 1 -- 172.21.15.7:0/2921144959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f002a7e0 con 0x55aae7346d60 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 msgr2=0x55aae734dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55aae7256a70 tx=0x7ff7f0011000 comp rx=0 tx=0).stop 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 shutdown_connections 2024-06-13T06:56:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7352530 0x55aae73670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae734dfe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 --2- 172.21.15.7:0/2921144959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7347680 0x55aae7351950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.898+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 >> 172.21.15.7:0/2921144959 conn(0x55aae7274170 msgr2=0x55aae734d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.899+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 shutdown_connections 2024-06-13T06:56:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.899+0000 7ff809763b80 1 -- 172.21.15.7:0/2921144959 wait complete. 2024-06-13T06:56:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.900+0000 7ff809763b80 1 -- start start 2024-06-13T06:56:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.900+0000 7ff809763b80 1 Processor -- start 2024-06-13T06:56:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.902+0000 7ff809763b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7367500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.903+0000 7ff809763b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 0x55aae73648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.903+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7367500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.903+0000 7ff7f7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7367500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60854/0 (socket says 172.21.15.7:60854) 2024-06-13T06:56:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.903+0000 7ff7f7e00700 1 -- 172.21.15.7:0/3557899714 learned_addr learned my addr 172.21.15.7:0/3557899714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.903+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 0x55aae7364de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.904+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81a1470 con 0x55aae7346d60 2024-06-13T06:56:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.904+0000 7ff7f7400700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 0x55aae73648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.905+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81bc950 con 0x55aae7347680 2024-06-13T06:56:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.905+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81a1b40 con 0x55aae7352530 2024-06-13T06:56:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.905+0000 7ff7fce00700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 0x55aae7364de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.905+0000 7ff809763b80 10 client.0 _ll_register_callbacks cb 0x55aae722f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7f7400700 1 -- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 msgr2=0x55aae7364de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7f7400700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 0x55aae7364de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7f7400700 1 -- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 msgr2=0x55aae7367500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7f7400700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7367500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7f7400700 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae7365320 con 0x55aae7347680 2024-06-13T06:56:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.906+0000 7ff7fce00700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 0x55aae7364de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.907+0000 7ff7f7400700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 0x55aae73648a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005cd0 tx=0x7ff7e8003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62bbf91fbad2cf20 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.907+0000 7ff7e2000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.907+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8028060 con 0x55aae7347680 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.908+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff7e800bbe0 con 0x55aae7347680 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.908+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f002f070 con 0x55aae7347680 2024-06-13T06:56:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.908+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7e802f050 con 0x55aae7347680 2024-06-13T06:56:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.909+0000 7ff809763b80 10 client.25334 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.909+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aae73cc250 con 0x55aae7347680 2024-06-13T06:56:25.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.909+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f002f070 con 0x55aae7347680 2024-06-13T06:56:25.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.909+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ff7e800f9d0 con 0x55aae7347680 2024-06-13T06:56:25.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 10 client.25334 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 10 client.25334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:56:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 20 client.25334 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae73cc990 0x55aae73cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 4 client.25334 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:56:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aae72a1580 con 0x55aae73cc990 2024-06-13T06:56:25.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.910+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae73cc990 0x55aae73cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:25.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.912+0000 7ff7f7e00700 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae73cc990 0x55aae73cedb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd67968b81fe5a6b server_cookie=eb449d9a9e7f619f in_seq=0 out_seq=0 2024-06-13T06:56:25.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.912+0000 7ff7e2000700 10 client.25334 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff7e2000700 1 -- 172.21.15.7:0/3557899714 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aae72a1580 con 0x55aae73cc990 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff7e2000700 10 client.25334 handle_command_reply: tid=0 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 client.25334 shutdown 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 msgr2=0x55aae73648a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 0x55aae73648a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005cd0 tx=0x7ff7e8003ea0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 shutdown_connections 2024-06-13T06:56:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aae7352530 0x55aae7364de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aae7346d60 0x55aae7367500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55aae73cc990 0x55aae73cedb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.913+0000 7ff809763b80 1 --2- 172.21.15.7:0/3557899714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55aae7347680 0x55aae73648a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.914+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 >> 172.21.15.7:0/3557899714 conn(0x55aae7274170 msgr2=0x55aae73495b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.914+0000 7ff7e2000700 0 client.25334 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.914+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 shutdown_connections 2024-06-13T06:56:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.914+0000 7ff809763b80 1 -- 172.21.15.7:0/3557899714 wait complete. 2024-06-13T06:56:25.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.914+0000 7ff809763b80 20 client.25334 trim_cache size 0 max 16384 2024-06-13T06:56:25.916 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:56:25.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 msgr2=0x7ff7dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 0x7ff7dc0a5230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c590 tx=0x7ff7f000b040 comp rx=0 tx=0).stop 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 msgr2=0x7ff7f810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 0x7ff7f810bb50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b390 tx=0x7ff7e400b6a0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 shutdown_connections 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7f810ec60 0x7ff7f810bb50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff7dc0a2d70 0x7ff7dc0a5230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7f8074860 0x7ff7f810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 --2- 172.21.15.7:0/2632730169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7f8075180 0x7ff7f810b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.915+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 >> 172.21.15.7:0/2632730169 conn(0x7ff7f80fdf10 msgr2=0x7ff7f80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.916+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 shutdown_connections 2024-06-13T06:56:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:25.916+0000 7ff7fd800700 1 -- 172.21.15.7:0/2632730169 wait complete. 2024-06-13T06:56:26.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T06:56:27.541 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:27.818 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:27 smithi007 ceph-mon[32077]: pgmap v521: 129 pgs: 129 active+clean; 951 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-13T06:56:27.819 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:27 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:27 smithi189 ceph-mon[36616]: pgmap v521: 129 pgs: 129 active+clean; 951 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-13T06:56:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:27 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:27 smithi175 ceph-mon[36886]: pgmap v521: 129 pgs: 129 active+clean; 951 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-06-13T06:56:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:27 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:56:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 25.0 s, 1605.6 tps, lat 62.530 ms stddev 68.945 2024-06-13T06:56:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.056+0000 7fb2c3a00640 1 -- 172.21.15.7:0/2339935390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc1017e0 msgr2=0x7fb2bc108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.056+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/2339935390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc108070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005fd0 tx=0x7fb2b402b3f0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 -- 172.21.15.7:0/2339935390 shutdown_connections 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/2339935390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc108070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/2339935390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc100e20 0x7fb2bc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/2339935390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 -- 172.21.15.7:0/2339935390 >> 172.21.15.7:0/2339935390 conn(0x7fb2bc0fb1c0 msgr2=0x7fb2bc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 -- 172.21.15.7:0/2339935390 shutdown_connections 2024-06-13T06:56:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.058+0000 7fb2c3a00640 1 -- 172.21.15.7:0/2339935390 wait complete. 2024-06-13T06:56:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.060+0000 7fb2c3a00640 1 Processor -- start 2024-06-13T06:56:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.060+0000 7fb2c3a00640 1 -- start start 2024-06-13T06:56:28.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.060+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.060+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc100e20 0x7fb2bc111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c3a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc1ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc10afb0 con 0x7fb2bc100e20 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc10ae30 con 0x7fb2bc0ffc30 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c3a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc10b130 con 0x7fb2bc1017e0 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc100e20 0x7fb2bc111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34492/0 (socket says 172.21.15.7:34492) 2024-06-13T06:56:28.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc1ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.061+0000 7fb2c1000640 1 -- 172.21.15.7:0/1836023499 learned_addr learned my addr 172.21.15.7:0/1836023499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1a00640 1 -- 172.21.15.7:0/1836023499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 msgr2=0x7fb2bc110e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1a00640 1 -- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc100e20 msgr2=0x7fb2bc111350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc100e20 0x7fb2bc111350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1a00640 1 -- 172.21.15.7:0/1836023499 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1ae7d0 con 0x7fb2bc1017e0 2024-06-13T06:56:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.062+0000 7fb2c1000640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.063+0000 7fb2c1a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc1ae290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4035040 tx=0x7fb2b400a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.064+0000 7fb2b9600640 1 -- 172.21.15.7:0/1836023499 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4037040 con 0x7fb2bc1017e0 2024-06-13T06:56:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.064+0000 7fb2b9600640 1 -- 172.21.15.7:0/1836023499 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4042070 con 0x7fb2bc1017e0 2024-06-13T06:56:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.064+0000 7fb2b9600640 1 -- 172.21.15.7:0/1836023499 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b403c3b0 con 0x7fb2bc1017e0 2024-06-13T06:56:28.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.064+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc1ae9a0 con 0x7fb2bc1017e0 2024-06-13T06:56:28.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.065+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc1aee70 con 0x7fb2bc1017e0 2024-06-13T06:56:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.071+0000 7fb2c69b6740 1 Processor -- start 2024-06-13T06:56:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.073+0000 7fb2b9600640 1 -- 172.21.15.7:0/1836023499 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb2b403d450 con 0x7fb2bc1017e0 2024-06-13T06:56:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.073+0000 7fb2c69b6740 1 -- start start 2024-06-13T06:56:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564287940420 con 0x5642879dcc00 2024-06-13T06:56:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642878e7c50 con 0x5642879e2760 2024-06-13T06:56:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.074+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56428793c3e0 con 0x5642879dd5f0 2024-06-13T06:56:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60896/0 (socket says 172.21.15.7:60896) 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2bbe00640 1 -- 172.21.15.7:0/4031960340 learned_addr learned my addr 172.21.15.7:0/4031960340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2b9600640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 0x7fb2a40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2b9600640 1 -- 172.21.15.7:0/1836023499 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb2b40efcc0 con 0x7fb2bc1017e0 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.075+0000 7fb2c1000640 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2c1a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 0x7fb2a40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 -- 172.21.15.7:0/4031960340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 msgr2=0x5642879e20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879e20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 -- 172.21.15.7:0/4031960340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 msgr2=0x5642879e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 -- 172.21.15.7:0/4031960340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564287654210 con 0x5642879dcc00 2024-06-13T06:56:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0014850 tx=0x7fb2b0009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86075c408258012b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.076+0000 7fb2ae000640 1 -- 172.21.15.7:0/4031960340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0014110 con 0x5642879dcc00 2024-06-13T06:56:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.077+0000 7fb2c1a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 0x7fb2a40a5260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40bbee0 tx=0x7fb2b400a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.077+0000 7fb2ae000640 1 -- 172.21.15.7:0/4031960340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b00142b0 con 0x5642879dcc00 2024-06-13T06:56:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.077+0000 7fb2ae000640 1 -- 172.21.15.7:0/4031960340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003850 con 0x5642879dcc00 2024-06-13T06:56:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.078+0000 7fb2c69b6740 1 -- 172.21.15.7:0/4031960340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 msgr2=0x5642879dd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.078+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0014850 tx=0x7fb2b0009630 comp rx=0 tx=0).stop 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 -- 172.21.15.7:0/4031960340 shutdown_connections 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879e20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/4031960340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879dd020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 -- 172.21.15.7:0/4031960340 >> 172.21.15.7:0/4031960340 conn(0x5642879a4fb0 msgr2=0x5642879d8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 -- 172.21.15.7:0/4031960340 shutdown_connections 2024-06-13T06:56:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.084+0000 7fb2c69b6740 1 -- 172.21.15.7:0/4031960340 wait complete. 2024-06-13T06:56:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.085+0000 7fb2c69b6740 1 -- start start 2024-06-13T06:56:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.085+0000 7fb2c69b6740 1 Processor -- start 2024-06-13T06:56:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.086+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879e2760 0x5642879ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5642879e56b0 con 0x5642879dcc00 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5642878fb0b0 con 0x5642879dd5f0 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642879e5830 con 0x5642879e2760 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c69b6740 10 client.0 _ll_register_callbacks cb 0x5642878e6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34516/0 (socket says 172.21.15.7:34516) 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.087+0000 7fb2c1000640 1 -- 172.21.15.7:0/934354317 learned_addr learned my addr 172.21.15.7:0/934354317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 -- 172.21.15.7:0/934354317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879e2760 msgr2=0x5642879ee4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879e2760 0x5642879ee4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 -- 172.21.15.7:0/934354317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 msgr2=0x5642879e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 -- 172.21.15.7:0/934354317 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564287720620 con 0x5642879dd5f0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2c1000640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8010500 tx=0x7fb2a80109c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23887d9e1dbcd16b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.088+0000 7fb2a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8018070 con 0x5642879dd5f0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b00142b0 con 0x5642879dd5f0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb2a8014640 con 0x5642879dd5f0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8024050 con 0x5642879dd5f0 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2c69b6740 10 client.25346 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.089+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642879f1890 con 0x5642879dd5f0 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b00142b0 con 0x5642879dd5f0 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2a8012ac0 con 0x5642879dd5f0 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 10 client.25346 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 20 client.25346 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5642879f3090 0x5642879f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 4 client.25346 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56428794db50 con 0x5642879f3090 2024-06-13T06:56:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.090+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5642879f3090 0x5642879f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.091+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5642879f3090 0x5642879f5530 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281cbe59cdb77ed0 server_cookie=4994d2e46cf1faa in_seq=0 out_seq=0 2024-06-13T06:56:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.091+0000 7fb2a2a00640 10 client.25346 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.092+0000 7fb2a2a00640 1 -- 172.21.15.7:0/934354317 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56428794db50 con 0x5642879f3090 2024-06-13T06:56:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.092+0000 7fb2a2a00640 10 client.25346 handle_command_reply: tid=0 2024-06-13T06:56:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.092+0000 7fb2c69b6740 1 client.25346 shutdown 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 msgr2=0x5642879e7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8010500 tx=0x7fb2a80109c0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 shutdown_connections 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5642879f3090 0x5642879f5530 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879e2760 0x5642879ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879dd5f0 0x5642879e7470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/934354317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 >> 172.21.15.7:0/934354317 conn(0x5642879a4fb0 msgr2=0x5642879d7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2a2a00640 0 client.25346 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.093+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 shutdown_connections 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.094+0000 7fb2c69b6740 1 -- 172.21.15.7:0/934354317 wait complete. 2024-06-13T06:56:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.094+0000 7fb2c69b6740 20 client.25346 trim_cache size 0 max 16384 2024-06-13T06:56:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 Processor -- start 2024-06-13T06:56:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 -- start start 2024-06-13T06:56:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1bbb30 con 0x5642879dcc00 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1b93d0 con 0x5642879e2760 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.130+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1b7220 con 0x5642879dd5f0 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.131+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.131+0000 7fb2c1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35848/0 (socket says 172.21.15.7:35848) 2024-06-13T06:56:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.131+0000 7fb2c1000640 1 -- 172.21.15.7:0/76495084 learned_addr learned my addr 172.21.15.7:0/76495084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.131+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.132+0000 7fb2bbe00640 1 -- 172.21.15.7:0/76495084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 msgr2=0x5642879ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.132+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.132+0000 7fb2bbe00640 1 -- 172.21.15.7:0/76495084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 msgr2=0x5642879db340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.132+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879db340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.132+0000 7fb2bbe00640 1 -- 172.21.15.7:0/76495084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642879dcfe0 con 0x5642879dcc00 2024-06-13T06:56:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.133+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006010 tx=0x7fb2b0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9865a2b98fa85dfc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.134+0000 7fb2ad600640 1 -- 172.21.15.7:0/76495084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000f930 con 0x5642879dcc00 2024-06-13T06:56:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.134+0000 7fb2ad600640 1 -- 172.21.15.7:0/76495084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b00144e0 con 0x5642879dcc00 2024-06-13T06:56:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.134+0000 7fb2ad600640 1 -- 172.21.15.7:0/76495084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00048e0 con 0x5642879dcc00 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.135+0000 7fb2c69b6740 1 -- 172.21.15.7:0/76495084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 msgr2=0x5642879e6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.135+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0006010 tx=0x7fb2b0009fd0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 -- 172.21.15.7:0/76495084 shutdown_connections 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/76495084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879e6310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 -- 172.21.15.7:0/76495084 >> 172.21.15.7:0/76495084 conn(0x5642879a4fb0 msgr2=0x5642879e5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 -- 172.21.15.7:0/76495084 shutdown_connections 2024-06-13T06:56:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.136+0000 7fb2c69b6740 1 -- 172.21.15.7:0/76495084 wait complete. 2024-06-13T06:56:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.137+0000 7fb2c69b6740 1 -- start start 2024-06-13T06:56:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.138+0000 7fb2c69b6740 1 Processor -- start 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1b7890 con 0x5642879dcc00 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1b8d00 con 0x5642879e2760 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc1b7f60 con 0x5642879dd5f0 2024-06-13T06:56:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2c69b6740 10 client.0 _ll_register_callbacks cb 0x7fb2a80075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60926/0 (socket says 172.21.15.7:60926) 2024-06-13T06:56:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.139+0000 7fb2bbe00640 1 -- 172.21.15.7:0/3847664135 learned_addr learned my addr 172.21.15.7:0/3847664135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2c1000640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 -- 172.21.15.7:0/3847664135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 msgr2=0x5642879ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 -- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 msgr2=0x5642879ecd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879ecd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.140+0000 7fb2bbe00640 1 -- 172.21.15.7:0/3847664135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642879ed370 con 0x5642879dcc00 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.141+0000 7fb2bbe00640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009c70 tx=0x7fb2b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401617965475dbb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.141+0000 7fb2a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:56:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.142+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b002d040 con 0x5642879dcc00 2024-06-13T06:56:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.142+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b00144e0 con 0x5642879dcc00 2024-06-13T06:56:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.142+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb2b000fc70 con 0x5642879dcc00 2024-06-13T06:56:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.143+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2b002d040 con 0x5642879dcc00 2024-06-13T06:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.145+0000 7fb2c69b6740 10 client.15207 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.145+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642879ee680 con 0x5642879dcc00 2024-06-13T06:56:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.145+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b00144e0 con 0x5642879dcc00 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2b0033070 con 0x5642879dcc00 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 10 client.15207 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 10 client.15207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 20 client.15207 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564287a5e1d0 0x564287a605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 4 client.15207 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:56:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.147+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564287947ea0 con 0x564287a5e1d0 2024-06-13T06:56:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.148+0000 7fb2c1000640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564287a5e1d0 0x564287a605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.148+0000 7fb2c1000640 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564287a5e1d0 0x564287a605f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6beb844db6c99dd8 server_cookie=4e5f296259b0b4f6 in_seq=0 out_seq=0 2024-06-13T06:56:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.148+0000 7fb2a2a00640 10 client.15207 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.171+0000 7fb2a2a00640 1 -- 172.21.15.7:0/3847664135 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564287947ea0 con 0x564287a5e1d0 2024-06-13T06:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.171+0000 7fb2a2a00640 10 client.15207 handle_command_reply: tid=0 2024-06-13T06:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.171+0000 7fb2c69b6740 1 client.15207 shutdown 2024-06-13T06:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.173+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 msgr2=0x5642879eb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.173+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009c70 tx=0x7fb2b0009c00 comp rx=0 tx=0).stop 2024-06-13T06:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.177+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 shutdown_connections 2024-06-13T06:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.177+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564287a5e1d0 0x564287a605f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.177+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5642879e2760 0x5642879ecd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.178+0000 7fb2a2a00640 0 client.15207 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.178+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642879dd5f0 0x5642879ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.178+0000 7fb2c69b6740 1 --2- 172.21.15.7:0/3847664135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642879dcc00 0x5642879eb830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.178+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 >> 172.21.15.7:0/3847664135 conn(0x5642879a4fb0 msgr2=0x5642879d6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.182+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 shutdown_connections 2024-06-13T06:56:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.183+0000 7fb2c69b6740 1 -- 172.21.15.7:0/3847664135 wait complete. 2024-06-13T06:56:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.183+0000 7fb2c69b6740 20 client.15207 trim_cache size 0 max 16384 2024-06-13T06:56:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:56:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:56:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.188+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 msgr2=0x7fb2a40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.188+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 0x7fb2a40a5260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40bbee0 tx=0x7fb2b400a040 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.188+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 msgr2=0x7fb2bc1ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.188+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc1ae290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4035040 tx=0x7fb2b400a2e0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 shutdown_connections 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2a40a2da0 0x7fb2a40a5260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2bc1017e0 0x7fb2bc1ae290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2bc100e20 0x7fb2bc111350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 --2- 172.21.15.7:0/1836023499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2bc0ffc30 0x7fb2bc110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 >> 172.21.15.7:0/1836023499 conn(0x7fb2bc0fb1c0 msgr2=0x7fb2bc104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 shutdown_connections 2024-06-13T06:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:28.190+0000 7fb2c3a00640 1 -- 172.21.15.7:0/1836023499 wait complete. 2024-06-13T06:56:29.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:29 smithi007 ceph-mon[32077]: pgmap v522: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 754 op/s 2024-06-13T06:56:29.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:56:29.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:56:29.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:29 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:56:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:29 smithi189 ceph-mon[36616]: pgmap v522: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 754 op/s 2024-06-13T06:56:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:56:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:56:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:29 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:56:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:29 smithi175 ceph-mon[36886]: pgmap v522: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 754 op/s 2024-06-13T06:56:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:56:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:56:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:29 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:56:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:31 smithi189 ceph-mon[36616]: pgmap v523: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-06-13T06:56:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:31 smithi175 ceph-mon[36886]: pgmap v523: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-06-13T06:56:31.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:31 smithi007 ceph-mon[32077]: pgmap v523: 129 pgs: 129 active+clean; 940 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 719 op/s 2024-06-13T06:56:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 -- 172.21.15.7:0/16570921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f666406a170 msgr2=0x7f666406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 --2- 172.21.15.7:0/16570921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f666406a170 0x7f666406a5f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6650009990 tx=0x7f6650009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 -- 172.21.15.7:0/16570921 shutdown_connections 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 --2- 172.21.15.7:0/16570921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f666410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 --2- 172.21.15.7:0/16570921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6664108500 0x7f66641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 --2- 172.21.15.7:0/16570921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f666406a170 0x7f666406a5f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.714+0000 7f666b600700 1 -- 172.21.15.7:0/16570921 >> 172.21.15.7:0/16570921 conn(0x7f6664078340 msgr2=0x7f6664078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.715+0000 7f666b600700 1 -- 172.21.15.7:0/16570921 shutdown_connections 2024-06-13T06:56:32.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.715+0000 7f666b600700 1 -- 172.21.15.7:0/16570921 wait complete. 2024-06-13T06:56:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.716+0000 7f666b600700 1 Processor -- start 2024-06-13T06:56:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 -- start start 2024-06-13T06:56:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f666406a170 0x7f6664073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6664108500 0x7f666406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6664110700 con 0x7f666406a170 2024-06-13T06:56:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6664110580 con 0x7f6664108500 2024-06-13T06:56:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.717+0000 7f666b600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6664110880 con 0x7f666406ab30 2024-06-13T06:56:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.718+0000 7f6663e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.718+0000 7f6663e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36444/0 (socket says 172.21.15.7:36444) 2024-06-13T06:56:32.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.718+0000 7f6663e00700 1 -- 172.21.15.7:0/1267150223 learned_addr learned my addr 172.21.15.7:0/1267150223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.718+0000 7f6668c00700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f666406a170 0x7f6664073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6669600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6664108500 0x7f666406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6663e00700 1 -- 172.21.15.7:0/1267150223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6664108500 msgr2=0x7f666406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6663e00700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6664108500 0x7f666406caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6663e00700 1 -- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f666406a170 msgr2=0x7f6664073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6663e00700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f666406a170 0x7f6664073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.719+0000 7f6663e00700 1 -- 172.21.15.7:0/1267150223 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666406d370 con 0x7f666406ab30 2024-06-13T06:56:32.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.720+0000 7f6663e00700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66500057d0 tx=0x7f6650005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.720+0000 7f6661600700 1 -- 172.21.15.7:0/1267150223 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650027070 con 0x7f666406ab30 2024-06-13T06:56:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.720+0000 7f6661600700 1 -- 172.21.15.7:0/1267150223 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650022470 con 0x7f666406ab30 2024-06-13T06:56:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.720+0000 7f6661600700 1 -- 172.21.15.7:0/1267150223 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000f6a0 con 0x7f666406ab30 2024-06-13T06:56:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.721+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666406d600 con 0x7f666406ab30 2024-06-13T06:56:32.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.722+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6664071540 con 0x7f666406ab30 2024-06-13T06:56:32.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.726+0000 7f6661600700 1 -- 172.21.15.7:0/1267150223 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6650022620 con 0x7f666406ab30 2024-06-13T06:56:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.730+0000 7f6675e92b80 1 Processor -- start 2024-06-13T06:56:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.730+0000 7f6675e92b80 1 -- start start 2024-06-13T06:56:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a3ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3a240 0x55a241a3a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a241948a10 con 0x55a241a3fb90 2024-06-13T06:56:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a24198bb00 con 0x55a241a404b0 2024-06-13T06:56:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6675e92b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a2419468f0 con 0x55a241a3a240 2024-06-13T06:56:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.731+0000 7f6668c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47976/0 (socket says 172.21.15.7:47976) 2024-06-13T06:56:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 -- 172.21.15.7:0/4051622631 learned_addr learned my addr 172.21.15.7:0/4051622631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6669600700 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a3ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6663e00700 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3a240 0x55a241a3a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 -- 172.21.15.7:0/4051622631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3a240 msgr2=0x55a241a3a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3a240 0x55a241a3a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 -- 172.21.15.7:0/4051622631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 msgr2=0x55a241a3ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.732+0000 7f6668c00700 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a3ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.733+0000 7f6668c00700 1 -- 172.21.15.7:0/4051622631 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a241933f90 con 0x55a241a404b0 2024-06-13T06:56:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.733+0000 7f6668c00700 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6658009af0 tx=0x7f6658009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d55f37ad672af13 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.733+0000 7f6656000700 1 -- 172.21.15.7:0/4051622631 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658017070 con 0x55a241a404b0 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.733+0000 7f6656000700 1 -- 172.21.15.7:0/4051622631 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665800edb0 con 0x55a241a404b0 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.733+0000 7f6656000700 1 -- 172.21.15.7:0/4051622631 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580123d0 con 0x55a241a404b0 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.734+0000 7f6675e92b80 1 -- 172.21.15.7:0/4051622631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 msgr2=0x55a241a39d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.734+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6658009af0 tx=0x7f6658009e00 comp rx=0 tx=0).stop 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.734+0000 7f6661600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 0x7f664c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6661600700 1 -- 172.21.15.7:0/1267150223 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f66500d04c0 con 0x7f666406ab30 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 -- 172.21.15.7:0/4051622631 shutdown_connections 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3a240 0x55a241a3a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a3ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4051622631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a39d00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 -- 172.21.15.7:0/4051622631 >> 172.21.15.7:0/4051622631 conn(0x55a241966170 msgr2=0x55a241a34ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 -- 172.21.15.7:0/4051622631 shutdown_connections 2024-06-13T06:56:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.735+0000 7f6675e92b80 1 -- 172.21.15.7:0/4051622631 wait complete. 2024-06-13T06:56:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.736+0000 7f6669600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 0x7f664c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.736+0000 7f6675e92b80 1 -- start start 2024-06-13T06:56:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.737+0000 7f6675e92b80 1 Processor -- start 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.737+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 0x55a241a461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.737+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.737+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6675e92b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a241927f60 con 0x55a241a3fb90 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6675e92b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a241987ac0 con 0x55a241a3a240 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6675e92b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a241a46e90 con 0x55a241a404b0 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6675e92b80 10 client.0 _ll_register_callbacks cb 0x55a241921380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6669600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6669600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36466/0 (socket says 172.21.15.7:36466) 2024-06-13T06:56:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6669600700 1 -- 172.21.15.7:0/521038068 learned_addr learned my addr 172.21.15.7:0/521038068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.738+0000 7f6668c00700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 0x55a241a461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.739+0000 7f6669600700 1 -- 172.21.15.7:0/521038068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 msgr2=0x55a241a461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.739+0000 7f6669600700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 0x55a241a461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.739+0000 7f6669600700 1 -- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 msgr2=0x55a241a434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.739+0000 7f6669600700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.740+0000 7f6669600700 1 -- 172.21.15.7:0/521038068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2418da8d0 con 0x55a241a404b0 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.740+0000 7f6668c00700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 0x55a241a461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:32.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.740+0000 7f6669600700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f665c014570 tx=0x7f665c004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b92391bab95bb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.740+0000 7f664aa00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.741+0000 7f6669600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 0x7f664c0a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f665c014c60 tx=0x7f665c007f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.741+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c022440 con 0x55a241a404b0 2024-06-13T06:56:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.741+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f665c0225e0 con 0x55a241a404b0 2024-06-13T06:56:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.741+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665800edb0 con 0x55a241a404b0 2024-06-13T06:56:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.741+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f665c0227e0 con 0x55a241a404b0 2024-06-13T06:56:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f6675e92b80 10 client.25357 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a241a51610 con 0x55a241a404b0 2024-06-13T06:56:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665800edb0 con 0x55a241a404b0 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f665c010070 con 0x55a241a404b0 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f6675e92b80 10 client.25357 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f6675e92b80 10 client.25357 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.742+0000 7f6675e92b80 20 client.25357 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.743+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241a54120 0x55a241a565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.743+0000 7f6675e92b80 4 client.25357 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.743+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a241999000 con 0x55a241a54120 2024-06-13T06:56:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.744+0000 7f6668c00700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241a54120 0x55a241a565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.746+0000 7f6668c00700 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241a54120 0x55a241a565c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77ad2a93b4d9cdc9 server_cookie=b8738c4bf2343c5c in_seq=0 out_seq=0 2024-06-13T06:56:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.746+0000 7f664aa00700 10 client.25357 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:56:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.749+0000 7f664aa00700 1 -- 172.21.15.7:0/521038068 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a241999000 con 0x55a241a54120 2024-06-13T06:56:32.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.749+0000 7f664aa00700 10 client.25357 handle_command_reply: tid=0 2024-06-13T06:56:32.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.749+0000 7f6675e92b80 1 client.25357 shutdown 2024-06-13T06:56:32.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.752+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 msgr2=0x55a241a43a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.752+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f665c014570 tx=0x7f665c004b60 comp rx=0 tx=0).stop 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 shutdown_connections 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241a54120 0x55a241a565c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a43a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3a240 0x55a241a461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 --2- 172.21.15.7:0/521038068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3fb90 0x55a241a434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f664aa00700 0 client.25357 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:56:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 >> 172.21.15.7:0/521038068 conn(0x55a241966170 msgr2=0x55a241a2e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 shutdown_connections 2024-06-13T06:56:32.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.754+0000 7f6675e92b80 1 -- 172.21.15.7:0/521038068 wait complete. 2024-06-13T06:56:32.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.755+0000 7f6675e92b80 20 client.25357 trim_cache size 0 max 16384 2024-06-13T06:56:32.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.787+0000 7f6675e92b80 1 Processor -- start 2024-06-13T06:56:32.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 -- start start 2024-06-13T06:56:32.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 0x55a241a42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66641c7e70 con 0x55a241a3a240 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66641c56d0 con 0x55a241a3fb90 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.788+0000 7f6675e92b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f666411c7f0 con 0x55a241a404b0 2024-06-13T06:56:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6663e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 0x55a241a42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6663e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 0x55a241a42440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47984/0 (socket says 172.21.15.7:47984) 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6663e00700 1 -- 172.21.15.7:0/4156151999 learned_addr learned my addr 172.21.15.7:0/4156151999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6668c00700 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 -- 172.21.15.7:0/4156151999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 msgr2=0x55a241a42440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 0x55a241a42440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 -- 172.21.15.7:0/4156151999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 msgr2=0x55a241a39d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a39d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.789+0000 7f6669600700 1 -- 172.21.15.7:0/4156151999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a241a56c40 con 0x55a241a404b0 2024-06-13T06:56:32.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.791+0000 7f6669600700 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a2f130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f665c01a010 tx=0x7f665c014720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d97e91e6a5b27b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.791+0000 7f6655600700 1 -- 172.21.15.7:0/4156151999 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c022440 con 0x55a241a404b0 2024-06-13T06:56:32.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.791+0000 7f6655600700 1 -- 172.21.15.7:0/4156151999 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c027070 con 0x55a241a404b0 2024-06-13T06:56:32.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.791+0000 7f6655600700 1 -- 172.21.15.7:0/4156151999 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c010dd0 con 0x55a241a404b0 2024-06-13T06:56:32.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.792+0000 7f6675e92b80 1 -- 172.21.15.7:0/4156151999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 msgr2=0x55a241a2f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.792+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a2f130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f665c01a010 tx=0x7f665c014720 comp rx=0 tx=0).stop 2024-06-13T06:56:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.793+0000 7f6675e92b80 1 -- 172.21.15.7:0/4156151999 shutdown_connections 2024-06-13T06:56:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.793+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a404b0 0x55a241a2f130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.793+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a39d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.793+0000 7f6675e92b80 1 --2- 172.21.15.7:0/4156151999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a3fb90 0x55a241a42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.793+0000 7f6675e92b80 1 -- 172.21.15.7:0/4156151999 >> 172.21.15.7:0/4156151999 conn(0x55a241966170 msgr2=0x55a241a3d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.794+0000 7f6675e92b80 1 -- 172.21.15.7:0/4156151999 shutdown_connections 2024-06-13T06:56:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.794+0000 7f6675e92b80 1 -- 172.21.15.7:0/4156151999 wait complete. 2024-06-13T06:56:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.795+0000 7f6675e92b80 1 -- start start 2024-06-13T06:56:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.796+0000 7f6675e92b80 1 Processor -- start 2024-06-13T06:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.796+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a4c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.797+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.797+0000 7f6675e92b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a5aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.797+0000 7f6669600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a5aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.797+0000 7f6663e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.797+0000 7f6663e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36482/0 (socket says 172.21.15.7:36482) 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.798+0000 7f6669600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a5aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47992/0 (socket says 172.21.15.7:47992) 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.798+0000 7f6663e00700 1 -- 172.21.15.7:0/92178669 learned_addr learned my addr 172.21.15.7:0/92178669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.798+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66641c3b90 con 0x55a241a3a240 2024-06-13T06:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.799+0000 7f6668c00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a4c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.799+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66641c5000 con 0x55a241a404b0 2024-06-13T06:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.799+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66641c4260 con 0x55a241a3fb90 2024-06-13T06:56:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6675e92b80 10 client.0 _ll_register_callbacks cb 0x55a241921380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6663e00700 1 -- 172.21.15.7:0/92178669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 msgr2=0x55a241a5aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6663e00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a5aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6663e00700 1 -- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 msgr2=0x55a241a4c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6663e00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a4c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.800+0000 7f6663e00700 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a241a5c920 con 0x55a241a3fb90 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.801+0000 7f6663e00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66500cdc50 tx=0x7f66500d3bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265a419bc5aa3416 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.802+0000 7f664aa00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.803+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650098d10 con 0x55a241a3fb90 2024-06-13T06:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.803+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f66500a51c0 con 0x55a241a3fb90 2024-06-13T06:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.803+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f665c027070 con 0x55a241a3fb90 2024-06-13T06:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.804+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f665009c3d0 con 0x55a241a3fb90 2024-06-13T06:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.804+0000 7f6675e92b80 10 client.25363 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.804+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a241abe240 con 0x55a241a3fb90 2024-06-13T06:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.804+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c027070 con 0x55a241a3fb90 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f66500a30b0 con 0x55a241a3fb90 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 10 client.25363 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 10 client.25363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 20 client.25363 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241abe920 0x55a241ac0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 4 client.25363 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.806+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a241993580 con 0x55a241abe920 2024-06-13T06:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.807+0000 7f6668c00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241abe920 0x55a241ac0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.808+0000 7f6668c00700 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241abe920 0x55a241ac0d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd2e89901fdbd694 server_cookie=418f77d55dcbec6a in_seq=0 out_seq=0 2024-06-13T06:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.808+0000 7f664aa00700 10 client.25363 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:56:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f664aa00700 1 -- 172.21.15.7:0/92178669 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a241993580 con 0x55a241abe920 2024-06-13T06:56:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f664aa00700 10 client.25363 handle_command_reply: tid=0 2024-06-13T06:56:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 client.25363 shutdown 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 msgr2=0x55a241a4c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66500cdc50 tx=0x7f66500d3bb0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 shutdown_connections 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a241a404b0 0x55a241a5aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a241a3a240 0x55a241a4c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55a241abe920 0x55a241ac0d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f664aa00700 0 client.25363 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.809+0000 7f6675e92b80 1 --2- 172.21.15.7:0/92178669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a241a3fb90 0x55a241a4c6f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.810+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 >> 172.21.15.7:0/92178669 conn(0x55a241966170 msgr2=0x55a241a40890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.811+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 shutdown_connections 2024-06-13T06:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.811+0000 7f6675e92b80 1 -- 172.21.15.7:0/92178669 wait complete. 2024-06-13T06:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.811+0000 7f6675e92b80 20 client.25363 trim_cache size 0 max 16384 2024-06-13T06:56:32.814 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 msgr2=0x7f664c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 0x7f664c0a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f665c014c60 tx=0x7f665c007f30 comp rx=0 tx=0).stop 2024-06-13T06:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 msgr2=0x7f6664073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66500057d0 tx=0x7f6650005800 comp rx=0 tx=0).stop 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 shutdown_connections 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f664c0a2ca0 0x7f664c0a5160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6664108500 0x7f666406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f666406a170 0x7f6664073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 --2- 172.21.15.7:0/1267150223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666406ab30 0x7f6664073f50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 >> 172.21.15.7:0/1267150223 conn(0x7f6664078340 msgr2=0x7f66640ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 shutdown_connections 2024-06-13T06:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:32.815+0000 7f666b600700 1 -- 172.21.15.7:0/1267150223 wait complete. 2024-06-13T06:56:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 30.0 s, 1672.0 tps, lat 57.113 ms stddev 71.898 2024-06-13T06:56:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:33 smithi189 ceph-mon[36616]: pgmap v524: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 727 op/s 2024-06-13T06:56:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:33 smithi175 ceph-mon[36886]: pgmap v524: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 727 op/s 2024-06-13T06:56:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:33 smithi007 ceph-mon[32077]: pgmap v524: 129 pgs: 129 active+clean; 941 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 727 op/s 2024-06-13T06:56:34.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T06:56:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:34 smithi189 ceph-mon[36616]: pgmap v525: 129 pgs: 129 active+clean; 942 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 744 op/s 2024-06-13T06:56:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:34 smithi175 ceph-mon[36886]: pgmap v525: 129 pgs: 129 active+clean; 942 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 744 op/s 2024-06-13T06:56:34.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:34 smithi007 ceph-mon[32077]: pgmap v525: 129 pgs: 129 active+clean; 942 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 744 op/s 2024-06-13T06:56:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:36.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:36 smithi007 ceph-mon[32077]: pgmap v526: 129 pgs: 129 active+clean; 946 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-06-13T06:56:37.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:36 smithi175 ceph-mon[36886]: pgmap v526: 129 pgs: 129 active+clean; 946 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-06-13T06:56:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:36 smithi189 ceph-mon[36616]: pgmap v526: 129 pgs: 129 active+clean; 946 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-06-13T06:56:37.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 35.0 s, 1656.7 tps, lat 62.937 ms stddev 90.226 2024-06-13T06:56:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.976+0000 7fcac3200700 1 -- 172.21.15.7:0/4197015435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac410ed20 msgr2=0x7fcac4111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.977+0000 7fcac3200700 1 --2- 172.21.15.7:0/4197015435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac410ed20 0x7fcac4111110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009990 tx=0x7fcabc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.980 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:38 smithi007 ceph-mon[32077]: pgmap v527: 129 pgs: 129 active+clean; 947 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 -- 172.21.15.7:0/4197015435 shutdown_connections 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 --2- 172.21.15.7:0/4197015435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac410ed20 0x7fcac4111110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 --2- 172.21.15.7:0/4197015435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac4101170 0x7fcac4101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 --2- 172.21.15.7:0/4197015435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 -- 172.21.15.7:0/4197015435 >> 172.21.15.7:0/4197015435 conn(0x7fcac40fcdb0 msgr2=0x7fcac40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.978+0000 7fcac3200700 1 -- 172.21.15.7:0/4197015435 shutdown_connections 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.979+0000 7fcac3200700 1 -- 172.21.15.7:0/4197015435 wait complete. 2024-06-13T06:56:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.981+0000 7fcac3200700 1 Processor -- start 2024-06-13T06:56:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.982+0000 7fcac3200700 1 -- start start 2024-06-13T06:56:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.982+0000 7fcac3200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 0x7fcac419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.982+0000 7fcac3200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcac3200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 0x7fcac419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcac3200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcac41149c0 con 0x7fcac4101b20 2024-06-13T06:56:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcac3200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4114840 con 0x7fcac4101170 2024-06-13T06:56:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcac3200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4114b40 con 0x7fcac410ed20 2024-06-13T06:56:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcabbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcac2800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 0x7fcac419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcabbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34214/0 (socket says 172.21.15.7:34214) 2024-06-13T06:56:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.983+0000 7fcabbe00700 1 -- 172.21.15.7:0/3670144034 learned_addr learned my addr 172.21.15.7:0/3670144034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac1e00700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 0x7fcac419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac2800700 1 -- 172.21.15.7:0/3670144034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 msgr2=0x7fcac419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac2800700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 0x7fcac419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac2800700 1 -- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 msgr2=0x7fcac419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac2800700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac2800700 1 -- 172.21.15.7:0/3670144034 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac419fdc0 con 0x7fcac410ed20 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.984+0000 7fcac1e00700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 0x7fcac419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.985+0000 7fcac2800700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 0x7fcac419f680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcabc006010 tx=0x7fcabc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.985+0000 7fcabaa00700 1 -- 172.21.15.7:0/3670144034 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc026070 con 0x7fcac410ed20 2024-06-13T06:56:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.986+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac41a0050 con 0x7fcac410ed20 2024-06-13T06:56:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.986+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac41a7930 con 0x7fcac410ed20 2024-06-13T06:56:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.986+0000 7fcabaa00700 1 -- 172.21.15.7:0/3670144034 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc021410 con 0x7fcac410ed20 2024-06-13T06:56:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.986+0000 7fcabaa00700 1 -- 172.21.15.7:0/3670144034 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc017630 con 0x7fcac410ed20 2024-06-13T06:56:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.990+0000 7fcabaa00700 1 -- 172.21.15.7:0/3670144034 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcabc029070 con 0x7fcac410ed20 2024-06-13T06:56:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.992+0000 7fcabaa00700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.992+0000 7fcabaa00700 1 -- 172.21.15.7:0/3670144034 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcabc0cfda0 con 0x7fcac410ed20 2024-06-13T06:56:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.994+0000 7fcac1e00700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.995+0000 7fcad32f0b80 1 Processor -- start 2024-06-13T06:56:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.996+0000 7fcad32f0b80 1 -- start start 2024-06-13T06:56:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.996+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab67140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.996+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab71ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.996+0000 7fcac1e00700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009480 tx=0x7fcab400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559baaa76a10 con 0x559baab66d60 2024-06-13T06:56:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559baaab9b00 con 0x559baab72530 2024-06-13T06:56:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559baaa748f0 con 0x559baab67680 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcabbe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab71ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcac1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab67140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.997+0000 7fcac2800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48014/0 (socket says 172.21.15.7:48014) 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 -- 172.21.15.7:0/524491032 learned_addr learned my addr 172.21.15.7:0/524491032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 -- 172.21.15.7:0/524491032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 msgr2=0x559baab71ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab71ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 -- 172.21.15.7:0/524491032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 msgr2=0x559baab67140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab67140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac2800700 1 -- 172.21.15.7:0/524491032 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559baaa61f90 con 0x559baab72530 2024-06-13T06:56:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcabbe00700 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab71ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.998+0000 7fcac1e00700 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab67140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:38.999+0000 7fcac2800700 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcabc09c280 tx=0x7fcabc09bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e33c90fbadcb5da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:39.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.000+0000 7fcab3400700 1 -- 172.21.15.7:0/524491032 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc096e60 con 0x559baab72530 2024-06-13T06:56:39.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.000+0000 7fcab3400700 1 -- 172.21.15.7:0/524491032 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc0a06a0 con 0x559baab72530 2024-06-13T06:56:39.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.000+0000 7fcab3400700 1 -- 172.21.15.7:0/524491032 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0bc320 con 0x559baab72530 2024-06-13T06:56:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 -- 172.21.15.7:0/524491032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 msgr2=0x559baab74920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcabc09c280 tx=0x7fcabc09bd00 comp rx=0 tx=0).stop 2024-06-13T06:56:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 -- 172.21.15.7:0/524491032 shutdown_connections 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab74920 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab67140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/524491032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab71ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.001+0000 7fcad32f0b80 1 -- 172.21.15.7:0/524491032 >> 172.21.15.7:0/524491032 conn(0x559baaa94170 msgr2=0x559baab63280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.002+0000 7fcad32f0b80 1 -- 172.21.15.7:0/524491032 shutdown_connections 2024-06-13T06:56:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.002+0000 7fcad32f0b80 1 -- 172.21.15.7:0/524491032 wait complete. 2024-06-13T06:56:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.003+0000 7fcad32f0b80 1 -- start start 2024-06-13T06:56:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.003+0000 7fcad32f0b80 1 Processor -- start 2024-06-13T06:56:39.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab71bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559baaa55f60 con 0x559baab66d60 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559baaab5ac0 con 0x559baab72530 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559baaab9b00 con 0x559baab67680 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.004+0000 7fcad32f0b80 10 client.0 _ll_register_callbacks cb 0x559baaa4f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.005+0000 7fcabbe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.005+0000 7fcabbe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36506/0 (socket says 172.21.15.7:36506) 2024-06-13T06:56:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.005+0000 7fcabbe00700 1 -- 172.21.15.7:0/858770765 learned_addr learned my addr 172.21.15.7:0/858770765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.005+0000 7fcac2800700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcac1e00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab71bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcabbe00700 1 -- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 msgr2=0x559baab6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcabbe00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcabbe00700 1 -- 172.21.15.7:0/858770765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 msgr2=0x559baab71bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcabbe00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab71bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.006+0000 7fcabbe00700 1 -- 172.21.15.7:0/858770765 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559baaa088d0 con 0x559baab67680 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.007+0000 7fcabbe00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0104a0 tx=0x7fcaac0107b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5e5b6cba12580f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.007+0000 7fcaa7e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.007+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018070 con 0x559baab67680 2024-06-13T06:56:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.008+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcaac014800 con 0x559baab67680 2024-06-13T06:56:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.008+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc0a06a0 con 0x559baab67680 2024-06-13T06:56:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.009+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaac022050 con 0x559baab67680 2024-06-13T06:56:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.009+0000 7fcad32f0b80 10 client.25372 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.009+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559baab7f3d0 con 0x559baab67680 2024-06-13T06:56:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.009+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc0a06a0 con 0x559baab67680 2024-06-13T06:56:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fcaac01dac0 con 0x559baab67680 2024-06-13T06:56:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 10 client.25372 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 10 client.25372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 20 client.25372 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baab81a60 0x559baab83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 4 client.25372 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559baaac7000 con 0x559baab81a60 2024-06-13T06:56:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.010+0000 7fcac1e00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baab81a60 0x559baab83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.011+0000 7fcaa7e00700 10 client.25372 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.011+0000 7fcac1e00700 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baab81a60 0x559baab83ea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a51af37c85b600 server_cookie=2088f11d7af82e0d in_seq=0 out_seq=0 2024-06-13T06:56:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.012+0000 7fcaa7e00700 1 -- 172.21.15.7:0/858770765 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559baaac7000 con 0x559baab81a60 2024-06-13T06:56:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.012+0000 7fcaa7e00700 10 client.25372 handle_command_reply: tid=0 2024-06-13T06:56:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.013+0000 7fcad32f0b80 1 client.25372 shutdown 2024-06-13T06:56:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.013+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 msgr2=0x559baab6eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.013+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0104a0 tx=0x7fcaac0107b0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 shutdown_connections 2024-06-13T06:56:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baab81a60 0x559baab83ea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab72530 0x559baab6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab71bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcaa7e00700 0 client.25372 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/858770765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab67680 0x559baab6eef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 >> 172.21.15.7:0/858770765 conn(0x559baaa94170 msgr2=0x559baab62e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 shutdown_connections 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.014+0000 7fcad32f0b80 1 -- 172.21.15.7:0/858770765 wait complete. 2024-06-13T06:56:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.015+0000 7fcad32f0b80 20 client.25372 trim_cache size 0 max 16384 2024-06-13T06:56:39.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.045+0000 7fcad32f0b80 1 Processor -- start 2024-06-13T06:56:39.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.045+0000 7fcad32f0b80 1 -- start start 2024-06-13T06:56:39.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab6f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0011230 con 0x559baab66d60 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa000ea90 con 0x559baab67680 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.046+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa000c8e0 con 0x559baab72530 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.047+0000 7fcac2800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.047+0000 7fcac2800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36522/0 (socket says 172.21.15.7:36522) 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.047+0000 7fcac2800700 1 -- 172.21.15.7:0/1624852653 learned_addr learned my addr 172.21.15.7:0/1624852653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:39.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.047+0000 7fcabbe00700 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 -- 172.21.15.7:0/1624852653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 msgr2=0x559baab897b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab897b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 -- 172.21.15.7:0/1624852653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 msgr2=0x559baab6f460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab6f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 -- 172.21.15.7:0/1624852653 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559baab84520 con 0x559baab72530 2024-06-13T06:56:39.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.048+0000 7fcac2800700 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009990 tx=0x7fcabc098590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d18bb1e00e6a722e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:39.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.049+0000 7fcab2a00700 1 -- 172.21.15.7:0/1624852653 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0b4070 con 0x559baab72530 2024-06-13T06:56:39.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.050+0000 7fcab2a00700 1 -- 172.21.15.7:0/1624852653 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc0a06a0 con 0x559baab72530 2024-06-13T06:56:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.050+0000 7fcab2a00700 1 -- 172.21.15.7:0/1624852653 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc096e60 con 0x559baab72530 2024-06-13T06:56:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.051+0000 7fcad32f0b80 1 -- 172.21.15.7:0/1624852653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 msgr2=0x559baab89cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.051+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009990 tx=0x7fcabc098590 comp rx=0 tx=0).stop 2024-06-13T06:56:39.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.051+0000 7fcad32f0b80 1 -- 172.21.15.7:0/1624852653 shutdown_connections 2024-06-13T06:56:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab72530 0x559baab89cf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab66d60 0x559baab6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/1624852653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab897b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 -- 172.21.15.7:0/1624852653 >> 172.21.15.7:0/1624852653 conn(0x559baaa94170 msgr2=0x559baab6d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 -- 172.21.15.7:0/1624852653 shutdown_connections 2024-06-13T06:56:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.052+0000 7fcad32f0b80 1 -- 172.21.15.7:0/1624852653 wait complete. 2024-06-13T06:56:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.053+0000 7fcad32f0b80 1 -- start start 2024-06-13T06:56:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.053+0000 7fcad32f0b80 1 Processor -- start 2024-06-13T06:56:39.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab72530 0x559baab74fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa000cf50 con 0x559baab72530 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa000e3c0 con 0x559baab67680 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa000d620 con 0x559baab66d60 2024-06-13T06:56:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcad32f0b80 10 client.0 _ll_register_callbacks cb 0x559baaa4f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcac1e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcac1e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36536/0 (socket says 172.21.15.7:36536) 2024-06-13T06:56:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcac1e00700 1 -- 172.21.15.7:0/2362838183 learned_addr learned my addr 172.21.15.7:0/2362838183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.054+0000 7fcabbe00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.055+0000 7fcabbe00700 1 -- 172.21.15.7:0/2362838183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 msgr2=0x559baab79ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.055+0000 7fcabbe00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.055+0000 7fcabbe00700 1 -- 172.21.15.7:0/2362838183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab72530 msgr2=0x559baab74fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.056+0000 7fcac2800700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab72530 0x559baab74fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.056+0000 7fcabbe00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab72530 0x559baab74fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.056+0000 7fcabbe00700 1 -- 172.21.15.7:0/2362838183 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559baab7f200 con 0x559baab67680 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.056+0000 7fcac1e00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.057+0000 7fcabbe00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab7a530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcaac01c450 tx=0x7fcaac004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1969cdb6801eb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.057+0000 7fcaa7e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.057+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018040 con 0x559baab67680 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.057+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcaac003940 con 0x559baab67680 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.057+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc0a06a0 con 0x559baab67680 2024-06-13T06:56:39.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.058+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaac036050 con 0x559baab67680 2024-06-13T06:56:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.058+0000 7fcad32f0b80 10 client.25382 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.058+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559baabf53a0 con 0x559baab67680 2024-06-13T06:56:39.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.059+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc0a06a0 con 0x559baab67680 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fcaac014ae0 con 0x559baab67680 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 10 client.25382 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 20 client.25382 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baabf5b00 0x559baabf7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 4 client.25382 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559baaac1580 con 0x559baabf5b00 2024-06-13T06:56:39.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.060+0000 7fcac1e00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baabf5b00 0x559baabf7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.061+0000 7fcac1e00700 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baabf5b00 0x559baabf7f20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c186e0d1f77e9a server_cookie=8b1beef8015e55ec in_seq=0 out_seq=0 2024-06-13T06:56:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.061+0000 7fcaa7e00700 10 client.25382 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.061+0000 7fcaa7e00700 1 -- 172.21.15.7:0/2362838183 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559baaac1580 con 0x559baabf5b00 2024-06-13T06:56:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.061+0000 7fcaa7e00700 10 client.25382 handle_command_reply: tid=0 2024-06-13T06:56:39.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.061+0000 7fcad32f0b80 1 client.25382 shutdown 2024-06-13T06:56:39.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 msgr2=0x559baab7a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab7a530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcaac01c450 tx=0x7fcaac004270 comp rx=0 tx=0).stop 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 shutdown_connections 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559baabf5b00 0x559baabf7f20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559baab72530 0x559baab74fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559baab66d60 0x559baab79ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 --2- 172.21.15.7:0/2362838183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559baab67680 0x559baab7a530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 >> 172.21.15.7:0/2362838183 conn(0x559baaa94170 msgr2=0x559baab70720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcaa7e00700 0 client.25382 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 shutdown_connections 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 1 -- 172.21.15.7:0/2362838183 wait complete. 2024-06-13T06:56:39.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.062+0000 7fcad32f0b80 20 client.25382 trim_cache size 0 max 16384 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 msgr2=0x7fcaa80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009480 tx=0x7fcab400b040 comp rx=0 tx=0).stop 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 msgr2=0x7fcac419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 0x7fcac419f680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcabc006010 tx=0x7fcabc0057c0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 shutdown_connections 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcac410ed20 0x7fcac419f680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcac4101170 0x7fcac419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 --2- 172.21.15.7:0/3670144034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac4101b20 0x7fcac419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 >> 172.21.15.7:0/3670144034 conn(0x7fcac40fcdb0 msgr2=0x7fcac4110f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 shutdown_connections 2024-06-13T06:56:39.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:39.064+0000 7fcac3200700 1 -- 172.21.15.7:0/3670144034 wait complete. 2024-06-13T06:56:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:38 smithi175 ceph-mon[36886]: pgmap v527: 129 pgs: 129 active+clean; 947 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-06-13T06:56:39.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:38 smithi189 ceph-mon[36616]: pgmap v527: 129 pgs: 129 active+clean; 947 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 880 op/s 2024-06-13T06:56:41.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:40 smithi175 ceph-mon[36886]: pgmap v528: 129 pgs: 129 active+clean; 959 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 17 MiB/s wr, 912 op/s 2024-06-13T06:56:41.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:40 smithi007 ceph-mon[32077]: pgmap v528: 129 pgs: 129 active+clean; 959 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 17 MiB/s wr, 912 op/s 2024-06-13T06:56:41.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:40 smithi189 ceph-mon[36616]: pgmap v528: 129 pgs: 129 active+clean; 959 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 17 MiB/s wr, 912 op/s 2024-06-13T06:56:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 -- 172.21.15.7:0/1210685030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48102c00 msgr2=0x7f2c48103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1210685030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48102c00 0x7f2c48103080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 -- 172.21.15.7:0/1210685030 shutdown_connections 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1210685030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481035c0 0x7f2c48109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1210685030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48102c00 0x7f2c48103080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1210685030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c48101a10 0x7f2c48101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.417+0000 7f2c4d200640 1 -- 172.21.15.7:0/1210685030 >> 172.21.15.7:0/1210685030 conn(0x7f2c480fcf60 msgr2=0x7f2c480ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.418+0000 7f2c4d200640 1 -- 172.21.15.7:0/1210685030 shutdown_connections 2024-06-13T06:56:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.418+0000 7f2c4d200640 1 -- 172.21.15.7:0/1210685030 wait complete. 2024-06-13T06:56:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 Processor -- start 2024-06-13T06:56:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 -- start start 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48101a10 0x7f2c48197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 0x7f2c48198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810d6e0 con 0x7f2c48102c00 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810d560 con 0x7f2c48101a10 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.420+0000 7f2c4d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810d860 con 0x7f2c481035c0 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.421+0000 7f2c46800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.421+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48101a10 0x7f2c48197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.421+0000 7f2c46800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51990/0 (socket says 172.21.15.7:51990) 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.421+0000 7f2c46800640 1 -- 172.21.15.7:0/1597372947 learned_addr learned my addr 172.21.15.7:0/1597372947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.421+0000 7f2c45400640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 0x7f2c48198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c46800640 1 -- 172.21.15.7:0/1597372947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48101a10 msgr2=0x7f2c48197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c46800640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48101a10 0x7f2c48197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c46800640 1 -- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 msgr2=0x7f2c48198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c46800640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 0x7f2c48198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c46800640 1 -- 172.21.15.7:0/1597372947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4819d1b0 con 0x7f2c481035c0 2024-06-13T06:56:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.422+0000 7f2c45400640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 0x7f2c48198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.423+0000 7f2c46800640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000cce0 tx=0x7f2c4000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.423+0000 7f2c3ea00640 1 -- 172.21.15.7:0/1597372947 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000f450 con 0x7f2c481035c0 2024-06-13T06:56:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.424+0000 7f2c3ea00640 1 -- 172.21.15.7:0/1597372947 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4000fad0 con 0x7f2c481035c0 2024-06-13T06:56:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.424+0000 7f2c3ea00640 1 -- 172.21.15.7:0/1597372947 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40018c10 con 0x7f2c481035c0 2024-06-13T06:56:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.424+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4819d4a0 con 0x7f2c481035c0 2024-06-13T06:56:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.424+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4819d9a0 con 0x7f2c481035c0 2024-06-13T06:56:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.433+0000 7f2c3ea00640 1 -- 172.21.15.7:0/1597372947 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2c4000f5f0 con 0x7f2c481035c0 2024-06-13T06:56:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.435+0000 7f2c3ea00640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 0x7f2c2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.436+0000 7f2c3ea00640 1 -- 172.21.15.7:0/1597372947 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2c400cf920 con 0x7f2c481035c0 2024-06-13T06:56:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.436+0000 7f2c45e00640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 0x7f2c2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.437+0000 7f2c5007e740 1 Processor -- start 2024-06-13T06:56:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 -- start start 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x563607215020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 0x56360721a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 0x56360721cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563607178420 con 0x563607214c00 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56360711fc50 con 0x5636072155f0 2024-06-13T06:56:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.438+0000 7f2c5007e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636071743e0 con 0x56360721a760 2024-06-13T06:56:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45e00640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 0x7f2c2c0a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c48199120 tx=0x7f2c3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 0x56360721a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c46800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 0x56360721cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c46800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 0x56360721cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51998/0 (socket says 172.21.15.7:51998) 2024-06-13T06:56:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c46800640 1 -- 172.21.15.7:0/3872565052 learned_addr learned my addr 172.21.15.7:0/3872565052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 -- 172.21.15.7:0/3872565052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 msgr2=0x56360721cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 0x56360721cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 -- 172.21.15.7:0/3872565052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 msgr2=0x563607215020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x563607215020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.439+0000 7f2c45400640 1 -- 172.21.15.7:0/3872565052 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563606e8c210 con 0x5636072155f0 2024-06-13T06:56:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.440+0000 7f2c45400640 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 0x56360721a0f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402a930 tx=0x7f2c34004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6b43499a73c3fa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.440+0000 7f2c33400640 1 -- 172.21.15.7:0/3872565052 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x5636072155f0 2024-06-13T06:56:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.441+0000 7f2c33400640 1 -- 172.21.15.7:0/3872565052 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c34005480 con 0x5636072155f0 2024-06-13T06:56:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.441+0000 7f2c33400640 1 -- 172.21.15.7:0/3872565052 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34031cb0 con 0x5636072155f0 2024-06-13T06:56:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.441+0000 7f2c5007e740 1 -- 172.21.15.7:0/3872565052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 msgr2=0x56360721a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 0x56360721a0f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c3402a930 tx=0x7f2c34004610 comp rx=0 tx=0).stop 2024-06-13T06:56:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 -- 172.21.15.7:0/3872565052 shutdown_connections 2024-06-13T06:56:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56360721a760 0x56360721cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5636072155f0 0x56360721a0f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3872565052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x563607215020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.442+0000 7f2c5007e740 1 -- 172.21.15.7:0/3872565052 >> 172.21.15.7:0/3872565052 conn(0x5636071dcfb0 msgr2=0x563607210680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.443+0000 7f2c5007e740 1 -- 172.21.15.7:0/3872565052 shutdown_connections 2024-06-13T06:56:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.443+0000 7f2c5007e740 1 -- 172.21.15.7:0/3872565052 wait complete. 2024-06-13T06:56:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.444+0000 7f2c5007e740 1 -- start start 2024-06-13T06:56:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.446+0000 7f2c5007e740 1 Processor -- start 2024-06-13T06:56:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x56360721ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x5636072264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56360721d6b0 con 0x563607214c00 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5636071330b0 con 0x56360721a760 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56360721d830 con 0x5636072155f0 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c5007e740 10 client.0 _ll_register_callbacks cb 0x56360711ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c45400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c45400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52004/0 (socket says 172.21.15.7:52004) 2024-06-13T06:56:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.448+0000 7f2c45400640 1 -- 172.21.15.7:0/3701365841 learned_addr learned my addr 172.21.15.7:0/3701365841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 -- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 msgr2=0x5636072264b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x5636072264b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 -- 172.21.15.7:0/3701365841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 msgr2=0x56360721ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x56360721ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 -- 172.21.15.7:0/3701365841 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563606f58620 con 0x5636072155f0 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c45400640 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c34031440 tx=0x7f2c34031c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33b20309093eb26 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.449+0000 7f2c27e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.450+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34052060 con 0x5636072155f0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.450+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2c34004980 con 0x5636072155f0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.450+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34005480 con 0x5636072155f0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.450+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x5636072155f0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.451+0000 7f2c5007e740 10 client.25390 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.451+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56360721d6b0 con 0x5636072155f0 2024-06-13T06:56:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.451+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c34005480 con 0x5636072155f0 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.451+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f2c3404c070 con 0x5636072155f0 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 10 client.25390 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 20 client.25390 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56360722ab00 0x56360722cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 4 client.25390 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563607185b50 con 0x56360722ab00 2024-06-13T06:56:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.452+0000 7f2c45e00640 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56360722ab00 0x56360722cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.453+0000 7f2c45e00640 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56360722ab00 0x56360722cf50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24605ac5d27ad6b server_cookie=be239e3030415bbf in_seq=0 out_seq=0 2024-06-13T06:56:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.453+0000 7f2c27e00640 10 client.25390 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.454+0000 7f2c27e00640 1 -- 172.21.15.7:0/3701365841 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563607185b50 con 0x56360722ab00 2024-06-13T06:56:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.454+0000 7f2c27e00640 10 client.25390 handle_command_reply: tid=0 2024-06-13T06:56:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.454+0000 7f2c5007e740 1 client.25390 shutdown 2024-06-13T06:56:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.454+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 msgr2=0x56360721f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.454+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c34031440 tx=0x7f2c34031c20 comp rx=0 tx=0).stop 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 shutdown_connections 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56360722ab00 0x56360722cf50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x5636072264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636072155f0 0x56360721f470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3701365841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563607214c00 0x56360721ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c27e00640 0 client.25390 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 >> 172.21.15.7:0/3701365841 conn(0x5636071dcfb0 msgr2=0x56360720f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 shutdown_connections 2024-06-13T06:56:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 1 -- 172.21.15.7:0/3701365841 wait complete. 2024-06-13T06:56:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.455+0000 7f2c5007e740 20 client.25390 trim_cache size 0 max 16384 2024-06-13T06:56:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.490+0000 7f2c5007e740 1 Processor -- start 2024-06-13T06:56:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.490+0000 7f2c5007e740 1 -- start start 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607210810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607226470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b3240 con 0x5636072155f0 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b0ae0 con 0x56360721a760 2024-06-13T06:56:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c5007e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481ae930 con 0x563607214c00 2024-06-13T06:56:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c46800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.491+0000 7f2c46800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36334/0 (socket says 172.21.15.7:36334) 2024-06-13T06:56:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 -- 172.21.15.7:0/3925758688 learned_addr learned my addr 172.21.15.7:0/3925758688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607210810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c45400640 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607226470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 -- 172.21.15.7:0/3925758688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 msgr2=0x563607210810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607210810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 -- 172.21.15.7:0/3925758688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 msgr2=0x563607226470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607226470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.492+0000 7f2c46800640 1 -- 172.21.15.7:0/3925758688 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360722d5d0 con 0x56360721a760 2024-06-13T06:56:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.493+0000 7f2c46800640 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c40018550 tx=0x7f2c4000af30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6216c9e404a84b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.493+0000 7f2c32a00640 1 -- 172.21.15.7:0/3925758688 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4009e030 con 0x56360721a760 2024-06-13T06:56:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.494+0000 7f2c32a00640 1 -- 172.21.15.7:0/3925758688 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c400a01f0 con 0x56360721a760 2024-06-13T06:56:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.494+0000 7f2c32a00640 1 -- 172.21.15.7:0/3925758688 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4009e030 con 0x56360721a760 2024-06-13T06:56:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.496+0000 7f2c5007e740 1 -- 172.21.15.7:0/3925758688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 msgr2=0x56360721e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.496+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c40018550 tx=0x7f2c4000af30 comp rx=0 tx=0).stop 2024-06-13T06:56:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 -- 172.21.15.7:0/3925758688 shutdown_connections 2024-06-13T06:56:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x56360721e550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607226470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 --2- 172.21.15.7:0/3925758688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607210810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 -- 172.21.15.7:0/3925758688 >> 172.21.15.7:0/3925758688 conn(0x5636071dcfb0 msgr2=0x563607210100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 -- 172.21.15.7:0/3925758688 shutdown_connections 2024-06-13T06:56:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.498+0000 7f2c5007e740 1 -- 172.21.15.7:0/3925758688 wait complete. 2024-06-13T06:56:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.499+0000 7f2c5007e740 1 -- start start 2024-06-13T06:56:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.499+0000 7f2c5007e740 1 Processor -- start 2024-06-13T06:56:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.501+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607221800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.501+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607221d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.501+0000 7f2c5007e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.501+0000 7f2c5007e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481aefa0 con 0x5636072155f0 2024-06-13T06:56:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c46800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c5007e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481b0410 con 0x56360721a760 2024-06-13T06:56:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c5007e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481af670 con 0x563607214c00 2024-06-13T06:56:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c46800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36344/0 (socket says 172.21.15.7:36344) 2024-06-13T06:56:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c46800640 1 -- 172.21.15.7:0/2087592673 learned_addr learned my addr 172.21.15.7:0/2087592673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c5007e740 10 client.0 _ll_register_callbacks cb 0x7f2c40018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.501+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607221800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c45400640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607221d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.502+0000 7f2c45e00640 1 -- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 msgr2=0x563607224ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c45e00640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c45e00640 1 -- 172.21.15.7:0/2087592673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 msgr2=0x563607221d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c45e00640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607221d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c45e00640 1 -- 172.21.15.7:0/2087592673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563607225290 con 0x563607214c00 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c46800640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c27e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.503+0000 7f2c45e00640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607221800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56360722d7a0 tx=0x7f2c380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee02233f64ab7521 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.504+0000 7f2c45400640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607221d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.504+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3802f670 con 0x563607214c00 2024-06-13T06:56:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.504+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2c3802f470 con 0x563607214c00 2024-06-13T06:56:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.504+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c400a01f0 con 0x563607214c00 2024-06-13T06:56:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.505+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c3802f670 con 0x563607214c00 2024-06-13T06:56:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.505+0000 7f2c5007e740 10 client.25396 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.505+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563607226670 con 0x563607214c00 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.505+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3802a070 con 0x563607214c00 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.506+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f2c3801e540 con 0x563607214c00 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 10 client.25396 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 20 client.25396 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5636072a6ee0 0x5636072a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 4 client.25396 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:56:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.507+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2c400a01f0 con 0x5636072a6ee0 2024-06-13T06:56:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.508+0000 7f2c45400640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5636072a6ee0 0x5636072a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.510+0000 7f2c45400640 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5636072a6ee0 0x5636072a9300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6525f4e8a94d74b4 server_cookie=f1af6ed0226e422b in_seq=0 out_seq=0 2024-06-13T06:56:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.510+0000 7f2c27e00640 10 client.25396 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.580+0000 7f2c27e00640 1 -- 172.21.15.7:0/2087592673 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f2c400a01f0 con 0x5636072a6ee0 2024-06-13T06:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.580+0000 7f2c27e00640 10 client.25396 handle_command_reply: tid=0 2024-06-13T06:56:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.580+0000 7f2c5007e740 1 client.25396 shutdown 2024-06-13T06:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.581+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 msgr2=0x563607221800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.581+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607221800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56360722d7a0 tx=0x7f2c380046c0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.581+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 shutdown_connections 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.581+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5636072a6ee0 0x5636072a9300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56360721a760 0x563607224ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636072155f0 0x563607221d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 --2- 172.21.15.7:0/2087592673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563607214c00 0x563607221800 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 >> 172.21.15.7:0/2087592673 conn(0x5636071dcfb0 msgr2=0x563607210100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c27e00640 0 client.25396 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 shutdown_connections 2024-06-13T06:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 1 -- 172.21.15.7:0/2087592673 wait complete. 2024-06-13T06:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.582+0000 7f2c5007e740 20 client.25396 trim_cache size 0 max 16384 2024-06-13T06:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:56:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.584+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 msgr2=0x7f2c2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.584+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 0x7f2c2c0a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2c48199120 tx=0x7f2c3800b390 comp rx=0 tx=0).stop 2024-06-13T06:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.584+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 msgr2=0x7f2c481a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.584+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000cce0 tx=0x7f2c4000a620 comp rx=0 tx=0).stop 2024-06-13T06:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 shutdown_connections 2024-06-13T06:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2c2c0a2cb0 0x7f2c2c0a5170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c481035c0 0x7f2c481a15c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c48102c00 0x7f2c48198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 --2- 172.21.15.7:0/1597372947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c48101a10 0x7f2c48197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 >> 172.21.15.7:0/1597372947 conn(0x7f2c480fcf60 msgr2=0x7f2c48105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 shutdown_connections 2024-06-13T06:56:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:41.585+0000 7f2c4d200640 1 -- 172.21.15.7:0/1597372947 wait complete. 2024-06-13T06:56:41.688 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T06:56:41.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T06:56:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 40.0 s, 2360.4 tps, lat 42.798 ms stddev 51.880 2024-06-13T06:56:43.137 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:42 smithi189 ceph-mon[36616]: pgmap v529: 129 pgs: 129 active+clean; 959 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 897 op/s 2024-06-13T06:56:43.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:42 smithi007 ceph-mon[32077]: pgmap v529: 129 pgs: 129 active+clean; 959 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 897 op/s 2024-06-13T06:56:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:42 smithi175 ceph-mon[36886]: pgmap v529: 129 pgs: 129 active+clean; 959 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 897 op/s 2024-06-13T06:56:44.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:44 smithi007 ceph-mon[32077]: pgmap v530: 129 pgs: 129 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 17 MiB/s wr, 960 op/s 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 -- 172.21.15.7:0/1656742792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93140769c0 msgr2=0x7f9314076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 --2- 172.21.15.7:0/1656742792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93140769c0 0x7f9314076da0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 -- 172.21.15.7:0/1656742792 shutdown_connections 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 --2- 172.21.15.7:0/1656742792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93141133c0 0x7f93141157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 --2- 172.21.15.7:0/1656742792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93140769c0 0x7f9314076da0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 --2- 172.21.15.7:0/1656742792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140772e0 0x7f9314112e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.021+0000 7f931aa00700 1 -- 172.21.15.7:0/1656742792 >> 172.21.15.7:0/1656742792 conn(0x7f93141001f0 msgr2=0x7f9314102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.022+0000 7f931aa00700 1 -- 172.21.15.7:0/1656742792 shutdown_connections 2024-06-13T06:56:45.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.022+0000 7f931aa00700 1 -- 172.21.15.7:0/1656742792 wait complete. 2024-06-13T06:56:45.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.023+0000 7f931aa00700 1 Processor -- start 2024-06-13T06:56:45.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.023+0000 7f931aa00700 1 -- start start 2024-06-13T06:56:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 0x7f9314073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 0x7f93140740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9314118f20 con 0x7f93140772e0 2024-06-13T06:56:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9314118da0 con 0x7f93141133c0 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f931aa00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93141190a0 con 0x7f93140769c0 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f9313400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 0x7f93140740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.024+0000 7f9318a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.025+0000 7f9318a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36364/0 (socket says 172.21.15.7:36364) 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.025+0000 7f9318a00700 1 -- 172.21.15.7:0/1309593489 learned_addr learned my addr 172.21.15.7:0/1309593489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.025+0000 7f9313400700 1 -- 172.21.15.7:0/1309593489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 msgr2=0x7f9314073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313e00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 0x7f9314073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313400700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 0x7f9314073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313400700 1 -- 172.21.15.7:0/1309593489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 msgr2=0x7f931406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313400700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313400700 1 -- 172.21.15.7:0/1309593489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931406d670 con 0x7f93140772e0 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9318a00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313e00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 0x7f9314073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.026+0000 7f9313400700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 0x7f93140740d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9f0 tx=0x7f930400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.027+0000 7f9310c00700 1 -- 172.21.15.7:0/1309593489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930400cbd0 con 0x7f93140772e0 2024-06-13T06:56:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.027+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931406d960 con 0x7f93140772e0 2024-06-13T06:56:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.027+0000 7f9310c00700 1 -- 172.21.15.7:0/1309593489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930400cd70 con 0x7f93140772e0 2024-06-13T06:56:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.027+0000 7f9310c00700 1 -- 172.21.15.7:0/1309593489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304018840 con 0x7f93140772e0 2024-06-13T06:56:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.027+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93140719a0 con 0x7f93140772e0 2024-06-13T06:56:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.034+0000 7f9310c00700 1 -- 172.21.15.7:0/1309593489 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f93040189e0 con 0x7f93140772e0 2024-06-13T06:56:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.035+0000 7f93253e9b80 1 Processor -- start 2024-06-13T06:56:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.035+0000 7f93253e9b80 1 -- start start 2024-06-13T06:56:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.036+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.036+0000 7f9310c00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 0x7f92f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.037+0000 7f9310c00700 1 -- 172.21.15.7:0/1309593489 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f93040cf4e0 con 0x7f93140772e0 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f9318a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 0x56476b99bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b99c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f93253e9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56476b8aaa90 con 0x56476b99c2c0 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f93253e9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56476b8edb80 con 0x56476b9a2530 2024-06-13T06:56:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f93253e9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56476b8a8970 con 0x56476b9a1c10 2024-06-13T06:56:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f9318a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52070/0 (socket says 172.21.15.7:52070) 2024-06-13T06:56:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.038+0000 7f9318a00700 1 -- 172.21.15.7:0/127456276 learned_addr learned my addr 172.21.15.7:0/127456276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.039+0000 7f9318a00700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b99c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.039+0000 7f9313400700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 0x56476b99bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.040+0000 7f9313e00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 0x7f92f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 -- 172.21.15.7:0/127456276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 msgr2=0x56476b9a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 -- 172.21.15.7:0/127456276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 msgr2=0x56476b99c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b99c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 -- 172.21.15.7:0/127456276 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476b896010 con 0x56476b9a2530 2024-06-13T06:56:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9318a00700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.041+0000 7f9313400700 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 0x56476b99bd80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930401e000 tx=0x7f930409c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382abac919ef9603 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.042+0000 7f9309600700 1 -- 172.21.15.7:0/127456276 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040a4610 con 0x56476b9a2530 2024-06-13T06:56:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.042+0000 7f9309600700 1 -- 172.21.15.7:0/127456276 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93040cf200 con 0x56476b9a2530 2024-06-13T06:56:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.042+0000 7f9309600700 1 -- 172.21.15.7:0/127456276 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930409b050 con 0x56476b9a2530 2024-06-13T06:56:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 -- 172.21.15.7:0/127456276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 msgr2=0x56476b99bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 0x56476b99bd80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930401e000 tx=0x7f930409c120 comp rx=0 tx=0).stop 2024-06-13T06:56:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f9313e00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 0x7f92f80a5230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300004690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 -- 172.21.15.7:0/127456276 shutdown_connections 2024-06-13T06:56:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b99c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a1c10 0x56476b9a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 --2- 172.21.15.7:0/127456276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a2530 0x56476b99bd80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 -- 172.21.15.7:0/127456276 >> 172.21.15.7:0/127456276 conn(0x56476b8c81f0 msgr2=0x56476b997070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 -- 172.21.15.7:0/127456276 shutdown_connections 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.043+0000 7f93253e9b80 1 -- 172.21.15.7:0/127456276 wait complete. 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.044+0000 7f93253e9b80 1 -- start start 2024-06-13T06:56:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.044+0000 7f93253e9b80 1 Processor -- start 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 0x56476b9a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 0x56476b9a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56476b889fe0 con 0x56476b9a1c10 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56476b8e9b40 con 0x56476b99c2c0 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56476b9a8f10 con 0x56476b9a2530 2024-06-13T06:56:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f93253e9b80 10 client.0 _ll_register_callbacks cb 0x56476b883400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f9313400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 0x56476b9a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f9313e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 0x56476b9a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f9318a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f9318a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52074/0 (socket says 172.21.15.7:52074) 2024-06-13T06:56:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.045+0000 7f9318a00700 1 -- 172.21.15.7:0/2059709871 learned_addr learned my addr 172.21.15.7:0/2059709871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9318a00700 1 -- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 msgr2=0x56476b9a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9318a00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 0x56476b9a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9318a00700 1 -- 172.21.15.7:0/2059709871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 msgr2=0x56476b9a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9318a00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 0x56476b9a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9318a00700 1 -- 172.21.15.7:0/2059709871 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56476b83c950 con 0x56476b9a2530 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9313e00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 0x56476b9a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.046+0000 7f9313400700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 0x56476b9a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:45.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.048+0000 7f92fe000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:45.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.048+0000 7f9318a00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f930c009ad0 tx=0x7f930c009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b18951ba96a349b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.050+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c012040 con 0x56476b9a2530 2024-06-13T06:56:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.050+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f930c016ce0 con 0x56476b9a2530 2024-06-13T06:56:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.050+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93040cf200 con 0x56476b9a2530 2024-06-13T06:56:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.052+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f930c027050 con 0x56476b9a2530 2024-06-13T06:56:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 10 client.25405 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56476b9b3770 con 0x56476b9a2530 2024-06-13T06:56:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93040cf200 con 0x56476b9a2530 2024-06-13T06:56:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f930c018070 con 0x56476b9a2530 2024-06-13T06:56:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 10 client.25405 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 20 client.25405 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476b9b5e80 0x56476b9b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 4 client.25405 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.053+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56476b8fb080 con 0x56476b9b5e80 2024-06-13T06:56:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.055+0000 7f9313e00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476b9b5e80 0x56476b9b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.056+0000 7f9313e00700 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476b9b5e80 0x56476b9b82a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb644b56fda25bd server_cookie=bd6947431fe8f37e in_seq=0 out_seq=0 2024-06-13T06:56:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.056+0000 7f92fe000700 10 client.25405 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.057+0000 7f92fe000700 1 -- 172.21.15.7:0/2059709871 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56476b8fb080 con 0x56476b9b5e80 2024-06-13T06:56:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.057+0000 7f92fe000700 10 client.25405 handle_command_reply: tid=0 2024-06-13T06:56:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.057+0000 7f93253e9b80 1 client.25405 shutdown 2024-06-13T06:56:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.058+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 msgr2=0x56476b9a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.058+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f930c009ad0 tx=0x7f930c009d80 comp rx=0 tx=0).stop 2024-06-13T06:56:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 shutdown_connections 2024-06-13T06:56:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476b9b5e80 0x56476b9b82a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a5ae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b99c2c0 0x56476b9a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 --2- 172.21.15.7:0/2059709871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a1c10 0x56476b9a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f92fe000700 0 client.25405 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.059+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 >> 172.21.15.7:0/2059709871 conn(0x56476b8c81f0 msgr2=0x56476b990810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.060+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 shutdown_connections 2024-06-13T06:56:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.060+0000 7f93253e9b80 1 -- 172.21.15.7:0/2059709871 wait complete. 2024-06-13T06:56:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.060+0000 7f93253e9b80 20 client.25405 trim_cache size 0 max 16384 2024-06-13T06:56:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.116+0000 7f93253e9b80 1 Processor -- start 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.116+0000 7f93253e9b80 1 -- start start 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b9911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4011230 con 0x56476b99c2c0 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400ea90 con 0x56476b9a1c10 2024-06-13T06:56:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.117+0000 7f93253e9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400c8e0 con 0x56476b9a2530 2024-06-13T06:56:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36394/0 (socket says 172.21.15.7:36394) 2024-06-13T06:56:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313400700 1 -- 172.21.15.7:0/368536955 learned_addr learned my addr 172.21.15.7:0/368536955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9318a00700 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313e00700 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b9911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313400700 1 -- 172.21.15.7:0/368536955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 msgr2=0x56476b9a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.118+0000 7f9313400700 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.119+0000 7f9313400700 1 -- 172.21.15.7:0/368536955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 msgr2=0x56476b9911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.119+0000 7f9313400700 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b9911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.119+0000 7f9313400700 1 -- 172.21.15.7:0/368536955 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476b9b8900 con 0x56476b9a1c10 2024-06-13T06:56:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.119+0000 7f9313400700 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f930400c5b0 tx=0x7f930400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497a4cab5296fc65 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.120+0000 7f9308c00700 1 -- 172.21.15.7:0/368536955 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040a4610 con 0x56476b9a1c10 2024-06-13T06:56:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.120+0000 7f9308c00700 1 -- 172.21.15.7:0/368536955 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93040cf200 con 0x56476b9a1c10 2024-06-13T06:56:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.120+0000 7f9308c00700 1 -- 172.21.15.7:0/368536955 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040a4610 con 0x56476b9a1c10 2024-06-13T06:56:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.121+0000 7f93253e9b80 1 -- 172.21.15.7:0/368536955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 msgr2=0x56476b997180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.121+0000 7f93253e9b80 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f930400c5b0 tx=0x7f930400ef90 comp rx=0 tx=0).stop 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 -- 172.21.15.7:0/368536955 shutdown_connections 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b9a2530 0x56476b9a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b99c2c0 0x56476b9911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 --2- 172.21.15.7:0/368536955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b997180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 -- 172.21.15.7:0/368536955 >> 172.21.15.7:0/368536955 conn(0x56476b8c81f0 msgr2=0x56476b9a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.122+0000 7f93253e9b80 1 -- 172.21.15.7:0/368536955 shutdown_connections 2024-06-13T06:56:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.123+0000 7f93253e9b80 1 -- 172.21.15.7:0/368536955 wait complete. 2024-06-13T06:56:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.124+0000 7f93253e9b80 1 -- start start 2024-06-13T06:56:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.124+0000 7f93253e9b80 1 Processor -- start 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 0x56476b9bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b9bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a2530 0x56476b9be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400cf50 con 0x56476b9a2530 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400e3c0 con 0x56476b9a1c10 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92f400d620 con 0x56476b99c2c0 2024-06-13T06:56:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f93253e9b80 10 client.0 _ll_register_callbacks cb 0x56476b883400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f9313e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 0x56476b9bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f9313e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 0x56476b9bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52090/0 (socket says 172.21.15.7:52090) 2024-06-13T06:56:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f9313e00700 1 -- 172.21.15.7:0/241250070 learned_addr learned my addr 172.21.15.7:0/241250070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.126+0000 7f9313400700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b9bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9318a00700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a2530 0x56476b9be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 -- 172.21.15.7:0/241250070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 msgr2=0x56476b9bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 0x56476b9bd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 -- 172.21.15.7:0/241250070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a2530 msgr2=0x56476b9be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a2530 0x56476b9be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 -- 172.21.15.7:0/241250070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56476ba1fc30 con 0x56476b9a1c10 2024-06-13T06:56:45.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.127+0000 7f9313400700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b9bddc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x56476b8e97d0 tx=0x7f9304097cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fbe93ff92d83ad server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.128+0000 7f92fe000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.128+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040a4610 con 0x56476b9a1c10 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.128+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f93040ba2f0 con 0x56476b9a1c10 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.128+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93040cf200 con 0x56476b9a1c10 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.129+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f930407f050 con 0x56476b9a1c10 2024-06-13T06:56:45.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.129+0000 7f93253e9b80 10 client.25415 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.129+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56476ba20f50 con 0x56476b9a1c10 2024-06-13T06:56:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.129+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93040cf200 con 0x56476b9a1c10 2024-06-13T06:56:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f930409a860 con 0x56476b9a1c10 2024-06-13T06:56:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 10 client.25415 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 10 client.25415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 20 client.25415 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476ba21180 0x56476ba23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 4 client.25415 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56476b8f5600 con 0x56476ba21180 2024-06-13T06:56:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.130+0000 7f9313e00700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476ba21180 0x56476ba23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:45.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.132+0000 7f92fe000700 10 client.25415 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.134+0000 7f9313e00700 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476ba21180 0x56476ba23570 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d61db4ad18186f1 server_cookie=5d5cb759eea17461 in_seq=0 out_seq=0 2024-06-13T06:56:45.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.159+0000 7f92fe000700 1 -- 172.21.15.7:0/241250070 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56476b8f5600 con 0x56476ba21180 2024-06-13T06:56:45.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.159+0000 7f92fe000700 10 client.25415 handle_command_reply: tid=0 2024-06-13T06:56:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.159+0000 7f93253e9b80 1 client.25415 shutdown 2024-06-13T06:56:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 msgr2=0x56476b9bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b9bddc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x56476b8e97d0 tx=0x7f9304097cc0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 shutdown_connections 2024-06-13T06:56:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56476ba21180 0x56476ba23570 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f92fe000700 0 client.25415 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56476b9a2530 0x56476b9be300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56476b99c2c0 0x56476b9bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 --2- 172.21.15.7:0/241250070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56476b9a1c10 0x56476b9bddc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.160+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 >> 172.21.15.7:0/241250070 conn(0x56476b8c81f0 msgr2=0x56476b9a30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.161+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 shutdown_connections 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.161+0000 7f93253e9b80 1 -- 172.21.15.7:0/241250070 wait complete. 2024-06-13T06:56:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.161+0000 7f93253e9b80 20 client.25415 trim_cache size 0 max 16384 2024-06-13T06:56:45.162 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:56:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T06:56:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5764d9dd-acc6-41d7-9880-04d938918ee6", 2024-06-13T06:56:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T06:56:45.163 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:56:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 msgr2=0x7f92f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 0x7f92f80a5230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f930000b2e0 tx=0x7f9300004690 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 msgr2=0x7f93140740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 0x7f93140740d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930400e9f0 tx=0x7f930400edb0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 shutdown_connections 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f92f80a2d70 0x7f92f80a5230 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93141133c0 0x7f931406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93140769c0 0x7f9314073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 --2- 172.21.15.7:0/1309593489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93140772e0 0x7f93140740d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.163+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 >> 172.21.15.7:0/1309593489 conn(0x7f93141001f0 msgr2=0x7f93141025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.164+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 shutdown_connections 2024-06-13T06:56:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:45.164+0000 7f931aa00700 1 -- 172.21.15.7:0/1309593489 wait complete. 2024-06-13T06:56:45.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:44 smithi189 ceph-mon[36616]: pgmap v530: 129 pgs: 129 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 17 MiB/s wr, 960 op/s 2024-06-13T06:56:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:44 smithi175 ceph-mon[36886]: pgmap v530: 129 pgs: 129 active+clean; 960 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 17 MiB/s wr, 960 op/s 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:45 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:45 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:45 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:47.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:56:47.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:47.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:46 smithi007 ceph-mon[32077]: pgmap v531: 129 pgs: 129 active+clean; 961 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 833 op/s 2024-06-13T06:56:47.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:56:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:56:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:46 smithi189 ceph-mon[36616]: pgmap v531: 129 pgs: 129 active+clean; 961 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 833 op/s 2024-06-13T06:56:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:56:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:56:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:46 smithi175 ceph-mon[36886]: pgmap v531: 129 pgs: 129 active+clean; 961 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 833 op/s 2024-06-13T06:56:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 45.0 s, 2812.5 tps, lat 35.509 ms stddev 33.660 2024-06-13T06:56:47.918 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:56:49.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:48 smithi007 ceph-mon[32077]: pgmap v532: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T06:56:49.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:48 smithi189 ceph-mon[36616]: pgmap v532: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T06:56:49.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:48 smithi175 ceph-mon[36886]: pgmap v532: 129 pgs: 129 active+clean; 961 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 -- 172.21.15.7:0/3446328676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75280746c0 msgr2=0x7f7528074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 --2- 172.21.15.7:0/3446328676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75280746c0 0x7f7528074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7518009990 tx=0x7f7518009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 -- 172.21.15.7:0/3446328676 shutdown_connections 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 --2- 172.21.15.7:0/3446328676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752810eef0 0x7f75281112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 --2- 172.21.15.7:0/3446328676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75280746c0 0x7f7528074aa0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7518009990 tx=0x7f7518009ca0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 --2- 172.21.15.7:0/3446328676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7528074fe0 0x7f752810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.660+0000 7f752ea00700 1 -- 172.21.15.7:0/3446328676 >> 172.21.15.7:0/3446328676 conn(0x7f75280795f0 msgr2=0x7f7528079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.661+0000 7f752ea00700 1 -- 172.21.15.7:0/3446328676 shutdown_connections 2024-06-13T06:56:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.661+0000 7f752ea00700 1 -- 172.21.15.7:0/3446328676 wait complete. 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.663+0000 7f752ea00700 1 Processor -- start 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.663+0000 7f752ea00700 1 -- start start 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.663+0000 7f752ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 0x7f7528073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.663+0000 7f752ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 0x7f752806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.663+0000 7f752ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752806d3b0 0x7f752806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.664+0000 7f752ea00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114a10 con 0x7f7528074fe0 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.664+0000 7f752ea00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114890 con 0x7f752810eef0 2024-06-13T06:56:50.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.664+0000 7f752ea00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114b90 con 0x7f752806d3b0 2024-06-13T06:56:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.664+0000 7f7527e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 0x7f7528073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.665+0000 7f7527400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 0x7f752806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.665+0000 7f7527400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 0x7f752806ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36424/0 (socket says 172.21.15.7:36424) 2024-06-13T06:56:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.665+0000 7f752ca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752806d3b0 0x7f752806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.665+0000 7f7527400700 1 -- 172.21.15.7:0/393795823 learned_addr learned my addr 172.21.15.7:0/393795823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.666+0000 7f7527e00700 1 -- 172.21.15.7:0/393795823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752806d3b0 msgr2=0x7f752806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.666+0000 7f7527e00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752806d3b0 0x7f752806d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.666+0000 7f7527e00700 1 -- 172.21.15.7:0/393795823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 msgr2=0x7f752806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.666+0000 7f7527e00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 0x7f752806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.666+0000 7f7527e00700 1 -- 172.21.15.7:0/393795823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7528071870 con 0x7f7528074fe0 2024-06-13T06:56:50.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.667+0000 7f7527e00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 0x7f7528073d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7518008000 tx=0x7f751800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.668+0000 7f7524c00700 1 -- 172.21.15.7:0/393795823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518003890 con 0x7f7528074fe0 2024-06-13T06:56:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.668+0000 7f7524c00700 1 -- 172.21.15.7:0/393795823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7518004830 con 0x7f7528074fe0 2024-06-13T06:56:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.668+0000 7f7524c00700 1 -- 172.21.15.7:0/393795823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518005350 con 0x7f7528074fe0 2024-06-13T06:56:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.669+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7528071b00 con 0x7f7528074fe0 2024-06-13T06:56:50.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.670+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7528071f90 con 0x7f7528074fe0 2024-06-13T06:56:50.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.675+0000 7f7524c00700 1 -- 172.21.15.7:0/393795823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7518009d10 con 0x7f7528074fe0 2024-06-13T06:56:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.677+0000 7f7524c00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 0x7f750c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.677+0000 7f7524c00700 1 -- 172.21.15.7:0/393795823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f75180cf9a0 con 0x7f7528074fe0 2024-06-13T06:56:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.678+0000 7f7527400700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 0x7f750c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.680+0000 7f7527400700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 0x7f750c0a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7514009680 tx=0x7f7514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.684+0000 7f7539291b80 1 Processor -- start 2024-06-13T06:56:50.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.685+0000 7f7539291b80 1 -- start start 2024-06-13T06:56:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.685+0000 7f7539291b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.685+0000 7f7539291b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x55594373da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7527e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7527e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52120/0 (socket says 172.21.15.7:52120) 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7527e00700 1 -- 172.21.15.7:0/170723269 learned_addr learned my addr 172.21.15.7:0/170723269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7539291b80 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555943748300 0x55594374a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55594364ca10 con 0x555943748300 2024-06-13T06:56:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55594368fb00 con 0x55594373d5b0 2024-06-13T06:56:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55594364a8f0 con 0x5559436b9e40 2024-06-13T06:56:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.686+0000 7f7527400700 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x55594373da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527400700 1 -- 172.21.15.7:0/170723269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 msgr2=0x55594373d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527400700 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527400700 1 -- 172.21.15.7:0/170723269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555943748300 msgr2=0x55594374a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527400700 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555943748300 0x55594374a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527400700 1 -- 172.21.15.7:0/170723269 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555943637f90 con 0x55594373d5b0 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.687+0000 7f7527e00700 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.688+0000 7f7527400700 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x55594373da30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f752806e590 tx=0x7f7514014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c12736c2e9e392 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.688+0000 7f751d600700 1 -- 172.21.15.7:0/170723269 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514028480 con 0x55594373d5b0 2024-06-13T06:56:50.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.689+0000 7f751d600700 1 -- 172.21.15.7:0/170723269 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751401d8f0 con 0x55594373d5b0 2024-06-13T06:56:50.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.689+0000 7f751d600700 1 -- 172.21.15.7:0/170723269 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751401a810 con 0x55594373d5b0 2024-06-13T06:56:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.690+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 msgr2=0x55594373da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.690+0000 7f7539291b80 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x55594373da30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f752806e590 tx=0x7f7514014000 comp rx=0 tx=0).stop 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.691+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 shutdown_connections 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.691+0000 7f7539291b80 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555943748300 0x55594374a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.691+0000 7f7539291b80 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x55594373d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.691+0000 7f7539291b80 1 --2- 172.21.15.7:0/170723269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x55594373da30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.691+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 >> 172.21.15.7:0/170723269 conn(0x55594366a170 msgr2=0x5559437394b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.694+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 shutdown_connections 2024-06-13T06:56:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.694+0000 7f7539291b80 1 -- 172.21.15.7:0/170723269 wait complete. 2024-06-13T06:56:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.695+0000 7f7539291b80 1 -- start start 2024-06-13T06:56:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.695+0000 7f7539291b80 1 Processor -- start 2024-06-13T06:56:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55594373d5b0 0x555943744eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x5559437453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55594362bf60 con 0x5559436b9e40 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55594368bac0 con 0x555943748300 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55594374afc0 con 0x55594373d5b0 2024-06-13T06:56:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.696+0000 7f7539291b80 10 client.0 _ll_register_callbacks cb 0x555943629720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45204/0 (socket says 172.21.15.7:45204) 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 -- 172.21.15.7:0/885017447 learned_addr learned my addr 172.21.15.7:0/885017447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527400700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55594373d5b0 0x555943744eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f752ca00700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x5559437453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 -- 172.21.15.7:0/885017447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55594373d5b0 msgr2=0x555943744eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55594373d5b0 0x555943744eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 -- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 msgr2=0x5559437453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x5559437453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f7527e00700 1 -- 172.21.15.7:0/885017447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559435de8d0 con 0x5559436b9e40 2024-06-13T06:56:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.697+0000 7f752ca00700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x5559437453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:56:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.698+0000 7f7527e00700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7518001eb0 tx=0x7f75180057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173c2f2f838b4bb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.699+0000 7f7512000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:56:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.700+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751809bf50 con 0x5559436b9e40 2024-06-13T06:56:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.701+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7518004ce0 con 0x5559436b9e40 2024-06-13T06:56:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.701+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f751401d8f0 con 0x5559436b9e40 2024-06-13T06:56:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.702+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75180a0300 con 0x5559436b9e40 2024-06-13T06:56:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.703+0000 7f7539291b80 10 client.15243 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.703+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55594375e060 con 0x5559436b9e40 2024-06-13T06:56:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.703+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f751401d8f0 con 0x5559436b9e40 2024-06-13T06:56:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.703+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f751809f070 con 0x5559436b9e40 2024-06-13T06:56:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 10 client.15243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 20 client.15243 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555943760580 0x555943762970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 4 client.15243 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.704+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55594369d000 con 0x555943760580 2024-06-13T06:56:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.705+0000 7f7527400700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555943760580 0x555943762970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.708+0000 7f7512000700 10 client.15243 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.708+0000 7f7527400700 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555943760580 0x555943762970 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c4ede1d0ab3790 server_cookie=6a816575805429f5 in_seq=0 out_seq=0 2024-06-13T06:56:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.711+0000 7f7512000700 1 -- 172.21.15.7:0/885017447 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55594369d000 con 0x555943760580 2024-06-13T06:56:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.711+0000 7f7512000700 10 client.15243 handle_command_reply: tid=0 2024-06-13T06:56:50.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.711+0000 7f7539291b80 1 client.15243 shutdown 2024-06-13T06:56:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 msgr2=0x555943747bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7518001eb0 tx=0x7f75180057d0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 shutdown_connections 2024-06-13T06:56:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555943760580 0x555943762970 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x5559437453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7512000700 0 client.15243 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x555943747bb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 --2- 172.21.15.7:0/885017447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55594373d5b0 0x555943744eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.713+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 >> 172.21.15.7:0/885017447 conn(0x55594366a170 msgr2=0x555943738740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.714+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 shutdown_connections 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.714+0000 7f7539291b80 1 -- 172.21.15.7:0/885017447 wait complete. 2024-06-13T06:56:50.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.714+0000 7f7539291b80 20 client.15243 trim_cache size 0 max 16384 2024-06-13T06:56:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.751+0000 7f7539291b80 1 Processor -- start 2024-06-13T06:56:50.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.752+0000 7f7539291b80 1 -- start start 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x5559437385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x5559437476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 0x5559437496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a6520 con 0x5559436b9e40 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a3d80 con 0x55594373d5b0 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7539291b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a1bf0 con 0x555943748300 2024-06-13T06:56:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7527400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x5559437476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7527400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x5559437476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36456/0 (socket says 172.21.15.7:36456) 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f7527400700 1 -- 172.21.15.7:0/3029313741 learned_addr learned my addr 172.21.15.7:0/3029313741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.753+0000 7f752ca00700 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 0x5559437496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.754+0000 7f752ca00700 1 -- 172.21.15.7:0/3029313741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 msgr2=0x5559437476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.754+0000 7f752ca00700 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x5559437476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.754+0000 7f752ca00700 1 -- 172.21.15.7:0/3029313741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 msgr2=0x5559437385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.754+0000 7f7527e00700 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x5559437385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.755+0000 7f752ca00700 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x5559437385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.755+0000 7f752ca00700 1 -- 172.21.15.7:0/3029313741 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555943739990 con 0x555943748300 2024-06-13T06:56:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.756+0000 7f752ca00700 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 0x5559437496f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5559437482d0 tx=0x7f75200104c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadab6508e888da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.756+0000 7f751cc00700 1 -- 172.21.15.7:0/3029313741 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520019030 con 0x555943748300 2024-06-13T06:56:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.756+0000 7f751cc00700 1 -- 172.21.15.7:0/3029313741 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7520010a20 con 0x555943748300 2024-06-13T06:56:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.756+0000 7f751cc00700 1 -- 172.21.15.7:0/3029313741 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520004600 con 0x555943748300 2024-06-13T06:56:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.757+0000 7f7539291b80 1 -- 172.21.15.7:0/3029313741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 msgr2=0x5559437496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.757+0000 7f7539291b80 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 0x5559437496f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5559437482d0 tx=0x7f75200104c0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 -- 172.21.15.7:0/3029313741 shutdown_connections 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555943748300 0x5559437496f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559436b9e40 0x5559437385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 --2- 172.21.15.7:0/3029313741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55594373d5b0 0x5559437476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 -- 172.21.15.7:0/3029313741 >> 172.21.15.7:0/3029313741 conn(0x55594366a170 msgr2=0x555943737eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.758+0000 7f7539291b80 1 -- 172.21.15.7:0/3029313741 shutdown_connections 2024-06-13T06:56:50.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.759+0000 7f7539291b80 1 -- 172.21.15.7:0/3029313741 wait complete. 2024-06-13T06:56:50.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.759+0000 7f7539291b80 1 -- start start 2024-06-13T06:56:50.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.760+0000 7f7539291b80 1 Processor -- start 2024-06-13T06:56:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x555943758af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55594373d5b0 0x555943759030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x555943753aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a2240 con 0x55594373d5b0 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a36b0 con 0x555943748300 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75281a2910 con 0x5559436b9e40 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.763+0000 7f7539291b80 10 client.0 _ll_register_callbacks cb 0x555943629720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.764+0000 7f752ca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x555943753aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.764+0000 7f752ca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x555943753aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36470/0 (socket says 172.21.15.7:36470) 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.764+0000 7f752ca00700 1 -- 172.21.15.7:0/401265684 learned_addr learned my addr 172.21.15.7:0/401265684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.765+0000 7f7527e00700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x555943758af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.765+0000 7f7527400700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55594373d5b0 0x555943759030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.767+0000 7f7527e00700 1 -- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 msgr2=0x555943753aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.767+0000 7f7527e00700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x555943753aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.767+0000 7f7527e00700 1 -- 172.21.15.7:0/401265684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55594373d5b0 msgr2=0x555943759030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:50.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.767+0000 7f7527e00700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55594373d5b0 0x555943759030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.767+0000 7f7527e00700 1 -- 172.21.15.7:0/401265684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555943759670 con 0x5559436b9e40 2024-06-13T06:56:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.771+0000 7f7527e00700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x555943758af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75180988d0 tx=0x7f75180cd1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111584d0732f68f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.771+0000 7f7512000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.772+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75180b0060 con 0x5559436b9e40 2024-06-13T06:56:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.772+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f751809b3a0 con 0x5559436b9e40 2024-06-13T06:56:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.772+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7520010a20 con 0x5559436b9e40 2024-06-13T06:56:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.773+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75180981f0 con 0x5559436b9e40 2024-06-13T06:56:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.773+0000 7f7539291b80 10 client.25429 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.773+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555943759670 con 0x5559436b9e40 2024-06-13T06:56:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7520010a20 con 0x5559436b9e40 2024-06-13T06:56:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f751809f070 con 0x5559436b9e40 2024-06-13T06:56:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 10 client.25429 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 20 client.25429 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55594374bb20 0x55594374df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 4 client.25429 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555943697580 con 0x55594374bb20 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.774+0000 7f7527400700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55594374bb20 0x55594374df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.775+0000 7f7527400700 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55594374bb20 0x55594374df40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73883ca49c6b73cd server_cookie=d9f46b0b937143e1 in_seq=0 out_seq=0 2024-06-13T06:56:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.775+0000 7f7512000700 10 client.25429 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.776+0000 7f7512000700 1 -- 172.21.15.7:0/401265684 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555943697580 con 0x55594374bb20 2024-06-13T06:56:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.776+0000 7f7512000700 10 client.25429 handle_command_reply: tid=0 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.776+0000 7f7539291b80 1 client.25429 shutdown 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.776+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 msgr2=0x555943758af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.776+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x555943758af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f75180988d0 tx=0x7f75180cd1d0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 shutdown_connections 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55594374bb20 0x55594374df40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555943748300 0x555943753aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559436b9e40 0x555943758af0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 --2- 172.21.15.7:0/401265684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55594373d5b0 0x555943759030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 >> 172.21.15.7:0/401265684 conn(0x55594366a170 msgr2=0x555943738740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7512000700 0 client.25429 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 shutdown_connections 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 1 -- 172.21.15.7:0/401265684 wait complete. 2024-06-13T06:56:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.777+0000 7f7539291b80 20 client.25429 trim_cache size 0 max 16384 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "5764d9dd-acc6-41d7-9880-04d938918ee6": { 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5764d9dd-acc6-41d7-9880-04d938918ee6", 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:56:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:56:50.780 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:56:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 msgr2=0x7f750c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 0x7f750c0a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7514009680 tx=0x7f7514009260 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 msgr2=0x7f7528073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 0x7f7528073d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7518008000 tx=0x7f751800bab0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 shutdown_connections 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752806d3b0 0x7f752806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7528074fe0 0x7f7528073d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f750c0a2d70 0x7f750c0a5230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 --2- 172.21.15.7:0/393795823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f752810eef0 0x7f752806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 >> 172.21.15.7:0/393795823 conn(0x7f75280795f0 msgr2=0x7f75280ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 shutdown_connections 2024-06-13T06:56:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:56:50.781+0000 7f752ea00700 1 -- 172.21.15.7:0/393795823 wait complete. 2024-06-13T06:56:51.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:50 smithi007 ceph-mon[32077]: pgmap v533: 129 pgs: 129 active+clean; 962 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.5 MiB/s wr, 755 op/s 2024-06-13T06:56:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:50 smithi189 ceph-mon[36616]: pgmap v533: 129 pgs: 129 active+clean; 962 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.5 MiB/s wr, 755 op/s 2024-06-13T06:56:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:50 smithi175 ceph-mon[36886]: pgmap v533: 129 pgs: 129 active+clean; 962 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.5 MiB/s wr, 755 op/s 2024-06-13T06:56:51.943 INFO:tasks.cephfs.filesystem:scrub status for tag:5764d9dd-acc6-41d7-9880-04d938918ee6 - {'path': '/', 'tag': '5764d9dd-acc6-41d7-9880-04d938918ee6', 'options': 'recursive,force'} 2024-06-13T06:56:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 50.0 s, 2214.8 tps, lat 45.242 ms stddev 50.668 2024-06-13T06:56:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:52 smithi189 ceph-mon[36616]: pgmap v534: 129 pgs: 129 active+clean; 966 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 6.9 MiB/s wr, 786 op/s 2024-06-13T06:56:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:52 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:56:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:52 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:56:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:52 smithi175 ceph-mon[36886]: pgmap v534: 129 pgs: 129 active+clean; 966 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 6.9 MiB/s wr, 786 op/s 2024-06-13T06:56:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:52 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:56:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:52 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:56:53.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:52 smithi007 ceph-mon[32077]: pgmap v534: 129 pgs: 129 active+clean; 966 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 6.9 MiB/s wr, 786 op/s 2024-06-13T06:56:53.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:52 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:56:53.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:52 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:56:54.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:53 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:56:54.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:53 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:56:54.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:53 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:56:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 -- 172.21.15.7:0/201697827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10a200 msgr2=0x7f34cc10a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 --2- 172.21.15.7:0/201697827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10a200 0x7f34cc10a620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009900 tx=0x7f34bc02a410 comp rx=0 tx=0).stop 2024-06-13T06:56:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 -- 172.21.15.7:0/201697827 shutdown_connections 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 --2- 172.21.15.7:0/201697827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10bdb0 0x7f34cc112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 --2- 172.21.15.7:0/201697827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10b3f0 0x7f34cc10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 --2- 172.21.15.7:0/201697827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10a200 0x7f34cc10a620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.777+0000 7f34d2c00640 1 -- 172.21.15.7:0/201697827 >> 172.21.15.7:0/201697827 conn(0x7f34cc0709c0 msgr2=0x7f34cc072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.778+0000 7f34d2c00640 1 -- 172.21.15.7:0/201697827 shutdown_connections 2024-06-13T06:56:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.778+0000 7f34d2c00640 1 -- 172.21.15.7:0/201697827 wait complete. 2024-06-13T06:56:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.779+0000 7f34d2c00640 1 Processor -- start 2024-06-13T06:56:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.779+0000 7f34d2c00640 1 -- start start 2024-06-13T06:56:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.779+0000 7f34d2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 0x7f34cc10ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.779+0000 7f34d2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.779+0000 7f34d2c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10bdb0 0x7f34cc10caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34d2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc115ee0 con 0x7f34cc10a200 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34d2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc115d60 con 0x7f34cc10b3f0 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34d2c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc116060 con 0x7f34cc10bdb0 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 0x7f34cc10ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34d0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10bdb0 0x7f34cc10caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58658/0 (socket says 172.21.15.7:58658) 2024-06-13T06:56:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 -- 172.21.15.7:0/3898683991 learned_addr learned my addr 172.21.15.7:0/3898683991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 0x7f34cc10ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48614/0 (socket says 172.21.15.7:48614) 2024-06-13T06:56:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 -- 172.21.15.7:0/3898683991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10bdb0 msgr2=0x7f34cc10caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10bdb0 0x7f34cc10caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 -- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 msgr2=0x7f34cc10ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 0x7f34cc10ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.780+0000 7f34cb400640 1 -- 172.21.15.7:0/3898683991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc10d3b0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34cb400640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34b800ea60 tx=0x7f34b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34c8c00640 1 -- 172.21.15.7:0/3898683991 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b800cde0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34c8c00640 1 -- 172.21.15.7:0/3898683991 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b80035f0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34c8c00640 1 -- 172.21.15.7:0/3898683991 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8010670 con 0x7f34cc10b3f0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1aeaa0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.781+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1aeeb0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.786+0000 7f34c8c00640 1 -- 172.21.15.7:0/3898683991 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f34b80037a0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.788+0000 7f34c8c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 0x7f34b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.788+0000 7f34c8c00640 1 -- 172.21.15.7:0/3898683991 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f34b80cf5d0 con 0x7f34cc10b3f0 2024-06-13T06:56:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.788+0000 7f34d5b71740 1 Processor -- start 2024-06-13T06:56:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.789+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 0x7f34b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 -- start start 2024-06-13T06:56:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 0x56115e25c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 0x56115e2610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e263bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56115e1bf420 con 0x56115e261760 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56115e166c50 con 0x56115e25c5f0 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.794+0000 7f34d5b71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56115e1bb3e0 con 0x56115e25bc00 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 0x56115e25c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 0x56115e2610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 0x56115e25c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55990/0 (socket says 172.21.15.7:55990) 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cbe00640 1 -- 172.21.15.7:0/489835006 learned_addr learned my addr 172.21.15.7:0/489835006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 -- 172.21.15.7:0/489835006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 msgr2=0x56115e25c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 0x56115e25c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 -- 172.21.15.7:0/489835006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 msgr2=0x56115e263bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e263bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.795+0000 7f34cb400640 1 -- 172.21.15.7:0/489835006 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115ded3210 con 0x56115e25c5f0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.796+0000 7f34cb400640 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 0x56115e2610f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34b80ccde0 tx=0x7f34b809b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2a6531620a435c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.796+0000 7f34c1200640 1 -- 172.21.15.7:0/489835006 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b809baf0 con 0x56115e25c5f0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.796+0000 7f34c1200640 1 -- 172.21.15.7:0/489835006 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b809fea0 con 0x56115e25c5f0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.796+0000 7f34c1200640 1 -- 172.21.15.7:0/489835006 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8003c50 con 0x56115e25c5f0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.797+0000 7f34d5b71740 1 -- 172.21.15.7:0/489835006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 msgr2=0x56115e2610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.797+0000 7f34d5b71740 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 0x56115e2610f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34b80ccde0 tx=0x7f34b809b630 comp rx=0 tx=0).stop 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.797+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 0x7f34b00a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031440 tx=0x7f34bc0040c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.798+0000 7f34d5b71740 1 -- 172.21.15.7:0/489835006 shutdown_connections 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.798+0000 7f34d5b71740 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e263bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.798+0000 7f34d5b71740 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25c5f0 0x56115e2610f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.798+0000 7f34d5b71740 1 --2- 172.21.15.7:0/489835006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25bc00 0x56115e25c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.798+0000 7f34d5b71740 1 -- 172.21.15.7:0/489835006 >> 172.21.15.7:0/489835006 conn(0x56115e223fb0 msgr2=0x56115e257680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.799+0000 7f34d5b71740 1 -- 172.21.15.7:0/489835006 shutdown_connections 2024-06-13T06:56:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.800+0000 7f34d5b71740 1 -- 172.21.15.7:0/489835006 wait complete. 2024-06-13T06:56:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.800+0000 7f34d5b71740 1 -- start start 2024-06-13T06:56:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.801+0000 7f34d5b71740 1 Processor -- start 2024-06-13T06:56:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e265f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e26d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56115e264710 con 0x56115e261760 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56115e17a0b0 con 0x56115e25bc00 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.802+0000 7f34d5b71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56115e264890 con 0x56115e25c5f0 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34d5b71740 10 client.0 _ll_register_callbacks cb 0x56115e165f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56000/0 (socket says 172.21.15.7:56000) 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 -- 172.21.15.7:0/3812708435 learned_addr learned my addr 172.21.15.7:0/3812708435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e265f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 -- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 msgr2=0x56115e265f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e265f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.803+0000 7f34cb400640 1 -- 172.21.15.7:0/3812708435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 msgr2=0x56115e26d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.804+0000 7f34d0c00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e26d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.804+0000 7f34cb400640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e26d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.804+0000 7f34cb400640 1 -- 172.21.15.7:0/3812708435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115df9f620 con 0x56115e25c5f0 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.804+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e265f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.804+0000 7f34d0c00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e26d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:56:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.805+0000 7f34cb400640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cf8c0 tx=0x7f34b809c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b698a21ee50b91 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.806+0000 7f34b5600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:56:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.806+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b807e020 con 0x56115e25c5f0 2024-06-13T06:56:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.806+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f34b8098160 con 0x56115e25c5f0 2024-06-13T06:56:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.807+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b809fea0 con 0x56115e25c5f0 2024-06-13T06:56:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.807+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34b80a4760 con 0x56115e25c5f0 2024-06-13T06:56:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.808+0000 7f34d5b71740 10 client.25441 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.808+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56115e264710 con 0x56115e25c5f0 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.808+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b809fea0 con 0x56115e25c5f0 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.808+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f34b8097840 con 0x56115e25c5f0 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.808+0000 7f34d5b71740 10 client.25441 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34d5b71740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34d5b71740 20 client.25441 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e271b50 0x56115e273f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34d5b71740 4 client.25441 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:56:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115e1ccb50 con 0x56115e271b50 2024-06-13T06:56:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.809+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e271b50 0x56115e273f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.811+0000 7f34cbe00640 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e271b50 0x56115e273f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7360cc20e271a322 server_cookie=3aee2acae8beff6d in_seq=0 out_seq=0 2024-06-13T06:56:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.811+0000 7f34b5600640 10 client.25441 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.812+0000 7f34b5600640 1 -- 172.21.15.7:0/3812708435 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56115e1ccb50 con 0x56115e271b50 2024-06-13T06:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.812+0000 7f34b5600640 10 client.25441 handle_command_reply: tid=0 2024-06-13T06:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.812+0000 7f34d5b71740 1 client.25441 shutdown 2024-06-13T06:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 msgr2=0x56115e2664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cf8c0 tx=0x7f34b809c340 comp rx=0 tx=0).stop 2024-06-13T06:56:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 shutdown_connections 2024-06-13T06:56:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e271b50 0x56115e273f70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e261760 0x56115e26d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2664d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 --2- 172.21.15.7:0/3812708435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e265f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 >> 172.21.15.7:0/3812708435 conn(0x56115e223fb0 msgr2=0x56115e256840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34b5600640 0 client.25441 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.813+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 shutdown_connections 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.814+0000 7f34d5b71740 1 -- 172.21.15.7:0/3812708435 wait complete. 2024-06-13T06:56:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.814+0000 7f34d5b71740 20 client.25441 trim_cache size 0 max 16384 2024-06-13T06:56:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 Processor -- start 2024-06-13T06:56:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 -- start start 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e223fb0 0x56115e25ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e25bc00 0x56115e26d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc080680 con 0x56115e25bc00 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc07df20 con 0x56115e223fb0 2024-06-13T06:56:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.854+0000 7f34d5b71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc07bd90 con 0x56115e25c5f0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.855+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e223fb0 0x56115e25ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.855+0000 7f34d0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.855+0000 7f34d0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56016/0 (socket says 172.21.15.7:56016) 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.855+0000 7f34d0c00640 1 -- 172.21.15.7:0/2426261540 learned_addr learned my addr 172.21.15.7:0/2426261540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.856+0000 7f34d0c00640 1 -- 172.21.15.7:0/2426261540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e223fb0 msgr2=0x56115e25ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.856+0000 7f34d0c00640 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e223fb0 0x56115e25ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.856+0000 7f34d0c00640 1 -- 172.21.15.7:0/2426261540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e25bc00 msgr2=0x56115e26d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.856+0000 7f34d0c00640 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e25bc00 0x56115e26d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.856+0000 7f34d0c00640 1 -- 172.21.15.7:0/2426261540 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115e25bfe0 con 0x56115e25c5f0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.857+0000 7f34d0c00640 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009f70 tx=0x7f34c40138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf3f5aa753f1863 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.859+0000 7f34b7e00640 1 -- 172.21.15.7:0/2426261540 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4003c30 con 0x56115e25c5f0 2024-06-13T06:56:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.859+0000 7f34b7e00640 1 -- 172.21.15.7:0/2426261540 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4003dd0 con 0x56115e25c5f0 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.859+0000 7f34b7e00640 1 -- 172.21.15.7:0/2426261540 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4011b80 con 0x56115e25c5f0 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 -- 172.21.15.7:0/2426261540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 msgr2=0x56115e2576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009f70 tx=0x7f34c40138f0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 -- 172.21.15.7:0/2426261540 shutdown_connections 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e2576d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e25bc00 0x56115e26d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 --2- 172.21.15.7:0/2426261540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e223fb0 0x56115e25ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.860+0000 7f34d5b71740 1 -- 172.21.15.7:0/2426261540 >> 172.21.15.7:0/2426261540 conn(0x56115e265b50 msgr2=0x56115e255fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.861+0000 7f34d5b71740 1 -- 172.21.15.7:0/2426261540 shutdown_connections 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.861+0000 7f34d5b71740 1 -- 172.21.15.7:0/2426261540 wait complete. 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.861+0000 7f34d5b71740 1 -- start start 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 Processor -- start 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e223fb0 0x56115e26afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e26c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc07c3e0 con 0x56115e223fb0 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc07d850 con 0x56115e25bc00 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc07cab0 con 0x56115e25c5f0 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.862+0000 7f34d5b71740 10 client.0 _ll_register_callbacks cb 0x7f34b8097e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58694/0 (socket says 172.21.15.7:58694) 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 -- 172.21.15.7:0/96112128 learned_addr learned my addr 172.21.15.7:0/96112128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cbe00640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e223fb0 0x56115e26afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 -- 172.21.15.7:0/96112128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 msgr2=0x56115e26c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e26c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 -- 172.21.15.7:0/96112128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e223fb0 msgr2=0x56115e26afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e223fb0 0x56115e26afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.863+0000 7f34cb400640 1 -- 172.21.15.7:0/96112128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115e26cb00 con 0x56115e25bc00 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.864+0000 7f34cb400640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f34b8099da0 tx=0x7f34b809ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f2838dff6bd6f23 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.864+0000 7f34b5600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.865+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b807e070 con 0x56115e25bc00 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.865+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f34b80a6d30 con 0x56115e25bc00 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.865+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c4003dd0 con 0x56115e25bc00 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.870+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34b809b450 con 0x56115e25bc00 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.870+0000 7f34d5b71740 10 client.25451 fetch_fsmap learned FSMap version 29 2024-06-13T06:56:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.870+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56115e2dcc10 con 0x56115e25bc00 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.870+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4003dd0 con 0x56115e25bc00 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.873+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f34b80ba2e0 con 0x56115e25bc00 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.873+0000 7f34d5b71740 10 client.25451 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.873+0000 7f34d5b71740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.873+0000 7f34d5b71740 20 client.25451 populate_metadata read hostname 'smithi007' 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.874+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e261760 0x56115e2df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.874+0000 7f34d5b71740 4 client.25451 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:56:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.874+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56115e1c6ea0 con 0x56115e261760 2024-06-13T06:56:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.875+0000 7f34cbe00640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e261760 0x56115e2df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:56:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.877+0000 7f34cbe00640 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e261760 0x56115e2df430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84cc80b10923e6a server_cookie=edfaf9f63878e95a in_seq=0 out_seq=0 2024-06-13T06:56:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.877+0000 7f34b5600640 10 client.25451 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34b5600640 1 -- 172.21.15.7:0/96112128 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56115e1c6ea0 con 0x56115e261760 2024-06-13T06:56:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34b5600640 10 client.25451 handle_command_reply: tid=0 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 client.25451 shutdown 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 msgr2=0x56115e26b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f34b8099da0 tx=0x7f34b809ac30 comp rx=0 tx=0).stop 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 shutdown_connections 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56115e261760 0x56115e2df430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56115e25c5f0 0x56115e26c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56115e25bc00 0x56115e26b500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 --2- 172.21.15.7:0/96112128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56115e223fb0 0x56115e26afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 >> 172.21.15.7:0/96112128 conn(0x56115e265b50 msgr2=0x56115e25ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.926+0000 7f34b5600640 0 client.25451 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:56:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.927+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 shutdown_connections 2024-06-13T06:56:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.927+0000 7f34d5b71740 1 -- 172.21.15.7:0/96112128 wait complete. 2024-06-13T06:56:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.927+0000 7f34d5b71740 20 client.25451 trim_cache size 0 max 16384 2024-06-13T06:56:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:56:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:56:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:56:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 msgr2=0x7f34b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 0x7f34b00a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f34bc031440 tx=0x7f34bc0040c0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 msgr2=0x7f34cc1104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34b800ea60 tx=0x7f34b800ef20 comp rx=0 tx=0).stop 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 shutdown_connections 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f34b00a2cb0 0x7f34b00a5170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34cc10bdb0 0x7f34cc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34cc10b3f0 0x7f34cc1104c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 --2- 172.21.15.7:0/3898683991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34cc10a200 0x7f34cc10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 >> 172.21.15.7:0/3898683991 conn(0x7f34cc0709c0 msgr2=0x7f34cc06b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:56:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 shutdown_connections 2024-06-13T06:56:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:56:54.928+0000 7f34d2c00640 1 -- 172.21.15.7:0/3898683991 wait complete. 2024-06-13T06:56:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:54 smithi189 ceph-mon[36616]: pgmap v535: 129 pgs: 129 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.6 MiB/s wr, 854 op/s 2024-06-13T06:56:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:54 smithi175 ceph-mon[36886]: pgmap v535: 129 pgs: 129 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.6 MiB/s wr, 854 op/s 2024-06-13T06:56:55.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:54 smithi007 ceph-mon[32077]: pgmap v535: 129 pgs: 129 active+clean; 979 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.6 MiB/s wr, 854 op/s 2024-06-13T06:56:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:56 smithi189 ceph-mon[36616]: pgmap v536: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 7.1 MiB/s wr, 709 op/s 2024-06-13T06:56:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:56 smithi175 ceph-mon[36886]: pgmap v536: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 7.1 MiB/s wr, 709 op/s 2024-06-13T06:56:57.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:56 smithi007 ceph-mon[32077]: pgmap v536: 129 pgs: 129 active+clean; 980 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 7.1 MiB/s wr, 709 op/s 2024-06-13T06:56:57.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 55.0 s, 2196.4 tps, lat 45.475 ms stddev 56.983 2024-06-13T06:56:59.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:56:58 smithi189 ceph-mon[36616]: pgmap v537: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.1 MiB/s wr, 732 op/s 2024-06-13T06:56:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:56:58 smithi175 ceph-mon[36886]: pgmap v537: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.1 MiB/s wr, 732 op/s 2024-06-13T06:56:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:56:58 smithi007 ceph-mon[32077]: pgmap v537: 129 pgs: 129 active+clean; 980 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.1 MiB/s wr, 732 op/s 2024-06-13T06:57:01.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:00 smithi189 ceph-mon[36616]: pgmap v538: 129 pgs: 129 active+clean; 980 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 8.1 MiB/s wr, 733 op/s 2024-06-13T06:57:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:00 smithi175 ceph-mon[36886]: pgmap v538: 129 pgs: 129 active+clean; 980 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 8.1 MiB/s wr, 733 op/s 2024-06-13T06:57:01.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:00 smithi007 ceph-mon[32077]: pgmap v538: 129 pgs: 129 active+clean; 980 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 8.1 MiB/s wr, 733 op/s 2024-06-13T06:57:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 60.0 s, 1884.4 tps, lat 52.716 ms stddev 66.537 2024-06-13T06:57:03.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:03 smithi189 ceph-mon[36616]: pgmap v539: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 9.2 MiB/s wr, 712 op/s 2024-06-13T06:57:03.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:03 smithi175 ceph-mon[36886]: pgmap v539: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 9.2 MiB/s wr, 712 op/s 2024-06-13T06:57:03.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:03 smithi007 ceph-mon[32077]: pgmap v539: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 9.2 MiB/s wr, 712 op/s 2024-06-13T06:57:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:05 smithi189 ceph-mon[36616]: pgmap v540: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-06-13T06:57:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:05 smithi175 ceph-mon[36886]: pgmap v540: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-06-13T06:57:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:05 smithi007 ceph-mon[32077]: pgmap v540: 129 pgs: 129 active+clean; 983 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 826 op/s 2024-06-13T06:57:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:07 smithi189 ceph-mon[36616]: pgmap v541: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.6 MiB/s wr, 731 op/s 2024-06-13T06:57:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:07 smithi175 ceph-mon[36886]: pgmap v541: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.6 MiB/s wr, 731 op/s 2024-06-13T06:57:07.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:07 smithi007 ceph-mon[32077]: pgmap v541: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.6 MiB/s wr, 731 op/s 2024-06-13T06:57:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 65.0 s, 1272.8 tps, lat 78.763 ms stddev 104.439 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.138+0000 7f437ee00640 1 -- 172.21.15.7:0/1963767368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378104e20 msgr2=0x7f43781052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.138+0000 7f437ee00640 1 --2- 172.21.15.7:0/1963767368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378104e20 0x7f43781052a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f436002a420 comp rx=0 tx=0).stop 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 -- 172.21.15.7:0/1963767368 shutdown_connections 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 --2- 172.21.15.7:0/1963767368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43781057e0 0x7f437810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 --2- 172.21.15.7:0/1963767368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378104e20 0x7f43781052a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 --2- 172.21.15.7:0/1963767368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378103c30 0x7f4378104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 -- 172.21.15.7:0/1963767368 >> 172.21.15.7:0/1963767368 conn(0x7f43780ff1c0 msgr2=0x7f43781015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 -- 172.21.15.7:0/1963767368 shutdown_connections 2024-06-13T06:57:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.139+0000 7f437ee00640 1 -- 172.21.15.7:0/1963767368 wait complete. 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.141+0000 7f437ee00640 1 Processor -- start 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.141+0000 7f437ee00640 1 -- start start 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.141+0000 7f437ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f437ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 0x7f437819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f437ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43781057e0 0x7f43781a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f437ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f437810f9a0 con 0x7f43781057e0 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f437ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f437810f820 con 0x7f4378104e20 2024-06-13T06:57:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f437ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f437810fb20 con 0x7f4378103c30 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58466/0 (socket says 172.21.15.7:58466) 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 -- 172.21.15.7:0/1126344716 learned_addr learned my addr 172.21.15.7:0/1126344716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377400640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 0x7f437819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 -- 172.21.15.7:0/1126344716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 msgr2=0x7f437819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 0x7f437819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 -- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43781057e0 msgr2=0x7f43781a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43781057e0 0x7f43781a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377e00640 1 -- 172.21.15.7:0/1126344716 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43781a1b10 con 0x7f4378103c30 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.142+0000 7f4377400640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 0x7f437819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.143+0000 7f4377e00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f437000eb70 tx=0x7f437000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.143+0000 7f4374c00640 1 -- 172.21.15.7:0/1126344716 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370003b70 con 0x7f4378103c30 2024-06-13T06:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.143+0000 7f4374c00640 1 -- 172.21.15.7:0/1126344716 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437000ee70 con 0x7f4378103c30 2024-06-13T06:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.143+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437806b420 con 0x7f4378103c30 2024-06-13T06:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.143+0000 7f4374c00640 1 -- 172.21.15.7:0/1126344716 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370010830 con 0x7f4378103c30 2024-06-13T06:57:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.144+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437806b9c0 con 0x7f4378103c30 2024-06-13T06:57:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.148+0000 7f4381d86740 1 Processor -- start 2024-06-13T06:57:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.148+0000 7f4381d86740 1 -- start start 2024-06-13T06:57:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 0x5564d125a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d125f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d1262300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5564d11c2420 con 0x5564d125a310 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564d1169c50 con 0x5564d125fe70 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4381d86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564d11be3e0 con 0x5564d125ad00 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4377400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d1262300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4377400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d1262300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39312/0 (socket says 172.21.15.7:39312) 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f4377400640 1 -- 172.21.15.7:0/1546719828 learned_addr learned my addr 172.21.15.7:0/1546719828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.149+0000 7f437ce00640 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 0x5564d125a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f437ce00640 1 -- 172.21.15.7:0/1546719828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 msgr2=0x5564d125f800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f437ce00640 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d125f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f437ce00640 1 -- 172.21.15.7:0/1546719828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 msgr2=0x5564d1262300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f437ce00640 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d1262300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f437ce00640 1 -- 172.21.15.7:0/1546719828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d0ed6210 con 0x5564d125a310 2024-06-13T06:57:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.150+0000 7f4374c00640 1 -- 172.21.15.7:0/1126344716 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f43700109d0 con 0x7f4378103c30 2024-06-13T06:57:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.152+0000 7f437ce00640 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 0x5564d125a730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43680096a0 tx=0x7f4368009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9847dc754a984236 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.152+0000 7f4374c00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 0x7f43580a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.152+0000 7f4374c00640 1 -- 172.21.15.7:0/1126344716 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f43700cfb70 con 0x7f4378103c30 2024-06-13T06:57:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.153+0000 7f4377e00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 0x7f43580a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.153+0000 7f436d600640 1 -- 172.21.15.7:0/1546719828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368015020 con 0x5564d125a310 2024-06-13T06:57:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.153+0000 7f436d600640 1 -- 172.21.15.7:0/1546719828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436800bda0 con 0x5564d125a310 2024-06-13T06:57:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.153+0000 7f436d600640 1 -- 172.21.15.7:0/1546719828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368010380 con 0x5564d125a310 2024-06-13T06:57:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.154+0000 7f4381d86740 1 -- 172.21.15.7:0/1546719828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 msgr2=0x5564d125a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.154+0000 7f4381d86740 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 0x5564d125a730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43680096a0 tx=0x7f4368009b60 comp rx=0 tx=0).stop 2024-06-13T06:57:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.154+0000 7f4377e00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 0x7f43580a5240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f43700a4770 tx=0x7f4370004780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 -- 172.21.15.7:0/1546719828 shutdown_connections 2024-06-13T06:57:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d1262300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d125f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 --2- 172.21.15.7:0/1546719828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125a310 0x5564d125a730 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 -- 172.21.15.7:0/1546719828 >> 172.21.15.7:0/1546719828 conn(0x5564d1226fb0 msgr2=0x5564d1262e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 -- 172.21.15.7:0/1546719828 shutdown_connections 2024-06-13T06:57:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.155+0000 7f4381d86740 1 -- 172.21.15.7:0/1546719828 wait complete. 2024-06-13T06:57:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.156+0000 7f4381d86740 1 -- start start 2024-06-13T06:57:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.158+0000 7f4381d86740 1 Processor -- start 2024-06-13T06:57:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d1268cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125fe70 0x5564d1270250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5564d1267370 con 0x5564d125fe70 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5564d117d0b0 con 0x5564d125a310 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564d12674f0 con 0x5564d125ad00 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.159+0000 7f4381d86740 10 client.0 _ll_register_callbacks cb 0x5564d1168f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f437ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d1268cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58476/0 (socket says 172.21.15.7:58476) 2024-06-13T06:57:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 -- 172.21.15.7:0/961545345 learned_addr learned my addr 172.21.15.7:0/961545345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f437ce00640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125fe70 0x5564d1270250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 -- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 msgr2=0x5564d1268cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d1268cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 -- 172.21.15.7:0/961545345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125fe70 msgr2=0x5564d1270250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125fe70 0x5564d1270250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.160+0000 7f4377400640 1 -- 172.21.15.7:0/961545345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d0fa2620 con 0x5564d125ad00 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.161+0000 7f4377400640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4360006010 tx=0x7f4360005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595c3dcb884a6195 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.161+0000 7f4366000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.161+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360035040 con 0x5564d125ad00 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.161+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f43600318b0 con 0x5564d125ad00 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.162+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436800bda0 con 0x5564d125ad00 2024-06-13T06:57:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.162+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4360031ab0 con 0x5564d125ad00 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.162+0000 7f4381d86740 10 client.25453 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.162+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564d1273630 con 0x5564d125ad00 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.162+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436800bda0 con 0x5564d125ad00 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f4360075020 con 0x5564d125ad00 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 10 client.25453 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 20 client.25453 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d1277110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 4 client.25453 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d11cfb50 con 0x5564d1274cd0 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.163+0000 7f437ce00640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d1277110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.164+0000 7f437ce00640 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d1277110 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9a984a2c9f3683 server_cookie=c4f7db97c5075ad2 in_seq=0 out_seq=0 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.164+0000 7f4366000640 10 client.25453 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.164+0000 7f4366000640 1 -- 172.21.15.7:0/961545345 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564d11cfb50 con 0x5564d1274cd0 2024-06-13T06:57:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.164+0000 7f4366000640 10 client.25453 handle_command_reply: tid=0 2024-06-13T06:57:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 client.25453 shutdown 2024-06-13T06:57:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 msgr2=0x5564d1269210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4360006010 tx=0x7f4360005270 comp rx=0 tx=0).stop 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 shutdown_connections 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d1277110 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125fe70 0x5564d1270250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4366000640 0 client.25453 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125ad00 0x5564d1269210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 --2- 172.21.15.7:0/961545345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d1268cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 >> 172.21.15.7:0/961545345 conn(0x5564d1226fb0 msgr2=0x5564d1259840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.165+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 shutdown_connections 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.166+0000 7f4381d86740 1 -- 172.21.15.7:0/961545345 wait complete. 2024-06-13T06:57:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.166+0000 7f4381d86740 20 client.25453 trim_cache size 0 max 16384 2024-06-13T06:57:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.191+0000 7f4381d86740 1 Processor -- start 2024-06-13T06:57:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.191+0000 7f4381d86740 1 -- start start 2024-06-13T06:57:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.192+0000 7f4381d86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 0x5564d12779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.192+0000 7f4381d86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d1270210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f4381d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d125c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f4381d86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43781a24b0 con 0x5564d125ad00 2024-06-13T06:57:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f4381d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4378071c40 con 0x5564d125fe70 2024-06-13T06:57:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f4381d86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f437806fa90 con 0x5564d125a310 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f437ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 0x5564d12779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f437ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 0x5564d12779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58484/0 (socket says 172.21.15.7:58484) 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f437ce00640 1 -- 172.21.15.7:0/78698167 learned_addr learned my addr 172.21.15.7:0/78698167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.193+0000 7f4377400640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d1270210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f437ce00640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d125c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f4377400640 1 -- 172.21.15.7:0/78698167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 msgr2=0x5564d12779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f4377400640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 0x5564d12779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f4377400640 1 -- 172.21.15.7:0/78698167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 msgr2=0x5564d125c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f4377400640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d125c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f4377400640 1 -- 172.21.15.7:0/78698167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d125bc00 con 0x5564d125ad00 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.194+0000 7f437ce00640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d125c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.195+0000 7f4377400640 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d1270210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43600314e0 tx=0x7f436002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd133e19c05ab4c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.195+0000 7f436cc00640 1 -- 172.21.15.7:0/78698167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360000e60 con 0x5564d125ad00 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.195+0000 7f436cc00640 1 -- 172.21.15.7:0/78698167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360086070 con 0x5564d125ad00 2024-06-13T06:57:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.195+0000 7f436cc00640 1 -- 172.21.15.7:0/78698167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360035040 con 0x5564d125ad00 2024-06-13T06:57:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 -- 172.21.15.7:0/78698167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 msgr2=0x5564d1270210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d1270210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f43600314e0 tx=0x7f436002a420 comp rx=0 tx=0).stop 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 -- 172.21.15.7:0/78698167 shutdown_connections 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125fe70 0x5564d125c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d1270210 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 --2- 172.21.15.7:0/78698167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125a310 0x5564d12779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 -- 172.21.15.7:0/78698167 >> 172.21.15.7:0/78698167 conn(0x5564d1226fb0 msgr2=0x5564d1268cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 -- 172.21.15.7:0/78698167 shutdown_connections 2024-06-13T06:57:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.197+0000 7f4381d86740 1 -- 172.21.15.7:0/78698167 wait complete. 2024-06-13T06:57:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.198+0000 7f4381d86740 1 -- start start 2024-06-13T06:57:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.198+0000 7f4381d86740 1 Processor -- start 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d126c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d126c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125fe70 0x5564d126f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4378070100 con 0x5564d125ad00 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4378071570 con 0x5564d125a310 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43780707d0 con 0x5564d125fe70 2024-06-13T06:57:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4381d86740 10 client.0 _ll_register_callbacks cb 0x7f4360009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f437ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d126c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.199+0000 7f4377400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d126c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f437ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d126c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:39332/0 (socket says 172.21.15.7:39332) 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f437ce00640 1 -- 172.21.15.7:0/68173239 learned_addr learned my addr 172.21.15.7:0/68173239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f4377400640 1 -- 172.21.15.7:0/68173239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125fe70 msgr2=0x5564d126f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f4377400640 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125fe70 0x5564d126f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f4377400640 1 -- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 msgr2=0x5564d126c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f4377400640 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d126c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.200+0000 7f4377400640 1 -- 172.21.15.7:0/68173239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d126fdb0 con 0x5564d125ad00 2024-06-13T06:57:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.202+0000 7f4377400640 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d126c7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f43600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8163d2857bcd559 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.202+0000 7f4366000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:57:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436008f060 con 0x5564d125ad00 2024-06-13T06:57:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f43600039b0 con 0x5564d125ad00 2024-06-13T06:57:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4360086070 con 0x5564d125ad00 2024-06-13T06:57:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4360009520 con 0x5564d125ad00 2024-06-13T06:57:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4381d86740 10 client.15264 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.203+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564d12ded70 con 0x5564d125ad00 2024-06-13T06:57:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360086070 con 0x5564d125ad00 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f4360075070 con 0x5564d125ad00 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 10 client.15264 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 20 client.15264 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d12e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 4 client.15264 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.205+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564d11c9ea0 con 0x5564d1274cd0 2024-06-13T06:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.206+0000 7f437ce00640 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d12e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.206+0000 7f437ce00640 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d12e16a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33d6b38bda2f51e server_cookie=3f5a56b40a7c31f6 in_seq=0 out_seq=0 2024-06-13T06:57:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.206+0000 7f4366000640 10 client.15264 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.221+0000 7f4366000640 1 -- 172.21.15.7:0/68173239 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5564d11c9ea0 con 0x5564d1274cd0 2024-06-13T06:57:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.221+0000 7f4366000640 10 client.15264 handle_command_reply: tid=0 2024-06-13T06:57:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.221+0000 7f4381d86740 1 client.15264 shutdown 2024-06-13T06:57:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.221+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 msgr2=0x5564d126c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.221+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d126c7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f43600048f0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 shutdown_connections 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5564d1274cd0 0x5564d12e16a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d125fe70 0x5564d126f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564d125ad00 0x5564d126c7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 --2- 172.21.15.7:0/68173239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5564d125a310 0x5564d126c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 >> 172.21.15.7:0/68173239 conn(0x5564d1226fb0 msgr2=0x5564d1265e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4366000640 0 client.15264 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 shutdown_connections 2024-06-13T06:57:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.222+0000 7f4381d86740 1 -- 172.21.15.7:0/68173239 wait complete. 2024-06-13T06:57:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.223+0000 7f4381d86740 20 client.15264 trim_cache size 0 max 16384 2024-06-13T06:57:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:57:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:57:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 msgr2=0x7f43580a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 0x7f43580a5240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f43700a4770 tx=0x7f4370004780 comp rx=0 tx=0).stop 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 msgr2=0x7f4378199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f437000eb70 tx=0x7f437000c6a0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 shutdown_connections 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f43580a2d80 0x7f43580a5240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43781057e0 0x7f43781a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4378104e20 0x7f437819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 --2- 172.21.15.7:0/1126344716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4378103c30 0x7f4378199ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.224+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 >> 172.21.15.7:0/1126344716 conn(0x7f43780ff1c0 msgr2=0x7f4378108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.225+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 shutdown_connections 2024-06-13T06:57:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:08.225+0000 7f437ee00640 1 -- 172.21.15.7:0/1126344716 wait complete. 2024-06-13T06:57:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:09 smithi189 ceph-mon[36616]: pgmap v542: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.8 MiB/s wr, 761 op/s 2024-06-13T06:57:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:09 smithi175 ceph-mon[36886]: pgmap v542: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.8 MiB/s wr, 761 op/s 2024-06-13T06:57:09.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:09 smithi007 ceph-mon[32077]: pgmap v542: 129 pgs: 129 active+clean; 983 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.8 MiB/s wr, 761 op/s 2024-06-13T06:57:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:11 smithi189 ceph-mon[36616]: pgmap v543: 129 pgs: 129 active+clean; 991 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 734 op/s 2024-06-13T06:57:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:11 smithi175 ceph-mon[36886]: pgmap v543: 129 pgs: 129 active+clean; 991 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 734 op/s 2024-06-13T06:57:11.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:11 smithi007 ceph-mon[32077]: pgmap v543: 129 pgs: 129 active+clean; 991 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 734 op/s 2024-06-13T06:57:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 70.0 s, 1993.6 tps, lat 50.294 ms stddev 57.860 2024-06-13T06:57:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:13 smithi189 ceph-mon[36616]: pgmap v544: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 713 op/s 2024-06-13T06:57:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:13 smithi175 ceph-mon[36886]: pgmap v544: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 713 op/s 2024-06-13T06:57:13.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:13 smithi007 ceph-mon[32077]: pgmap v544: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 713 op/s 2024-06-13T06:57:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:15 smithi189 ceph-mon[36616]: pgmap v545: 129 pgs: 129 active+clean; 999 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-06-13T06:57:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:15 smithi175 ceph-mon[36886]: pgmap v545: 129 pgs: 129 active+clean; 999 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-06-13T06:57:15.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:15 smithi007 ceph-mon[32077]: pgmap v545: 129 pgs: 129 active+clean; 999 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 750 op/s 2024-06-13T06:57:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:17 smithi189 ceph-mon[36616]: pgmap v546: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-06-13T06:57:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:17 smithi175 ceph-mon[36886]: pgmap v546: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-06-13T06:57:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:17 smithi007 ceph-mon[32077]: pgmap v546: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 638 op/s 2024-06-13T06:57:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 75.0 s, 1781.0 tps, lat 55.897 ms stddev 86.221 2024-06-13T06:57:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:19 smithi189 ceph-mon[36616]: pgmap v547: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-13T06:57:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:19 smithi175 ceph-mon[36886]: pgmap v547: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-13T06:57:19.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:19 smithi007 ceph-mon[32077]: pgmap v547: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-06-13T06:57:21.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:21 smithi189 ceph-mon[36616]: pgmap v548: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-06-13T06:57:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:21 smithi175 ceph-mon[36886]: pgmap v548: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-06-13T06:57:21.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:21 smithi007 ceph-mon[32077]: pgmap v548: 129 pgs: 129 active+clean; 1001 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-06-13T06:57:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.434+0000 7fc4fcc00640 1 -- 172.21.15.7:0/3876366073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 msgr2=0x7fc4f8109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.434+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/3876366073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f8109b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009ca0 tx=0x7fc4f002a8c0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 -- 172.21.15.7:0/3876366073 shutdown_connections 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/3876366073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f8109b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/3876366073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f8100040 0x7fc4f8107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/3876366073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f80ff650 0x7fc4f80ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 -- 172.21.15.7:0/3876366073 >> 172.21.15.7:0/3876366073 conn(0x7fc4f80fb1c0 msgr2=0x7fc4f80fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 -- 172.21.15.7:0/3876366073 shutdown_connections 2024-06-13T06:57:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.435+0000 7fc4fcc00640 1 -- 172.21.15.7:0/3876366073 wait complete. 2024-06-13T06:57:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.439+0000 7fc4fcc00640 1 Processor -- start 2024-06-13T06:57:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 -- start start 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ff650 0x7fc4f8197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 0x7fc4f8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f810d6a0 con 0x7fc4f80ff650 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f810d520 con 0x7fc4f8100040 2024-06-13T06:57:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4fcc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f810d820 con 0x7fc4f81076d0 2024-06-13T06:57:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 0x7fc4f8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44232/0 (socket says 172.21.15.7:44232) 2024-06-13T06:57:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.440+0000 7fc4f6800640 1 -- 172.21.15.7:0/51217598 learned_addr learned my addr 172.21.15.7:0/51217598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ff650 0x7fc4f8197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f6800640 1 -- 172.21.15.7:0/51217598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 msgr2=0x7fc4f8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f6800640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 0x7fc4f8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f6800640 1 -- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ff650 msgr2=0x7fc4f8197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f6800640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ff650 0x7fc4f8197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f6800640 1 -- 172.21.15.7:0/51217598 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f819f810 con 0x7fc4f81076d0 2024-06-13T06:57:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.441+0000 7fc4f5400640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 0x7fc4f8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.442+0000 7fc4f6800640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0031ae0 tx=0x7fc4f00319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.442+0000 7fc4eea00640 1 -- 172.21.15.7:0/51217598 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0003b70 con 0x7fc4f81076d0 2024-06-13T06:57:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.442+0000 7fc4eea00640 1 -- 172.21.15.7:0/51217598 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0003d10 con 0x7fc4f81076d0 2024-06-13T06:57:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.442+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f8100f60 con 0x7fc4f81076d0 2024-06-13T06:57:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.443+0000 7fc4eea00640 1 -- 172.21.15.7:0/51217598 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00047f0 con 0x7fc4f81076d0 2024-06-13T06:57:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.443+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f8101500 con 0x7fc4f81076d0 2024-06-13T06:57:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 Processor -- start 2024-06-13T06:57:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4eea00640 1 -- 172.21.15.7:0/51217598 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc4f0040820 con 0x7fc4f81076d0 2024-06-13T06:57:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 -- start start 2024-06-13T06:57:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c8f70 0x556bfb5cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.450+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bfb51e420 con 0x556bfb5c3e00 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bfb4c5c50 con 0x556bfb5c8f70 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556bfb51a3e0 con 0x556bfb5c3410 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44244/0 (socket says 172.21.15.7:44244) 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4f5400640 1 -- 172.21.15.7:0/4277098717 learned_addr learned my addr 172.21.15.7:0/4277098717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.451+0000 7fc4f5400640 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c8f70 0x556bfb5cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4f5400640 1 -- 172.21.15.7:0/4277098717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c8f70 msgr2=0x556bfb5cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4f5400640 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c8f70 0x556bfb5cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4f5400640 1 -- 172.21.15.7:0/4277098717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 msgr2=0x556bfb5c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4f5400640 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4f5400640 1 -- 172.21.15.7:0/4277098717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfb232210 con 0x556bfb5c3410 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.452+0000 7fc4eea00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 0x7fc4dc0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.453+0000 7fc4f5400640 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40139f0 tx=0x7fc4e4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e55a5eb38a9e9c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.453+0000 7fc4e3000640 1 -- 172.21.15.7:0/4277098717 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400f980 con 0x556bfb5c3410 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.453+0000 7fc4e3000640 1 -- 172.21.15.7:0/4277098717 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4010430 con 0x556bfb5c3410 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.453+0000 7fc4e3000640 1 -- 172.21.15.7:0/4277098717 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400e610 con 0x556bfb5c3410 2024-06-13T06:57:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.454+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 0x7fc4dc0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/4277098717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 msgr2=0x556bfb5c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40139f0 tx=0x7fc4e4005850 comp rx=0 tx=0).stop 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/4277098717 shutdown_connections 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c8f70 0x556bfb5cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/4277098717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c3410 0x556bfb5c3830 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.455+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/4277098717 >> 172.21.15.7:0/4277098717 conn(0x556bfb582fb0 msgr2=0x556bfb5bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.456+0000 7fc4eea00640 1 -- 172.21.15.7:0/51217598 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc4f00f07e0 con 0x7fc4f81076d0 2024-06-13T06:57:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.456+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/4277098717 shutdown_connections 2024-06-13T06:57:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.456+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 0x7fc4dc0c7180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x556bfb5c4200 tx=0x7fc4e800b770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.457+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/4277098717 wait complete. 2024-06-13T06:57:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 -- start start 2024-06-13T06:57:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 Processor -- start 2024-06-13T06:57:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bfb5cbd60 con 0x556bfb5c3e00 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556bfb4d90b0 con 0x556bfb5c3410 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556bfb5cbee0 con 0x556bfb5c8f70 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.458+0000 7fc4ffa1f740 10 client.0 _ll_register_callbacks cb 0x556bfb4c4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.459+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.459+0000 7fc4f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.459+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36784/0 (socket says 172.21.15.7:36784) 2024-06-13T06:57:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.459+0000 7fc4f5e00640 1 -- 172.21.15.7:0/2258045508 learned_addr learned my addr 172.21.15.7:0/2258045508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f5e00640 1 -- 172.21.15.7:0/2258045508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 msgr2=0x556bfb5d4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f5e00640 1 -- 172.21.15.7:0/2258045508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 msgr2=0x556bfb5cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f5e00640 1 -- 172.21.15.7:0/2258045508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfb2fe620 con 0x556bfb5c3410 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.460+0000 7fc4f6800640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.461+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80116f0 tx=0x7fc4e800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5edef63d881cc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.461+0000 7fc4d7400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:57:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.462+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e801e8f0 con 0x556bfb5c3410 2024-06-13T06:57:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.462+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc4e801ea90 con 0x556bfb5c3410 2024-06-13T06:57:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.462+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e4010430 con 0x556bfb5c3410 2024-06-13T06:57:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.463+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4e801ec90 con 0x556bfb5c3410 2024-06-13T06:57:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.463+0000 7fc4ffa1f740 10 client.25478 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.464+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bfb5d7f40 con 0x556bfb5c3410 2024-06-13T06:57:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.464+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e4010430 con 0x556bfb5c3410 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc4e801c4d0 con 0x556bfb5c3410 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 10 client.25478 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 20 client.25478 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb5d9630 0x556bfb5dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 4 client.25478 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.465+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bfb52bb50 con 0x556bfb5d9630 2024-06-13T06:57:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.466+0000 7fc4f5400640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb5d9630 0x556bfb5dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.466+0000 7fc4f5400640 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb5d9630 0x556bfb5dbaf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a216edef9665092f server_cookie=f059e01cf7ad680f in_seq=0 out_seq=0 2024-06-13T06:57:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.466+0000 7fc4d7400640 10 client.25478 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.467+0000 7fc4d7400640 1 -- 172.21.15.7:0/2258045508 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556bfb52bb50 con 0x556bfb5d9630 2024-06-13T06:57:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.467+0000 7fc4d7400640 10 client.25478 handle_command_reply: tid=0 2024-06-13T06:57:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.468+0000 7fc4ffa1f740 1 client.25478 shutdown 2024-06-13T06:57:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.468+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 msgr2=0x556bfb5cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.468+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80116f0 tx=0x7fc4e800b390 comp rx=0 tx=0).stop 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 shutdown_connections 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb5d9630 0x556bfb5dbaf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5c3e00 0x556bfb5cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/2258045508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3410 0x556bfb5cd590 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80116f0 tx=0x7fc4e800b390 comp rx=0 tx=0).stop 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4d7400640 0 client.25478 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 >> 172.21.15.7:0/2258045508 conn(0x556bfb582fb0 msgr2=0x556bfb5b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 shutdown_connections 2024-06-13T06:57:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.469+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/2258045508 wait complete. 2024-06-13T06:57:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.470+0000 7fc4ffa1f740 20 client.25478 trim_cache size 0 max 16384 2024-06-13T06:57:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 Processor -- start 2024-06-13T06:57:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 -- start start 2024-06-13T06:57:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb5be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81a2370 con 0x556bfb5d5c60 2024-06-13T06:57:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f819fc10 con 0x556bfb5c3e00 2024-06-13T06:57:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.502+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8071c50 con 0x556bfb5c8f70 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44264/0 (socket says 172.21.15.7:44264) 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 -- 172.21.15.7:0/626483123 learned_addr learned my addr 172.21.15.7:0/626483123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f6800640 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb5be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 -- 172.21.15.7:0/626483123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 msgr2=0x556bfb5c8750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 -- 172.21.15.7:0/626483123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 msgr2=0x556bfb5be910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb5be910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.503+0000 7fc4f5400640 1 -- 172.21.15.7:0/626483123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfb5b16d0 con 0x556bfb5c8f70 2024-06-13T06:57:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.504+0000 7fc4f5400640 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40058c0 tx=0x7fc4e400f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdf368b4771925 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.504+0000 7fc4e2600640 1 -- 172.21.15.7:0/626483123 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40052c0 con 0x556bfb5c8f70 2024-06-13T06:57:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.504+0000 7fc4e2600640 1 -- 172.21.15.7:0/626483123 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e400fe60 con 0x556bfb5c8f70 2024-06-13T06:57:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.504+0000 7fc4e2600640 1 -- 172.21.15.7:0/626483123 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40035f0 con 0x556bfb5c8f70 2024-06-13T06:57:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.505+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/626483123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 msgr2=0x556bfb5d4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.505+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40058c0 tx=0x7fc4e400f840 comp rx=0 tx=0).stop 2024-06-13T06:57:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.506+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/626483123 shutdown_connections 2024-06-13T06:57:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.506+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb5be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.506+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d4ad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.507+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/626483123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.507+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/626483123 >> 172.21.15.7:0/626483123 conn(0x556bfb582fb0 msgr2=0x556bfb5c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.507+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/626483123 shutdown_connections 2024-06-13T06:57:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.508+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/626483123 wait complete. 2024-06-13T06:57:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.508+0000 7fc4ffa1f740 1 -- start start 2024-06-13T06:57:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.508+0000 7fc4ffa1f740 1 Processor -- start 2024-06-13T06:57:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb642ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80722c0 con 0x556bfb5d5c60 2024-06-13T06:57:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8073730 con 0x556bfb5c3e00 2024-06-13T06:57:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8072990 con 0x556bfb5c8f70 2024-06-13T06:57:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4ffa1f740 10 client.0 _ll_register_callbacks cb 0x7fc4e8010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4f5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44280/0 (socket says 172.21.15.7:44280) 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4f5400640 1 -- 172.21.15.7:0/1535297006 learned_addr learned my addr 172.21.15.7:0/1535297006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.510+0000 7fc4f6800640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb642ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f6800640 1 -- 172.21.15.7:0/1535297006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 msgr2=0x556bfb5d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f6800640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f6800640 1 -- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 msgr2=0x556bfb5d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f6800640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f6800640 1 -- 172.21.15.7:0/1535297006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfb5d4d10 con 0x556bfb5d5c60 2024-06-13T06:57:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.511+0000 7fc4f5400640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.514+0000 7fc4f6800640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb642ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00edf40 tx=0x7fc4f00c0be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327dd605fce5a6d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.514+0000 7fc4d7400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:57:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.515+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00c10a0 con 0x556bfb5d5c60 2024-06-13T06:57:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.515+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc4f00e10b0 con 0x556bfb5d5c60 2024-06-13T06:57:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.515+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e400fe60 con 0x556bfb5d5c60 2024-06-13T06:57:21.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.515+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4f00c5450 con 0x556bfb5d5c60 2024-06-13T06:57:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.516+0000 7fc4ffa1f740 10 client.15276 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.516+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bfb644300 con 0x556bfb5d5c60 2024-06-13T06:57:21.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e400fe60 con 0x556bfb5d5c60 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc4f00dc030 con 0x556bfb5d5c60 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 10 client.15276 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 20 client.15276 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb644ce0 0x556bfb647100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 4 client.15276 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:57:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bfb525ea0 con 0x556bfb644ce0 2024-06-13T06:57:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.517+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb644ce0 0x556bfb647100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.518+0000 7fc4f5e00640 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb644ce0 0x556bfb647100 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4ef1d3d49431a1 server_cookie=3d0bf866c59344b1 in_seq=0 out_seq=0 2024-06-13T06:57:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.518+0000 7fc4d7400640 10 client.15276 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.535+0000 7fc4d7400640 1 -- 172.21.15.7:0/1535297006 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556bfb525ea0 con 0x556bfb644ce0 2024-06-13T06:57:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.535+0000 7fc4d7400640 10 client.15276 handle_command_reply: tid=0 2024-06-13T06:57:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 client.15276 shutdown 2024-06-13T06:57:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 msgr2=0x556bfb642ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb642ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00edf40 tx=0x7fc4f00c0be0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 shutdown_connections 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556bfb644ce0 0x556bfb647100 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bfb5d5c60 0x556bfb642ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bfb5c8f70 0x556bfb5d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 --2- 172.21.15.7:0/1535297006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556bfb5c3e00 0x556bfb5d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.536+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 >> 172.21.15.7:0/1535297006 conn(0x556bfb582fb0 msgr2=0x556bfb5c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.537+0000 7fc4d7400640 0 client.15276 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.537+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 shutdown_connections 2024-06-13T06:57:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.537+0000 7fc4ffa1f740 1 -- 172.21.15.7:0/1535297006 wait complete. 2024-06-13T06:57:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.537+0000 7fc4ffa1f740 20 client.15276 trim_cache size 0 max 16384 2024-06-13T06:57:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:57:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:57:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:57:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.538+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 msgr2=0x7fc4dc0c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.538+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 0x7fc4dc0c7180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x556bfb5c4200 tx=0x7fc4e800b770 comp rx=0 tx=0).stop 2024-06-13T06:57:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.538+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 msgr2=0x7fc4f819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.538+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0031ae0 tx=0x7fc4f00319f0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 shutdown_connections 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc4dc0c4cc0 0x7fc4dc0c7180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4f81076d0 0x7fc4f819f1a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4f8100040 0x7fc4f8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 --2- 172.21.15.7:0/51217598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ff650 0x7fc4f8197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 >> 172.21.15.7:0/51217598 conn(0x7fc4f80fb1c0 msgr2=0x7fc4f8103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 shutdown_connections 2024-06-13T06:57:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:21.539+0000 7fc4fcc00640 1 -- 172.21.15.7:0/51217598 wait complete. 2024-06-13T06:57:21.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:57:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 80.0 s, 1395.8 tps, lat 71.426 ms stddev 81.914 2024-06-13T06:57:23.261 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:23 smithi007 ceph-mon[32077]: pgmap v549: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 717 op/s 2024-06-13T06:57:23.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:23 smithi189 ceph-mon[36616]: pgmap v549: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 717 op/s 2024-06-13T06:57:23.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:23 smithi175 ceph-mon[36886]: pgmap v549: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 717 op/s 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 -- 172.21.15.7:0/3148888904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406ab10 msgr2=0x7fc13410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 --2- 172.21.15.7:0/3148888904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406ab10 0x7fc13410f320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0099c0 tx=0x7fc12c009cd0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 -- 172.21.15.7:0/3148888904 shutdown_connections 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 --2- 172.21.15.7:0/3148888904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406ab10 0x7fc13410f320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 --2- 172.21.15.7:0/3148888904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1341084e0 0x7fc1341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 --2- 172.21.15.7:0/3148888904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc13406a170 0x7fc13406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 -- 172.21.15.7:0/3148888904 >> 172.21.15.7:0/3148888904 conn(0x7fc1340782c0 msgr2=0x7fc1340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 -- 172.21.15.7:0/3148888904 shutdown_connections 2024-06-13T06:57:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.118+0000 7fc139000700 1 -- 172.21.15.7:0/3148888904 wait complete. 2024-06-13T06:57:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.121+0000 7fc139000700 1 Processor -- start 2024-06-13T06:57:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.121+0000 7fc139000700 1 -- start start 2024-06-13T06:57:25.121 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:25 smithi007 ceph-mon[32077]: pgmap v550: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 771 op/s 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406a170 0x7fc13419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13406ab10 0x7fc13419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341127b0 con 0x7fc13406ab10 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc134112630 con 0x7fc1341084e0 2024-06-13T06:57:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc139000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc134112930 con 0x7fc13406a170 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc132800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc132800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36812/0 (socket says 172.21.15.7:36812) 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 -- 172.21.15.7:0/2855408342 learned_addr learned my addr 172.21.15.7:0/2855408342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.123+0000 7fc131e00700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406a170 0x7fc13419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc12be00700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13406ab10 0x7fc13419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 -- 172.21.15.7:0/2855408342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406a170 msgr2=0x7fc13419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406a170 0x7fc13419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 -- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13406ab10 msgr2=0x7fc13419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13406ab10 0x7fc13419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.124+0000 7fc132800700 1 -- 172.21.15.7:0/2855408342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13419fe30 con 0x7fc1341084e0 2024-06-13T06:57:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.125+0000 7fc132800700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0047c0 tx=0x7fc12c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.125+0000 7fc12aa00700 1 -- 172.21.15.7:0/2855408342 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c004830 con 0x7fc1341084e0 2024-06-13T06:57:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.125+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a00c0 con 0x7fc1341084e0 2024-06-13T06:57:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.126+0000 7fc12aa00700 1 -- 172.21.15.7:0/2855408342 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc12c005020 con 0x7fc1341084e0 2024-06-13T06:57:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.126+0000 7fc12aa00700 1 -- 172.21.15.7:0/2855408342 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c003ec0 con 0x7fc1341084e0 2024-06-13T06:57:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.126+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a79a0 con 0x7fc1341084e0 2024-06-13T06:57:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.133+0000 7fc12aa00700 1 -- 172.21.15.7:0/2855408342 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc12c029060 con 0x7fc1341084e0 2024-06-13T06:57:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.134+0000 7fc1438c1b80 1 Processor -- start 2024-06-13T06:57:25.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.135+0000 7fc1438c1b80 1 -- start start 2024-06-13T06:57:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.136+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 0x55e817129510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817129e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e81702fa10 con 0x55e81712fcc0 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e817072b00 con 0x55e81712f3a0 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e81702d8f0 con 0x55e817129a50 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36822/0 (socket says 172.21.15.7:36822) 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 -- 172.21.15.7:0/406409510 learned_addr learned my addr 172.21.15.7:0/406409510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817129e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 -- 172.21.15.7:0/406409510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 msgr2=0x55e817129e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817129e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.137+0000 7fc12be00700 1 -- 172.21.15.7:0/406409510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 msgr2=0x55e817129510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.138+0000 7fc131e00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 0x55e817129510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.138+0000 7fc12be00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 0x55e817129510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.138+0000 7fc12be00700 1 -- 172.21.15.7:0/406409510 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81701af90 con 0x55e81712f3a0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.138+0000 7fc131e00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 0x55e817129510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.138+0000 7fc12aa00700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 0x7fc1180a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.139+0000 7fc12be00700 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc11c013930 tx=0x7fc11c00b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d21698ee4b90cf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.140+0000 7fc131e00700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 0x7fc1180a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.140+0000 7fc123400700 1 -- 172.21.15.7:0/406409510 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c004220 con 0x55e81712f3a0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.140+0000 7fc123400700 1 -- 172.21.15.7:0/406409510 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c0043c0 con 0x55e81712f3a0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.140+0000 7fc123400700 1 -- 172.21.15.7:0/406409510 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0116a0 con 0x55e81712f3a0 2024-06-13T06:57:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.140+0000 7fc12aa00700 1 -- 172.21.15.7:0/2855408342 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc12c0cfca0 con 0x7fc1341084e0 2024-06-13T06:57:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.142+0000 7fc1438c1b80 1 -- 172.21.15.7:0/406409510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 msgr2=0x55e81712f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.142+0000 7fc131e00700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 0x7fc1180a51e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc1240060b0 tx=0x7fc12400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.142+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc11c013930 tx=0x7fc11c00b460 comp rx=0 tx=0).stop 2024-06-13T06:57:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 -- 172.21.15.7:0/406409510 shutdown_connections 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817129e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e81712f780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/406409510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712fcc0 0x55e817129510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 -- 172.21.15.7:0/406409510 >> 172.21.15.7:0/406409510 conn(0x55e81704d170 msgr2=0x55e81711bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 -- 172.21.15.7:0/406409510 shutdown_connections 2024-06-13T06:57:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.143+0000 7fc1438c1b80 1 -- 172.21.15.7:0/406409510 wait complete. 2024-06-13T06:57:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.144+0000 7fc1438c1b80 1 -- start start 2024-06-13T06:57:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.144+0000 7fc1438c1b80 1 Processor -- start 2024-06-13T06:57:25.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e817132d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e817133240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e81700ef60 con 0x55e81712f3a0 2024-06-13T06:57:25.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e81706eac0 con 0x55e81712fcc0 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8171366a0 con 0x55e817129a50 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.145+0000 7fc1438c1b80 10 client.0 _ll_register_callbacks cb 0x55e817008380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44318/0 (socket says 172.21.15.7:44318) 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 -- 172.21.15.7:0/2653140744 learned_addr learned my addr 172.21.15.7:0/2653140744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc132800700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e817133240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 -- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 msgr2=0x55e817133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e817133240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.146+0000 7fc131e00700 1 -- 172.21.15.7:0/2653140744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 msgr2=0x55e817132d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:57:25.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.147+0000 7fc131e00700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e817132d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.147+0000 7fc131e00700 1 -- 172.21.15.7:0/2653140744 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e816fc18d0 con 0x55e817129a50 2024-06-13T06:57:25.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.148+0000 7fc131e00700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc124017c70 tx=0x7fc124004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7b58d8f81f7e24 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.148+0000 7fc117e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:25.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.149+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124027060 con 0x55e817129a50 2024-06-13T06:57:25.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.149+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc124015c70 con 0x55e817129a50 2024-06-13T06:57:25.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.149+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11c0043c0 con 0x55e817129a50 2024-06-13T06:57:25.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.149+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc124022400 con 0x55e817129a50 2024-06-13T06:57:25.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc1438c1b80 10 client.25486 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:25.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e817140e80 con 0x55e817129a50 2024-06-13T06:57:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c0043c0 con 0x55e817129a50 2024-06-13T06:57:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc1240214c0 con 0x55e817129a50 2024-06-13T06:57:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc1438c1b80 10 client.25486 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc1438c1b80 10 client.25486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.150+0000 7fc1438c1b80 20 client.25486 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.151+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e817143410 0x55e817145890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.151+0000 7fc1438c1b80 4 client.25486 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.151+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e817080000 con 0x55e817143410 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.151+0000 7fc12be00700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e817143410 0x55e817145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.152+0000 7fc12be00700 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e817143410 0x55e817145890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a449ba8fb7ca11b0 server_cookie=8a8403fe9836bcb1 in_seq=0 out_seq=0 2024-06-13T06:57:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.152+0000 7fc117e00700 10 client.25486 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc117e00700 1 -- 172.21.15.7:0/2653140744 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e817080000 con 0x55e817143410 2024-06-13T06:57:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc117e00700 10 client.25486 handle_command_reply: tid=0 2024-06-13T06:57:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc1438c1b80 1 client.25486 shutdown 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 msgr2=0x55e817135a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc124017c70 tx=0x7fc124004b40 comp rx=0 tx=0).stop 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 shutdown_connections 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.153+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e817143410 0x55e817145890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e817133240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e817135a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2653140744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e817132d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 >> 172.21.15.7:0/2653140744 conn(0x55e81704d170 msgr2=0x55e817115790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc117e00700 0 client.25486 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.154+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 shutdown_connections 2024-06-13T06:57:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.155+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2653140744 wait complete. 2024-06-13T06:57:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.155+0000 7fc1438c1b80 20 client.25486 trim_cache size 0 max 16384 2024-06-13T06:57:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.190+0000 7fc1438c1b80 1 Processor -- start 2024-06-13T06:57:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.190+0000 7fc1438c1b80 1 -- start start 2024-06-13T06:57:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e817129a50 0x55e817129510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e81712fcc0 0x55e817116130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341ae820 con 0x55e817129a50 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341ac080 con 0x55e81712f3a0 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341a9fa0 con 0x55e81712fcc0 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.191+0000 7fc12be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36836/0 (socket says 172.21.15.7:36836) 2024-06-13T06:57:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 -- 172.21.15.7:0/2550522362 learned_addr learned my addr 172.21.15.7:0/2550522362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 -- 172.21.15.7:0/2550522362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e81712fcc0 msgr2=0x55e817116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e81712fcc0 0x55e817116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 -- 172.21.15.7:0/2550522362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e817129a50 msgr2=0x55e817129510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e817129a50 0x55e817129510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.192+0000 7fc12be00700 1 -- 172.21.15.7:0/2550522362 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e817140c90 con 0x55e81712f3a0 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.193+0000 7fc12be00700 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005950 tx=0x7fc11c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e94b4b2af97ba2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.193+0000 7fc122a00700 1 -- 172.21.15.7:0/2550522362 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c02c070 con 0x55e81712f3a0 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.193+0000 7fc122a00700 1 -- 172.21.15.7:0/2550522362 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c00bb80 con 0x55e81712f3a0 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.193+0000 7fc122a00700 1 -- 172.21.15.7:0/2550522362 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0052a0 con 0x55e81712f3a0 2024-06-13T06:57:25.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2550522362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 msgr2=0x55e817131fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005950 tx=0x7fc11c015040 comp rx=0 tx=0).stop 2024-06-13T06:57:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2550522362 shutdown_connections 2024-06-13T06:57:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e81712fcc0 0x55e817116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e817129a50 0x55e817129510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2550522362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712f3a0 0x55e817131fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.195+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2550522362 >> 172.21.15.7:0/2550522362 conn(0x55e81704d170 msgr2=0x55e81712e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.196+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2550522362 shutdown_connections 2024-06-13T06:57:25.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.196+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2550522362 wait complete. 2024-06-13T06:57:25.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.196+0000 7fc1438c1b80 1 -- start start 2024-06-13T06:57:25.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 Processor -- start 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e81714b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e81714b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341aa540 con 0x55e81712f3a0 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341ab9b0 con 0x55e81712fcc0 2024-06-13T06:57:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc1341aac10 con 0x55e817129a50 2024-06-13T06:57:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.197+0000 7fc1438c1b80 10 client.0 _ll_register_callbacks cb 0x55e817008380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44332/0 (socket says 172.21.15.7:44332) 2024-06-13T06:57:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 -- 172.21.15.7:0/2324062401 learned_addr learned my addr 172.21.15.7:0/2324062401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc132800700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e81714b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 -- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 msgr2=0x55e81714b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e81714b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc131e00700 1 -- 172.21.15.7:0/2324062401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 msgr2=0x55e81714b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.198+0000 7fc12be00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e81714b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc131e00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e81714b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc131e00700 1 -- 172.21.15.7:0/2324062401 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8171be240 con 0x55e817129a50 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc12be00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e81714b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc132800700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e81714b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc131e00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc124005f10 tx=0x7fc124008500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=749c499caeda5e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.199+0000 7fc117e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.200+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124031080 con 0x55e817129a50 2024-06-13T06:57:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.200+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc124015a70 con 0x55e817129a50 2024-06-13T06:57:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.201+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc11c00bb80 con 0x55e817129a50 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.203+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc124027020 con 0x55e817129a50 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.203+0000 7fc1438c1b80 10 client.25489 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.203+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8171bf5c0 con 0x55e817129a50 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c00bb80 con 0x55e817129a50 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc124022c00 con 0x55e817129a50 2024-06-13T06:57:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 10 client.25489 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 20 client.25489 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8171bfd60 0x55e8171c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 4 client.25489 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e81707a580 con 0x55e8171bfd60 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.204+0000 7fc12be00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8171bfd60 0x55e8171c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.205+0000 7fc12be00700 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8171bfd60 0x55e8171c2180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bc9586c907a5f9 server_cookie=a2b084795ad2ea0b in_seq=0 out_seq=0 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.205+0000 7fc117e00700 10 client.25489 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc117e00700 1 -- 172.21.15.7:0/2324062401 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e81707a580 con 0x55e8171bfd60 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc117e00700 10 client.25489 handle_command_reply: tid=0 2024-06-13T06:57:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 client.25489 shutdown 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 msgr2=0x55e81714ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc124005f10 tx=0x7fc124008500 comp rx=0 tx=0).stop 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 shutdown_connections 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8171bfd60 0x55e8171c2180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e81712fcc0 0x55e81714b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e817129a50 0x55e81714ae70 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc124005f10 tx=0x7fc124008500 comp rx=0 tx=0).stop 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 --2- 172.21.15.7:0/2324062401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e81712f3a0 0x55e81714b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc117e00700 0 client.25489 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.206+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 >> 172.21.15.7:0/2324062401 conn(0x55e81704d170 msgr2=0x55e8171325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.207+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 shutdown_connections 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.207+0000 7fc1438c1b80 1 -- 172.21.15.7:0/2324062401 wait complete. 2024-06-13T06:57:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.208+0000 7fc1438c1b80 20 client.25489 trim_cache size 0 max 16384 2024-06-13T06:57:25.210 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:57:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T06:57:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T06:57:25.210 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 msgr2=0x7fc1180a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 0x7fc1180a51e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc1240060b0 tx=0x7fc12400c040 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 msgr2=0x7fc13419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0047c0 tx=0x7fc12c005ef0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 shutdown_connections 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc1180a2d20 0x7fc1180a51e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1341084e0 0x7fc13419f6f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc13406a170 0x7fc13419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 --2- 172.21.15.7:0/2855408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13406ab10 0x7fc13419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 >> 172.21.15.7:0/2855408342 conn(0x7fc1340782c0 msgr2=0x7fc13410d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 shutdown_connections 2024-06-13T06:57:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:25.210+0000 7fc139000700 1 -- 172.21.15.7:0/2855408342 wait complete. 2024-06-13T06:57:25.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:25 smithi189 ceph-mon[36616]: pgmap v550: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 771 op/s 2024-06-13T06:57:25.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:25 smithi175 ceph-mon[36886]: pgmap v550: 129 pgs: 129 active+clean; 1002 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 MiB/s wr, 771 op/s 2024-06-13T06:57:26.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T06:57:26.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:57:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:27.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:27 smithi175 ceph-mon[36886]: pgmap v551: 129 pgs: 129 active+clean; 1002 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 720 op/s 2024-06-13T06:57:27.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:27 smithi007 ceph-mon[32077]: pgmap v551: 129 pgs: 129 active+clean; 1002 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 720 op/s 2024-06-13T06:57:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:27 smithi189 ceph-mon[36616]: pgmap v551: 129 pgs: 129 active+clean; 1002 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 720 op/s 2024-06-13T06:57:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 85.0 s, 1752.2 tps, lat 57.255 ms stddev 67.014 2024-06-13T06:57:29.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:29 smithi007 ceph-mon[32077]: pgmap v552: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.5 MiB/s wr, 649 op/s 2024-06-13T06:57:29.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:29 smithi175 ceph-mon[36886]: pgmap v552: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.5 MiB/s wr, 649 op/s 2024-06-13T06:57:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:29 smithi189 ceph-mon[36616]: pgmap v552: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.5 MiB/s wr, 649 op/s 2024-06-13T06:57:30.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:30 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.381+0000 7f90bda00700 1 -- 172.21.15.7:0/2017042706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b81046a0 msgr2=0x7f90b8104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.381+0000 7f90bda00700 1 --2- 172.21.15.7:0/2017042706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b81046a0 0x7f90b8104b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009990 tx=0x7f90a8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.382+0000 7f90bda00700 1 -- 172.21.15.7:0/2017042706 shutdown_connections 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.382+0000 7f90bda00700 1 --2- 172.21.15.7:0/2017042706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b8105060 0x7f90b8111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.382+0000 7f90bda00700 1 --2- 172.21.15.7:0/2017042706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b810a6a0 0x7f90b810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.382+0000 7f90bda00700 1 --2- 172.21.15.7:0/2017042706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b81046a0 0x7f90b8104b20 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009990 tx=0x7f90a8009ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.382+0000 7f90bda00700 1 -- 172.21.15.7:0/2017042706 >> 172.21.15.7:0/2017042706 conn(0x7f90b81001d0 msgr2=0x7f90b81025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.384+0000 7f90bda00700 1 -- 172.21.15.7:0/2017042706 shutdown_connections 2024-06-13T06:57:30.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:30 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:57:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.384+0000 7f90bda00700 1 -- 172.21.15.7:0/2017042706 wait complete. 2024-06-13T06:57:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.386+0000 7f90bda00700 1 Processor -- start 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.386+0000 7f90bda00700 1 -- start start 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.387+0000 7f90bda00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 0x7f90b8073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.387+0000 7f90bda00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.387+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 0x7f90b8073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90bda00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 0x7f90b806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90bda00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8114a70 con 0x7f90b8105060 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90bda00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81148f0 con 0x7f90b806d3b0 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90bda00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8114bf0 con 0x7f90b810a6a0 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90b6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90b6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44348/0 (socket says 172.21.15.7:44348) 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90b6000700 1 -- 172.21.15.7:0/364636647 learned_addr learned my addr 172.21.15.7:0/364636647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.388+0000 7f90b7400700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 0x7f90b806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6a00700 1 -- 172.21.15.7:0/364636647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 msgr2=0x7f90b806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6a00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6a00700 1 -- 172.21.15.7:0/364636647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 msgr2=0x7f90b806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6a00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 0x7f90b806d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6a00700 1 -- 172.21.15.7:0/364636647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b80719c0 con 0x7f90b8105060 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b7400700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 0x7f90b806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.389+0000 7f90b6000700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.390+0000 7f90b6a00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 0x7f90b8073de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.390+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400cb70 con 0x7f90b8105060 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.391+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a400ee90 con 0x7f90b8105060 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.391+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4003dc0 con 0x7f90b8105060 2024-06-13T06:57:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.391+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b8071cb0 con 0x7f90b8105060 2024-06-13T06:57:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.391+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b80721a0 con 0x7f90b8105060 2024-06-13T06:57:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.393+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9098005330 con 0x7f90b8105060 2024-06-13T06:57:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.399+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f90a400cd10 con 0x7f90b8105060 2024-06-13T06:57:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.402+0000 7f90af400700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 0x7f909c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.402+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a40cf460 con 0x7f90b8105060 2024-06-13T06:57:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.403+0000 7f90b6000700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 0x7f909c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:30.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.404+0000 7f90b6000700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 0x7f909c0a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90b806e630 tx=0x7f90a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:30.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.406+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f90a409c020 con 0x7f90b8105060 2024-06-13T06:57:30.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:30 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:57:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.923+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9098005620 con 0x7f90b8105060 2024-06-13T06:57:30.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.925+0000 7f90af400700 1 -- 172.21.15.7:0/364636647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11872 (secure 0 0 0) 0x7f90a409b920 con 0x7f90b8105060 2024-06-13T06:57:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:57:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:56:05.053870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T06:57:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.929+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 msgr2=0x7f909c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.929+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 0x7f909c0a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90b806e630 tx=0x7f90a8005fb0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.929+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 msgr2=0x7f90b8073de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.929+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 0x7f90b8073de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400ed00 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 shutdown_connections 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2d70 0x7f909c0a5230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b806d3b0 0x7f90b806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8105060 0x7f90b8073de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 --2- 172.21.15.7:0/364636647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810a6a0 0x7f90b806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.930+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 >> 172.21.15.7:0/364636647 conn(0x7f90b81001d0 msgr2=0x7f90b8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.931+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 shutdown_connections 2024-06-13T06:57:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:30.931+0000 7f90bda00700 1 -- 172.21.15.7:0/364636647 wait complete. 2024-06-13T06:57:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-06-13T06:57:31.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:31 smithi007 ceph-mon[32077]: pgmap v553: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-06-13T06:57:31.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:31 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/364636647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:31.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:31.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:31.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:31 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:57:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:31 smithi189 ceph-mon[36616]: pgmap v553: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-06-13T06:57:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:31 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/364636647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:31 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:31 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:31 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:57:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:31 smithi175 ceph-mon[36886]: pgmap v553: 129 pgs: 129 active+clean; 1002 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 MiB/s wr, 592 op/s 2024-06-13T06:57:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:31 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/364636647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:57:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:57:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:57:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:31 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:57:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 90.0 s, 1480.6 tps, lat 67.234 ms stddev 80.492 2024-06-13T06:57:33.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:33 smithi007 ceph-mon[32077]: pgmap v554: 129 pgs: 129 active+clean; 1010 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-06-13T06:57:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:33 smithi189 ceph-mon[36616]: pgmap v554: 129 pgs: 129 active+clean; 1010 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-06-13T06:57:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:33 smithi175 ceph-mon[36886]: pgmap v554: 129 pgs: 129 active+clean; 1010 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 -- 172.21.15.7:0/1994187177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974105610 msgr2=0x7f597410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 --2- 172.21.15.7:0/1994187177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974105610 0x7f597410bea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f596c009ca0 tx=0x7f596c02a8c0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 -- 172.21.15.7:0/1994187177 shutdown_connections 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 --2- 172.21.15.7:0/1994187177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974105610 0x7f597410bea0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 --2- 172.21.15.7:0/1994187177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f59741050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 --2- 172.21.15.7:0/1994187177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974103a60 0x7f5974103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.680+0000 7f5978e00640 1 -- 172.21.15.7:0/1994187177 >> 172.21.15.7:0/1994187177 conn(0x7f59740ff010 msgr2=0x7f5974101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.681+0000 7f5978e00640 1 -- 172.21.15.7:0/1994187177 shutdown_connections 2024-06-13T06:57:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.681+0000 7f5978e00640 1 -- 172.21.15.7:0/1994187177 wait complete. 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.681+0000 7f5978e00640 1 Processor -- start 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 -- start start 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f597419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 0x7f59741a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f597410f730 con 0x7f5974103a60 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f597410f5b0 con 0x7f5974104c50 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5978e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f597410f8b0 con 0x7f5974105610 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f596aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f597419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5973400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5973e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 0x7f59741a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5973e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 0x7f59741a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37620/0 (socket says 172.21.15.7:37620) 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5973400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60922/0 (socket says 172.21.15.7:60922) 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f596aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f597419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59938/0 (socket says 172.21.15.7:59938) 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.682+0000 7f5973e00640 1 -- 172.21.15.7:0/1896395808 learned_addr learned my addr 172.21.15.7:0/1896395808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 -- 172.21.15.7:0/1896395808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 msgr2=0x7f59741a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 0x7f59741a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 -- 172.21.15.7:0/1896395808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 msgr2=0x7f597419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f597419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 -- 172.21.15.7:0/1896395808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59741a1920 con 0x7f5974103a60 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5973400640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9c0 tx=0x7f596400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5970c00640 1 -- 172.21.15.7:0/1896395808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400ccc0 con 0x7f5974103a60 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5970c00640 1 -- 172.21.15.7:0/1896395808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59640035f0 con 0x7f5974103a60 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.683+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59741a1c10 con 0x7f5974103a60 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.684+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741a2110 con 0x7f5974103a60 2024-06-13T06:57:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.684+0000 7f5970c00640 1 -- 172.21.15.7:0/1896395808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400ce60 con 0x7f5974103a60 2024-06-13T06:57:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.689+0000 7f5970c00640 1 -- 172.21.15.7:0/1896395808 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f59640037a0 con 0x7f5974103a60 2024-06-13T06:57:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 Processor -- start 2024-06-13T06:57:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 -- start start 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a6782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a67adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a5d6420 con 0x55cd6a678920 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a57dc50 con 0x55cd6a672dc0 2024-06-13T06:57:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.690+0000 7f597d391740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a5d23e0 con 0x55cd6a6737b0 2024-06-13T06:57:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59954/0 (socket says 172.21.15.7:59954) 2024-06-13T06:57:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 -- 172.21.15.7:0/1572763813 learned_addr learned my addr 172.21.15.7:0/1572763813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f596aa00640 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a67adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 -- 172.21.15.7:0/1572763813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 msgr2=0x55cd6a6782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a6782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 -- 172.21.15.7:0/1572763813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 msgr2=0x55cd6a67adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a67adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.691+0000 7f5973e00640 1 -- 172.21.15.7:0/1572763813 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6a2ea210 con 0x55cd6a672dc0 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.692+0000 7f596aa00640 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a67adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.692+0000 7f5973e00640 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f596c009430 tx=0x7f596c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97457982c7458f94 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.692+0000 7f5968c00640 1 -- 172.21.15.7:0/1572763813 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c031650 con 0x55cd6a672dc0 2024-06-13T06:57:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.693+0000 7f5968c00640 1 -- 172.21.15.7:0/1572763813 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596c036030 con 0x55cd6a672dc0 2024-06-13T06:57:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.693+0000 7f5968c00640 1 -- 172.21.15.7:0/1572763813 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c02f610 con 0x55cd6a672dc0 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.693+0000 7f5970c00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 0x7f59580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.693+0000 7f5970c00640 1 -- 172.21.15.7:0/1896395808 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f59640cf910 con 0x7f5974103a60 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 -- 172.21.15.7:0/1572763813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 msgr2=0x55cd6a6731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f596c009430 tx=0x7f596c005ef0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f5973400640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 0x7f59580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 -- 172.21.15.7:0/1572763813 shutdown_connections 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a67adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a6782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 --2- 172.21.15.7:0/1572763813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a672dc0 0x55cd6a6731e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 -- 172.21.15.7:0/1572763813 >> 172.21.15.7:0/1572763813 conn(0x55cd6a63afb0 msgr2=0x55cd6a66e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 -- 172.21.15.7:0/1572763813 shutdown_connections 2024-06-13T06:57:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.694+0000 7f597d391740 1 -- 172.21.15.7:0/1572763813 wait complete. 2024-06-13T06:57:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.695+0000 7f5973400640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 0x7f59580a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59640cd120 tx=0x7f59640ccfa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.695+0000 7f597d391740 1 -- start start 2024-06-13T06:57:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.695+0000 7f597d391740 1 Processor -- start 2024-06-13T06:57:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a685c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a68ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a67b6f0 con 0x55cd6a678920 2024-06-13T06:57:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a5910b0 con 0x55cd6a6737b0 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd6a67b870 con 0x55cd6a672dc0 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f597d391740 10 client.0 _ll_register_callbacks cb 0x55cd6a57cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f5973e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f5973e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37642/0 (socket says 172.21.15.7:37642) 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f5973e00640 1 -- 172.21.15.7:0/3625607483 learned_addr learned my addr 172.21.15.7:0/3625607483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.696+0000 7f5973e00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a68ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 -- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 msgr2=0x55cd6a685c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a685c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 -- 172.21.15.7:0/3625607483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 msgr2=0x55cd6a68ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a68ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 -- 172.21.15.7:0/3625607483 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6a3b6620 con 0x55cd6a672dc0 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5973e00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f596c02ade0 tx=0x7f596c009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b119f19278208eb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5961600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.697+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c004d50 con 0x55cd6a672dc0 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596c036030 con 0x55cd6a672dc0 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f596c003970 con 0x55cd6a672dc0 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f596c05e050 con 0x55cd6a672dc0 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f597d391740 10 client.25504 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd6a67b6f0 con 0x55cd6a672dc0 2024-06-13T06:57:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.698+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596c036030 con 0x55cd6a672dc0 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f596c0305c0 con 0x55cd6a672dc0 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 20 client.25504 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a680f40 0x55cd6a683380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 4 client.25504 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.699+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd6a5e3b50 con 0x55cd6a680f40 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.700+0000 7f596aa00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a680f40 0x55cd6a683380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.700+0000 7f596aa00640 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a680f40 0x55cd6a683380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f74188b402045ce server_cookie=10ed21613ca0ddb3 in_seq=0 out_seq=0 2024-06-13T06:57:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.700+0000 7f5961600640 10 client.25504 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.702+0000 7f5961600640 1 -- 172.21.15.7:0/3625607483 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd6a5e3b50 con 0x55cd6a680f40 2024-06-13T06:57:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.703+0000 7f5961600640 10 client.25504 handle_command_reply: tid=0 2024-06-13T06:57:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.704+0000 7f597d391740 1 client.25504 shutdown 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.705+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 msgr2=0x55cd6a685750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.705+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f596c02ade0 tx=0x7f596c009360 comp rx=0 tx=0).stop 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 shutdown_connections 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a680f40 0x55cd6a683380 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a678920 0x55cd6a68ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a685c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 --2- 172.21.15.7:0/3625607483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a672dc0 0x55cd6a685750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f5961600640 0 client.25504 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.706+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 >> 172.21.15.7:0/3625607483 conn(0x55cd6a63afb0 msgr2=0x55cd6a66d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.707+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 shutdown_connections 2024-06-13T06:57:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.707+0000 7f597d391740 1 -- 172.21.15.7:0/3625607483 wait complete. 2024-06-13T06:57:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.708+0000 7f597d391740 20 client.25504 trim_cache size 0 max 16384 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 Processor -- start 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 -- start start 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a67d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a678920 0x55cd6a671e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59741aa930 con 0x55cd6a672dc0 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a81d0 con 0x55cd6a6737b0 2024-06-13T06:57:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.763+0000 7f597d391740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a6020 con 0x55cd6a678920 2024-06-13T06:57:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.764+0000 7f596aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.764+0000 7f596aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59970/0 (socket says 172.21.15.7:59970) 2024-06-13T06:57:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.764+0000 7f596aa00640 1 -- 172.21.15.7:0/288884846 learned_addr learned my addr 172.21.15.7:0/288884846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.764+0000 7f5973e00640 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a67d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.765+0000 7f5973e00640 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a678920 0x55cd6a671e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.766+0000 7f596aa00640 1 -- 172.21.15.7:0/288884846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a678920 msgr2=0x55cd6a671e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.766+0000 7f596aa00640 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a678920 0x55cd6a671e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.766+0000 7f596aa00640 1 -- 172.21.15.7:0/288884846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 msgr2=0x55cd6a67d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.766+0000 7f596aa00640 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a67d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.766+0000 7f596aa00640 1 -- 172.21.15.7:0/288884846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6a686740 con 0x55cd6a6737b0 2024-06-13T06:57:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.767+0000 7f596aa00640 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f595c009b70 tx=0x7f595c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d85393ed4f9d1d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.768+0000 7f5963e00640 1 -- 172.21.15.7:0/288884846 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c01c620 con 0x55cd6a6737b0 2024-06-13T06:57:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.768+0000 7f5963e00640 1 -- 172.21.15.7:0/288884846 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c01cca0 con 0x55cd6a6737b0 2024-06-13T06:57:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.768+0000 7f5963e00640 1 -- 172.21.15.7:0/288884846 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c01bc20 con 0x55cd6a6737b0 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 -- 172.21.15.7:0/288884846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 msgr2=0x55cd6a68cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f595c009b70 tx=0x7f595c010800 comp rx=0 tx=0).stop 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 -- 172.21.15.7:0/288884846 shutdown_connections 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a678920 0x55cd6a671e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a6737b0 0x55cd6a68cc90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 --2- 172.21.15.7:0/288884846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a67d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.769+0000 7f597d391740 1 -- 172.21.15.7:0/288884846 >> 172.21.15.7:0/288884846 conn(0x55cd6a63afb0 msgr2=0x55cd6a671730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.771+0000 7f597d391740 1 -- 172.21.15.7:0/288884846 shutdown_connections 2024-06-13T06:57:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.771+0000 7f597d391740 1 -- 172.21.15.7:0/288884846 wait complete. 2024-06-13T06:57:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.771+0000 7f597d391740 1 -- start start 2024-06-13T06:57:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 Processor -- start 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a68a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a68aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a6690 con 0x55cd6a672dc0 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a7b00 con 0x55cd6a678920 2024-06-13T06:57:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59741a6d60 con 0x55cd6a6737b0 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.772+0000 7f597d391740 10 client.0 _ll_register_callbacks cb 0x7f596c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f5973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a68a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f596aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a68aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f596aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a68aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37648/0 (socket says 172.21.15.7:37648) 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f5973e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59986/0 (socket says 172.21.15.7:59986) 2024-06-13T06:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.773+0000 7f596aa00640 1 -- 172.21.15.7:0/1203918094 learned_addr learned my addr 172.21.15.7:0/1203918094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 -- 172.21.15.7:0/1203918094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 msgr2=0x55cd6a68aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a68aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 -- 172.21.15.7:0/1203918094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 msgr2=0x55cd6a68a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a68a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 -- 172.21.15.7:0/1203918094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6a68c4a0 con 0x55cd6a678920 2024-06-13T06:57:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.774+0000 7f5973e00640 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f596c009430 tx=0x7f596c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901fc221e987e335 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.775+0000 7f5961600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:57:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.777+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c030d70 con 0x55cd6a678920 2024-06-13T06:57:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.777+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f596c053040 con 0x55cd6a678920 2024-06-13T06:57:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.778+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c01cca0 con 0x55cd6a678920 2024-06-13T06:57:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.778+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f596c030d70 con 0x55cd6a678920 2024-06-13T06:57:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.779+0000 7f597d391740 10 client.25517 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.779+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd6a7047b0 con 0x55cd6a678920 2024-06-13T06:57:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.779+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f595c01cca0 con 0x55cd6a678920 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f596c02f610 con 0x55cd6a678920 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 10 client.25517 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 20 client.25517 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a705090 0x55cd6a7074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 4 client.25517 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:57:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.780+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd6a5ddea0 con 0x55cd6a705090 2024-06-13T06:57:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.781+0000 7f596aa00640 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a705090 0x55cd6a7074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.785+0000 7f596aa00640 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a705090 0x55cd6a7074b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a4900585448743b server_cookie=d54746f61fedd05e in_seq=0 out_seq=0 2024-06-13T06:57:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.785+0000 7f5961600640 10 client.25517 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.844+0000 7f5961600640 1 -- 172.21.15.7:0/1203918094 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cd6a5ddea0 con 0x55cd6a705090 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.844+0000 7f5961600640 10 client.25517 handle_command_reply: tid=0 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.844+0000 7f597d391740 1 client.25517 shutdown 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.844+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 msgr2=0x55cd6a68beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.844+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f596c009430 tx=0x7f596c02a8c0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 shutdown_connections 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55cd6a705090 0x55cd6a7074b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cd6a678920 0x55cd6a68beb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd6a6737b0 0x55cd6a68aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 --2- 172.21.15.7:0/1203918094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cd6a672dc0 0x55cd6a68a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 >> 172.21.15.7:0/1203918094 conn(0x55cd6a63afb0 msgr2=0x55cd6a66d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f5961600640 0 client.25517 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.845+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 shutdown_connections 2024-06-13T06:57:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.846+0000 7f597d391740 1 -- 172.21.15.7:0/1203918094 wait complete. 2024-06-13T06:57:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.846+0000 7f597d391740 20 client.25517 trim_cache size 0 max 16384 2024-06-13T06:57:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:57:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:57:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.847+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 msgr2=0x7f59580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.847+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 0x7f59580a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59640cd120 tx=0x7f59640ccfa0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.847+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 msgr2=0x7f5974199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.847+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9c0 tx=0x7f596400ee80 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 shutdown_connections 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59580a2b80 0x7f59580a5040 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59640cd120 tx=0x7f59640ccfa0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5974105610 0x7f59741a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5974104c50 0x7f597419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 --2- 172.21.15.7:0/1896395808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5974103a60 0x7f5974199d10 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f596400e9c0 tx=0x7f596400ee80 comp rx=0 tx=0).stop 2024-06-13T06:57:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.848+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 >> 172.21.15.7:0/1896395808 conn(0x7f59740ff010 msgr2=0x7f5974107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.851+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 shutdown_connections 2024-06-13T06:57:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:34.851+0000 7f5978e00640 1 -- 172.21.15.7:0/1896395808 wait complete. 2024-06-13T06:57:35.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:35 smithi007 ceph-mon[32077]: pgmap v555: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 652 op/s 2024-06-13T06:57:35.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T06:57:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:35 smithi189 ceph-mon[36616]: pgmap v555: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 652 op/s 2024-06-13T06:57:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:35 smithi175 ceph-mon[36886]: pgmap v555: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 MiB/s wr, 652 op/s 2024-06-13T06:57:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:37.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:37 smithi007 ceph-mon[32077]: pgmap v556: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-06-13T06:57:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:37 smithi189 ceph-mon[36616]: pgmap v556: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-06-13T06:57:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:37 smithi175 ceph-mon[36886]: pgmap v556: 129 pgs: 129 active+clean; 1018 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-06-13T06:57:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 95.0 s, 1411.9 tps, lat 71.704 ms stddev 139.207 2024-06-13T06:57:39.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:39 smithi007 ceph-mon[32077]: pgmap v557: 129 pgs: 129 active+clean; 1018 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.044+0000 7fecb9600700 1 -- 172.21.15.7:0/2481770932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb41121a0 msgr2=0x7fecb4119d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.044+0000 7fecb9600700 1 --2- 172.21.15.7:0/2481770932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb41121a0 0x7fecb4119d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fecac009f90 tx=0x7fecac00b330 comp rx=0 tx=0).stop 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 -- 172.21.15.7:0/2481770932 shutdown_connections 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 --2- 172.21.15.7:0/2481770932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb41121a0 0x7fecb4119d70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 --2- 172.21.15.7:0/2481770932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb410d0c0 0x7fecb410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 --2- 172.21.15.7:0/2481770932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecb410da70 0x7fecb4111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 -- 172.21.15.7:0/2481770932 >> 172.21.15.7:0/2481770932 conn(0x7fecb406d680 msgr2=0x7fecb410b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 -- 172.21.15.7:0/2481770932 shutdown_connections 2024-06-13T06:57:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.045+0000 7fecb9600700 1 -- 172.21.15.7:0/2481770932 wait complete. 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.047+0000 7fecb9600700 1 Processor -- start 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.048+0000 7fecb9600700 1 -- start start 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.048+0000 7fecb9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 0x7fecb4119bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.048+0000 7fecb9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecb410da70 0x7fecb4116cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.048+0000 7fecb9600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb9600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecb411d1e0 con 0x7fecb410da70 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb9600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecb411d060 con 0x7fecb410d0c0 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb9600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fecb411d360 con 0x7fecb41121a0 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 0x7fecb4119bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37656/0 (socket says 172.21.15.7:37656) 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 0x7fecb4119bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60004/0 (socket says 172.21.15.7:60004) 2024-06-13T06:57:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.049+0000 7fecb3400700 1 -- 172.21.15.7:0/760312439 learned_addr learned my addr 172.21.15.7:0/760312439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 -- 172.21.15.7:0/760312439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 msgr2=0x7fecb4119bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 0x7fecb4119bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 -- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecb410da70 msgr2=0x7fecb4116cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecb410da70 0x7fecb4116cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 -- 172.21.15.7:0/760312439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb4117aa0 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.050+0000 7fecb3400700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fecac00d010 tx=0x7fecac009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.052+0000 7fecab400700 1 -- 172.21.15.7:0/760312439 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac021070 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.052+0000 7fecab400700 1 -- 172.21.15.7:0/760312439 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecac01d450 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.052+0000 7fecab400700 1 -- 172.21.15.7:0/760312439 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac005260 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.053+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb41b3fe0 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.053+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb41b4440 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.063+0000 7fecc3ec3b80 1 Processor -- start 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.063+0000 7fecab400700 1 -- 172.21.15.7:0/760312439 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fecac024070 con 0x7fecb41121a0 2024-06-13T06:57:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.063+0000 7fecc3ec3b80 1 -- start start 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.064+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b0b90 0x55809e1b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1ab240 0x55809e1ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55809e0b9a10 con 0x55809e1b0b90 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55809e0fcb00 con 0x55809e1b14b0 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55809e0b78f0 con 0x55809e1ab240 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecb2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b0b90 0x55809e1b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60016/0 (socket says 172.21.15.7:60016) 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.065+0000 7fecb2a00700 1 -- 172.21.15.7:0/210061438 learned_addr learned my addr 172.21.15.7:0/210061438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.066+0000 7fecb2000700 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1ab240 0x55809e1ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.066+0000 7fecb2a00700 1 -- 172.21.15.7:0/210061438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1ab240 msgr2=0x55809e1ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.066+0000 7fecb2a00700 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1ab240 0x55809e1ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.066+0000 7fecb2a00700 1 -- 172.21.15.7:0/210061438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b0b90 msgr2=0x55809e1b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.067+0000 7fecb2a00700 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b0b90 0x55809e1b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.067+0000 7fecb2a00700 1 -- 172.21.15.7:0/210061438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55809e0a4f90 con 0x55809e1b14b0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.067+0000 7fecab400700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 0x7fec980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.067+0000 7fecab400700 1 -- 172.21.15.7:0/760312439 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fecac00f020 con 0x7fecb41121a0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.068+0000 7fecb2a00700 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feca4009d60 tx=0x7feca4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80edce1cf5387987 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.069+0000 7fecb2000700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 0x7fec980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.069+0000 7fec9fe00700 1 -- 172.21.15.7:0/210061438 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca401e070 con 0x55809e1b14b0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.069+0000 7fec9fe00700 1 -- 172.21.15.7:0/210061438 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca4019b10 con 0x55809e1b14b0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.069+0000 7fec9fe00700 1 -- 172.21.15.7:0/210061438 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca400eba0 con 0x55809e1b14b0 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.070+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/210061438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 msgr2=0x55809e1aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.070+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feca4009d60 tx=0x7feca4011720 comp rx=0 tx=0).stop 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.071+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/210061438 shutdown_connections 2024-06-13T06:57:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.071+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1ab240 0x55809e1ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.071+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b0b90 0x55809e1b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.071+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/210061438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b14b0 0x55809e1aad00 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feca4009d60 tx=0x7feca4011720 comp rx=0 tx=0).stop 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.071+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/210061438 >> 172.21.15.7:0/210061438 conn(0x55809e0d7170 msgr2=0x55809e1a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.072+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/210061438 shutdown_connections 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.072+0000 7fecb2000700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 0x7fec980a5160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feca0005950 tx=0x7feca0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.072+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/210061438 wait complete. 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.073+0000 7fecc3ec3b80 1 -- start start 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.073+0000 7fecc3ec3b80 1 Processor -- start 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.075+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 0x55809e1b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.075+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.075+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 0x55809e1b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 0x55809e1b7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60018/0 (socket says 172.21.15.7:60018) 2024-06-13T06:57:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecb2000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecb2000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37686/0 (socket says 172.21.15.7:37686) 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55809e098f60 con 0x55809e1b4cf0 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55809e0f8ac0 con 0x55809e1ab240 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55809e1b7e90 con 0x55809e1b0b90 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.076+0000 7fecc3ec3b80 10 client.0 _ll_register_callbacks cb 0x55809e092380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.077+0000 7fecb3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.077+0000 7fecb3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32780/0 (socket says 172.21.15.7:32780) 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.077+0000 7fecb3400700 1 -- 172.21.15.7:0/1465475688 learned_addr learned my addr 172.21.15.7:0/1465475688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb2000700 1 -- 172.21.15.7:0/1465475688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 msgr2=0x55809e1b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb2000700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 0x55809e1b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb2000700 1 -- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 msgr2=0x55809e1b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb2000700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb2000700 1 -- 172.21.15.7:0/1465475688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55809e04b8d0 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.078+0000 7fecb3400700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.079+0000 7fec9cc00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.079+0000 7fecb2000700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feca00046e0 tx=0x7feca0005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad04b4376b9fb3a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.080+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0018070 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.080+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7feca00128a0 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.081+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca4019b10 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.082+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feca0018070 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.082+0000 7fecc3ec3b80 10 client.25519 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.082+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55809e1c2ca0 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca4019b10 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7feca001d420 con 0x55809e1b0b90 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 10 client.25519 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 10 client.25519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 20 client.25519 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e1c5710 0x55809e1c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 4 client.25519 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.083+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55809e10a000 con 0x55809e1c5710 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.087+0000 7fecb2a00700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e1c5710 0x55809e1c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.088+0000 7fecb2a00700 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e1c5710 0x55809e1c7b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec4b35944a2bfd4b server_cookie=aa8b1bdd7b7b22a7 in_seq=0 out_seq=0 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.088+0000 7fec9cc00700 10 client.25519 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.090+0000 7fec9cc00700 1 -- 172.21.15.7:0/1465475688 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55809e10a000 con 0x55809e1c5710 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.090+0000 7fec9cc00700 10 client.25519 handle_command_reply: tid=0 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.090+0000 7fecc3ec3b80 1 client.25519 shutdown 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 msgr2=0x55809e1b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feca00046e0 tx=0x7feca0005ee0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 shutdown_connections 2024-06-13T06:57:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1b4cf0 0x55809e1b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e1c5710 0x55809e1c7b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1ab240 0x55809e1b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/1465475688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1b0b90 0x55809e1b4760 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feca00046e0 tx=0x7feca0005ee0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 >> 172.21.15.7:0/1465475688 conn(0x55809e0d7170 msgr2=0x55809e1a0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.092+0000 7fec9cc00700 0 client.25519 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.093+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 shutdown_connections 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.093+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/1465475688 wait complete. 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.093+0000 7fecc3ec3b80 20 client.25519 trim_cache size 0 max 16384 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.131+0000 7fecc3ec3b80 1 Processor -- start 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.131+0000 7fecc3ec3b80 1 -- start start 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1aff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41bf850 con 0x55809e1ab240 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41bd0b0 con 0x55809e1b4cf0 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41baf00 con 0x55809e1a6320 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.132+0000 7fecb2000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.133+0000 7fecb2000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60024/0 (socket says 172.21.15.7:60024) 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.133+0000 7fecb2000700 1 -- 172.21.15.7:0/3960335170 learned_addr learned my addr 172.21.15.7:0/3960335170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.133+0000 7fecb2a00700 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.133+0000 7fecb3400700 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.134+0000 7fecb2a00700 1 -- 172.21.15.7:0/3960335170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 msgr2=0x55809e1aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.134+0000 7fecb2a00700 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.134+0000 7fecb2a00700 1 -- 172.21.15.7:0/3960335170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 msgr2=0x55809e1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.134+0000 7fecb2a00700 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.134+0000 7fecb2a00700 1 -- 172.21.15.7:0/3960335170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55809e1b2cd0 con 0x55809e1ab240 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.135+0000 7fecb2a00700 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1aad00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feca4008000 tx=0x7feca400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bcb20c77a8871f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.135+0000 7fec9f400700 1 -- 172.21.15.7:0/3960335170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca401e040 con 0x55809e1ab240 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.135+0000 7fec9f400700 1 -- 172.21.15.7:0/3960335170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca400e4d0 con 0x55809e1ab240 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.135+0000 7fec9f400700 1 -- 172.21.15.7:0/3960335170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca401e040 con 0x55809e1ab240 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.136+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/3960335170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 msgr2=0x55809e1aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.136+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1aad00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feca4008000 tx=0x7feca400e720 comp rx=0 tx=0).stop 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.137+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/3960335170 shutdown_connections 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.137+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.137+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1aad00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.137+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/3960335170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.137+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/3960335170 >> 172.21.15.7:0/3960335170 conn(0x55809e0d7170 msgr2=0x55809e1af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.138+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/3960335170 shutdown_connections 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.138+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/3960335170 wait complete. 2024-06-13T06:57:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.139+0000 7fecc3ec3b80 1 -- start start 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.139+0000 7fecc3ec3b80 1 Processor -- start 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.142+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.142+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecc3ec3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.142+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecb2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37702/0 (socket says 172.21.15.7:37702) 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41bb570 con 0x55809e1ab240 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41bc9e0 con 0x55809e1b4cf0 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecc3ec3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41bbc40 con 0x55809e1a6320 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecc3ec3b80 10 client.0 _ll_register_callbacks cb 0x55809e092380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecb2a00700 1 -- 172.21.15.7:0/82649204 learned_addr learned my addr 172.21.15.7:0/82649204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.143+0000 7fecb3400700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.144+0000 7fecb2000700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2000700 1 -- 172.21.15.7:0/82649204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 msgr2=0x55809e1bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2000700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2000700 1 -- 172.21.15.7:0/82649204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 msgr2=0x55809e1cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2000700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2000700 1 -- 172.21.15.7:0/82649204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55809e1ccf30 con 0x55809e1ab240 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb2a00700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.145+0000 7fecb3400700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.146+0000 7fec9cc00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.146+0000 7fecb2000700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1bd870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feca000cf50 tx=0x7feca001db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cacf6dccc6483c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.147+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0039070 con 0x55809e1ab240 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.148+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca400e4d0 con 0x55809e1ab240 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.148+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7feca0004830 con 0x55809e1ab240 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.148+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feca0039070 con 0x55809e1ab240 2024-06-13T06:57:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.148+0000 7fecc3ec3b80 10 client.15318 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.148+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55809e22fe10 con 0x55809e1ab240 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.151+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca400e4d0 con 0x55809e1ab240 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.152+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7feca003c030 con 0x55809e1ab240 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.152+0000 7fecc3ec3b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.152+0000 7fecc3ec3b80 10 client.15318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.152+0000 7fecc3ec3b80 20 client.15318 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.153+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e230570 0x55809e232990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.153+0000 7fecc3ec3b80 4 client.15318 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.153+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55809e104580 con 0x55809e230570 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.154+0000 7fecb2a00700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e230570 0x55809e232990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.155+0000 7fecb2a00700 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e230570 0x55809e232990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bca78460413ea93 server_cookie=45b5b23b20bb0428 in_seq=0 out_seq=0 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.155+0000 7fec9cc00700 10 client.15318 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.156+0000 7fec9cc00700 1 -- 172.21.15.7:0/82649204 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55809e104580 con 0x55809e230570 2024-06-13T06:57:39.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.156+0000 7fec9cc00700 10 client.15318 handle_command_reply: tid=0 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.156+0000 7fecc3ec3b80 1 client.15318 shutdown 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.156+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 msgr2=0x55809e1bd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.156+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1bd870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feca000cf50 tx=0x7feca001db60 comp rx=0 tx=0).stop 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 shutdown_connections 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55809e230570 0x55809e232990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55809e1b4cf0 0x55809e1cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55809e1a6320 0x55809e1bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 --2- 172.21.15.7:0/82649204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55809e1ab240 0x55809e1bd870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fec9cc00700 0 client.15318 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 >> 172.21.15.7:0/82649204 conn(0x55809e0d7170 msgr2=0x55809e1b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.157+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 shutdown_connections 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.158+0000 7fecc3ec3b80 1 -- 172.21.15.7:0/82649204 wait complete. 2024-06-13T06:57:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.158+0000 7fecc3ec3b80 20 client.15318 trim_cache size 0 max 16384 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.159+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 msgr2=0x7fec980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.159+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 0x7fec980a5160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feca0005950 tx=0x7feca0005f50 comp rx=0 tx=0).stop 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.159+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 msgr2=0x7fecb41171f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.159+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fecac00d010 tx=0x7fecac009510 comp rx=0 tx=0).stop 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 shutdown_connections 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb41121a0 0x7fecb41171f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fec980a2ca0 0x7fec980a5160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fecb410d0c0 0x7fecb4119bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 --2- 172.21.15.7:0/760312439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecb410da70 0x7fecb4116cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:39.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 >> 172.21.15.7:0/760312439 conn(0x7fecb406d680 msgr2=0x7fecb41118f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:39.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.160+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 shutdown_connections 2024-06-13T06:57:39.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:39.161+0000 7fecb9600700 1 -- 172.21.15.7:0/760312439 wait complete. 2024-06-13T06:57:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:39 smithi189 ceph-mon[36616]: pgmap v557: 129 pgs: 129 active+clean; 1018 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-06-13T06:57:39.616 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:39 smithi175 ceph-mon[36886]: pgmap v557: 129 pgs: 129 active+clean; 1018 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-06-13T06:57:41.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T06:57:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:41 smithi189 ceph-mon[36616]: pgmap v558: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 853 op/s 2024-06-13T06:57:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:41 smithi175 ceph-mon[36886]: pgmap v558: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 853 op/s 2024-06-13T06:57:41.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:41 smithi007 ceph-mon[32077]: pgmap v558: 129 pgs: 129 active+clean; 1019 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 853 op/s 2024-06-13T06:57:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 100.0 s, 1910.7 tps, lat 51.373 ms stddev 61.034 2024-06-13T06:57:43.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:43 smithi007 ceph-mon[32077]: pgmap v559: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-06-13T06:57:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:43 smithi189 ceph-mon[36616]: pgmap v559: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-06-13T06:57:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:43 smithi175 ceph-mon[36886]: pgmap v559: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 960 op/s 2024-06-13T06:57:45.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:45 smithi007 ceph-mon[32077]: pgmap v560: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-06-13T06:57:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:45 smithi189 ceph-mon[36616]: pgmap v560: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-06-13T06:57:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:45 smithi175 ceph-mon[36886]: pgmap v560: 129 pgs: 129 active+clean; 1020 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 -- 172.21.15.7:0/920937385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803006a170 msgr2=0x7f803006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 --2- 172.21.15.7:0/920937385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803006a170 0x7f803006a5f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f801c0098f0 tx=0x7f801c009c00 comp rx=0 tx=0).stop 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 -- 172.21.15.7:0/920937385 shutdown_connections 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 --2- 172.21.15.7:0/920937385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006ab30 0x7f803010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 --2- 172.21.15.7:0/920937385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8030108500 0x7f80301088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 --2- 172.21.15.7:0/920937385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803006a170 0x7f803006a5f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.877+0000 7f8034c00700 1 -- 172.21.15.7:0/920937385 >> 172.21.15.7:0/920937385 conn(0x7f8030078340 msgr2=0x7f8030078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.878+0000 7f8034c00700 1 -- 172.21.15.7:0/920937385 shutdown_connections 2024-06-13T06:57:45.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.878+0000 7f8034c00700 1 -- 172.21.15.7:0/920937385 wait complete. 2024-06-13T06:57:45.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.879+0000 7f8034c00700 1 Processor -- start 2024-06-13T06:57:45.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.879+0000 7f8034c00700 1 -- start start 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006a170 0x7f803019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f803006ab30 0x7f803019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80301127b0 con 0x7f803006a170 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8030112630 con 0x7f803006ab30 2024-06-13T06:57:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.880+0000 7f8034c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8030112930 con 0x7f8030108500 2024-06-13T06:57:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.881+0000 7f802e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.881+0000 7f802e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35442/0 (socket says 172.21.15.7:35442) 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.881+0000 7f802e800700 1 -- 172.21.15.7:0/4205466926 learned_addr learned my addr 172.21.15.7:0/4205466926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.881+0000 7f802d400700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f803006ab30 0x7f803019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.881+0000 7f802de00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006a170 0x7f803019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.882+0000 7f802e800700 1 -- 172.21.15.7:0/4205466926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f803006ab30 msgr2=0x7f803019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.882+0000 7f802e800700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f803006ab30 0x7f803019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.882+0000 7f802e800700 1 -- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006a170 msgr2=0x7f803019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.882+0000 7f802e800700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006a170 0x7f803019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.882+0000 7f802e800700 1 -- 172.21.15.7:0/4205466926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8030199030 con 0x7f8030108500 2024-06-13T06:57:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.883+0000 7f802e800700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f802800ef30 tx=0x7f802800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.883+0000 7f8026a00700 1 -- 172.21.15.7:0/4205466926 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028004bb0 con 0x7f8030108500 2024-06-13T06:57:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.883+0000 7f8026a00700 1 -- 172.21.15.7:0/4205466926 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8028004d50 con 0x7f8030108500 2024-06-13T06:57:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.883+0000 7f8026a00700 1 -- 172.21.15.7:0/4205466926 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028004390 con 0x7f8030108500 2024-06-13T06:57:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.884+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8030199320 con 0x7f8030108500 2024-06-13T06:57:45.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.885+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8030101390 con 0x7f8030108500 2024-06-13T06:57:45.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.891+0000 7f8026a00700 1 -- 172.21.15.7:0/4205466926 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f802800c9f0 con 0x7f8030108500 2024-06-13T06:57:45.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.892+0000 7f803f5aeb80 1 Processor -- start 2024-06-13T06:57:45.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.893+0000 7f803f5aeb80 1 -- start start 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.893+0000 7f8026a00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 0x7f80140a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.893+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.894+0000 7f8026a00700 1 -- 172.21.15.7:0/4205466926 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80280d3020 con 0x7f8030108500 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.894+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9bf9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.894+0000 7f802de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.894+0000 7f802de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45750/0 (socket says 172.21.15.7:45750) 2024-06-13T06:57:45.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.894+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f802de00700 1 -- 172.21.15.7:0/1749309563 learned_addr learned my addr 172.21.15.7:0/1749309563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:45.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f802d400700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 0x7f80140a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602b9b08a10 con 0x5602b9c0cb10 2024-06-13T06:57:45.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f802d400700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9bf9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5602b9b4bb00 con 0x5602b9b75e40 2024-06-13T06:57:45.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.895+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602b9b068f0 con 0x5602b9bf95b0 2024-06-13T06:57:45.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802e800700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802de00700 1 -- 172.21.15.7:0/1749309563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 msgr2=0x5602b9bf9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802de00700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9bf9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802de00700 1 -- 172.21.15.7:0/1749309563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 msgr2=0x5602b9c0ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802de00700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c0ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.896+0000 7f802de00700 1 -- 172.21.15.7:0/1749309563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b9af3f90 con 0x5602b9b75e40 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.897+0000 7f802e800700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c0ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.897+0000 7f802de00700 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8020009d00 tx=0x7f80200116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a310b2f5ae6a0feb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.897+0000 7f802d400700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 0x7f80140a51e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5602b9bf6920 tx=0x7f801c009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.897+0000 7f801b400700 1 -- 172.21.15.7:0/1749309563 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020011e20 con 0x5602b9b75e40 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.898+0000 7f801b400700 1 -- 172.21.15.7:0/1749309563 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8020009e50 con 0x5602b9b75e40 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.898+0000 7f801b400700 1 -- 172.21.15.7:0/1749309563 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000f830 con 0x5602b9b75e40 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.898+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 msgr2=0x5602b9bf9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.898+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8020009d00 tx=0x7f80200116c0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 shutdown_connections 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c0ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9b75e40 0x5602b9bf9070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1749309563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9bf9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 >> 172.21.15.7:0/1749309563 conn(0x5602b9b26170 msgr2=0x5602b9bf54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 shutdown_connections 2024-06-13T06:57:45.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.899+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1749309563 wait complete. 2024-06-13T06:57:45.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.900+0000 7f803f5aeb80 1 -- start start 2024-06-13T06:57:45.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 Processor -- start 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9c01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9bff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602b9ae7f60 con 0x5602b9bf95b0 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5602b9b47ac0 con 0x5602b9bff4c0 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602b9c0f7d0 con 0x5602b9c0cb10 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.901+0000 7f803f5aeb80 10 client.0 _ll_register_callbacks cb 0x5602b9ae5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.902+0000 7f802d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.902+0000 7f802d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35458/0 (socket says 172.21.15.7:35458) 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.902+0000 7f802d400700 1 -- 172.21.15.7:0/2606749779 learned_addr learned my addr 172.21.15.7:0/2606749779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.902+0000 7f802e800700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9bff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 -- 172.21.15.7:0/2606749779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 msgr2=0x5602b9bff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9bff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 -- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 msgr2=0x5602b9c01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9c01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 -- 172.21.15.7:0/2606749779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b9a9a8d0 con 0x5602b9c0cb10 2024-06-13T06:57:45.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.903+0000 7f802d400700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f801c016000 tx=0x7f801c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9373182a84cd1fc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.904+0000 7f800fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:45.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.904+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c003a90 con 0x5602b9c0cb10 2024-06-13T06:57:45.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.904+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8020009e50 con 0x5602b9c0cb10 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.904+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f801c003890 con 0x5602b9c0cb10 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.905+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f801c003a90 con 0x5602b9c0cb10 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.905+0000 7f803f5aeb80 10 client.25537 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.905+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602b9c1a4c0 con 0x5602b9c0cb10 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.906+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8020009e50 con 0x5602b9c0cb10 2024-06-13T06:57:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f801c030020 con 0x5602b9c0cb10 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 10 client.25537 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 10 client.25537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 20 client.25537 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c1ca50 0x5602b9c1ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 4 client.25537 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602b9b59000 con 0x5602b9c1ca50 2024-06-13T06:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.907+0000 7f802de00700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c1ca50 0x5602b9c1ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.908+0000 7f802de00700 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c1ca50 0x5602b9c1ee40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cf85bc916c41b1f server_cookie=48f3d3d15be07ed7 in_seq=0 out_seq=0 2024-06-13T06:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.908+0000 7f800fe00700 10 client.25537 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:57:45.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f800fe00700 1 -- 172.21.15.7:0/2606749779 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5602b9b59000 con 0x5602b9c1ca50 2024-06-13T06:57:45.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f800fe00700 10 client.25537 handle_command_reply: tid=0 2024-06-13T06:57:45.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 client.25537 shutdown 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 msgr2=0x5602b9bfef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f801c016000 tx=0x7f801c000bc0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 shutdown_connections 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9bff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9c01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.910+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c1ca50 0x5602b9c1ee40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2606749779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9bfef80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 >> 172.21.15.7:0/2606749779 conn(0x5602b9b26170 msgr2=0x5602b9bf4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f800fe00700 0 client.25537 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 shutdown_connections 2024-06-13T06:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2606749779 wait complete. 2024-06-13T06:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.911+0000 7f803f5aeb80 20 client.25537 trim_cache size 0 max 16384 2024-06-13T06:57:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.940+0000 7f803f5aeb80 1 Processor -- start 2024-06-13T06:57:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.940+0000 7f803f5aeb80 1 -- start start 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.940+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9bfc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.940+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.940+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9c24710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80301ae720 con 0x5602b9bf95b0 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80301abf80 con 0x5602b9bff4c0 2024-06-13T06:57:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80301a9dd0 con 0x5602b9c0cb10 2024-06-13T06:57:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f802d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f802d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45768/0 (socket says 172.21.15.7:45768) 2024-06-13T06:57:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f802de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9bfc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f802d400700 1 -- 172.21.15.7:0/1121358057 learned_addr learned my addr 172.21.15.7:0/1121358057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.941+0000 7f802e800700 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9c24710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.942+0000 7f802d400700 1 -- 172.21.15.7:0/1121358057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 msgr2=0x5602b9c24710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.942+0000 7f802d400700 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9c24710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.942+0000 7f802d400700 1 -- 172.21.15.7:0/1121358057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 msgr2=0x5602b9bfc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.942+0000 7f802d400700 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9bfc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.942+0000 7f802d400700 1 -- 172.21.15.7:0/1121358057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b9c1f4a0 con 0x5602b9bff4c0 2024-06-13T06:57:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.943+0000 7f802de00700 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9bfc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.944+0000 7f802d400700 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f801c029640 tx=0x7f801c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77f5cb2b74434c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.945+0000 7f801aa00700 1 -- 172.21.15.7:0/1121358057 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c00f5a0 con 0x5602b9bff4c0 2024-06-13T06:57:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.946+0000 7f801aa00700 1 -- 172.21.15.7:0/1121358057 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c003c30 con 0x5602b9bff4c0 2024-06-13T06:57:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.946+0000 7f801aa00700 1 -- 172.21.15.7:0/1121358057 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c00f5a0 con 0x5602b9bff4c0 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1121358057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 msgr2=0x5602b9c017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f801c029640 tx=0x7f801c009290 comp rx=0 tx=0).stop 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1121358057 shutdown_connections 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9c0cb10 0x5602b9c24710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9bf95b0 0x5602b9bfc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/1121358057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c017c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.947+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1121358057 >> 172.21.15.7:0/1121358057 conn(0x5602b9b26170 msgr2=0x5602b9bfc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.948+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1121358057 shutdown_connections 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.948+0000 7f803f5aeb80 1 -- 172.21.15.7:0/1121358057 wait complete. 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.948+0000 7f803f5aeb80 1 -- start start 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.949+0000 7f803f5aeb80 1 Processor -- start 2024-06-13T06:57:45.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9c14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80301aa440 con 0x5602b9c0cb10 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80301ab8b0 con 0x5602b9bff4c0 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80301aab10 con 0x5602b9bf95b0 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f803f5aeb80 10 client.0 _ll_register_callbacks cb 0x5602b9ae5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f802d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f802d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45770/0 (socket says 172.21.15.7:45770) 2024-06-13T06:57:45.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.950+0000 7f802d400700 1 -- 172.21.15.7:0/2405030014 learned_addr learned my addr 172.21.15.7:0/2405030014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:45.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.951+0000 7f802de00700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9c14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.951+0000 7f802e800700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.952+0000 7f802d400700 1 -- 172.21.15.7:0/2405030014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 msgr2=0x5602b9c14a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.952+0000 7f802d400700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9c14a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.952+0000 7f802d400700 1 -- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 msgr2=0x5602b9c238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.952+0000 7f802d400700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.952+0000 7f802d400700 1 -- 172.21.15.7:0/2405030014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b9c23ea0 con 0x5602b9bff4c0 2024-06-13T06:57:45.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f802e800700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f802d400700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f801c010ec0 tx=0x7f801c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deaf6b03227b2e6c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f800fe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c003de0 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801c003c30 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.953+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f801c00bcc0 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f801c00f5a0 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 10 client.25553 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602b9c02720 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c003c30 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f801c030070 con 0x5602b9bff4c0 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 10 client.25553 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 10 client.25553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 20 client.25553 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c02e50 0x5602b9c05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 4 client.25553 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:57:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.954+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602b9b53580 con 0x5602b9c02e50 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.956+0000 7f802de00700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c02e50 0x5602b9c05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.957+0000 7f802de00700 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c02e50 0x5602b9c05270 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1da895eb577338a6 server_cookie=567e44999d9bc030 in_seq=0 out_seq=0 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.957+0000 7f800fe00700 10 client.25553 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.958+0000 7f800fe00700 1 -- 172.21.15.7:0/2405030014 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602b9b53580 con 0x5602b9c02e50 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.958+0000 7f800fe00700 10 client.25553 handle_command_reply: tid=0 2024-06-13T06:57:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.958+0000 7f803f5aeb80 1 client.25553 shutdown 2024-06-13T06:57:45.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 msgr2=0x5602b9c14fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f801c010ec0 tx=0x7f801c00b810 comp rx=0 tx=0).stop 2024-06-13T06:57:45.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 shutdown_connections 2024-06-13T06:57:45.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602b9c0cb10 0x5602b9c238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5602b9c02e50 0x5602b9c05270 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602b9bf95b0 0x5602b9c14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 --2- 172.21.15.7:0/2405030014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602b9bff4c0 0x5602b9c14fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f800fe00700 0 client.25553 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:57:45.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.959+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 >> 172.21.15.7:0/2405030014 conn(0x5602b9b26170 msgr2=0x5602b9bfc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.960+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 shutdown_connections 2024-06-13T06:57:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.960+0000 7f803f5aeb80 1 -- 172.21.15.7:0/2405030014 wait complete. 2024-06-13T06:57:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.960+0000 7f803f5aeb80 20 client.25553 trim_cache size 0 max 16384 2024-06-13T06:57:45.961 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:57:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 msgr2=0x7f80140a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 0x7f80140a51e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5602b9bf6920 tx=0x7f801c009c00 comp rx=0 tx=0).stop 2024-06-13T06:57:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 msgr2=0x7f80301988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f802800ef30 tx=0x7f802800c3a0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 shutdown_connections 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8030108500 0x7f80301988d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803006a170 0x7f803019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80140a2d20 0x7f80140a51e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 --2- 172.21.15.7:0/4205466926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f803006ab30 0x7f803019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 >> 172.21.15.7:0/4205466926 conn(0x7f8030078340 msgr2=0x7f80300ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.962+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 shutdown_connections 2024-06-13T06:57:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:45.963+0000 7f8034c00700 1 -- 172.21.15.7:0/4205466926 wait complete. 2024-06-13T06:57:46.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T06:57:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:47 smithi189 ceph-mon[36616]: pgmap v561: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-06-13T06:57:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:47 smithi175 ceph-mon[36886]: pgmap v561: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-06-13T06:57:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:47 smithi007 ceph-mon[32077]: pgmap v561: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-06-13T06:57:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 105.0 s, 1454.9 tps, lat 69.292 ms stddev 86.880 2024-06-13T06:57:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 -- 172.21.15.7:0/3383779457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8105610 msgr2=0x7efff810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 --2- 172.21.15.7:0/3383779457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8105610 0x7efff810bea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efff0009be0 tx=0x7efff002a6c0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 -- 172.21.15.7:0/3383779457 shutdown_connections 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 --2- 172.21.15.7:0/3383779457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8105610 0x7efff810bea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 --2- 172.21.15.7:0/3383779457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8104c50 0x7efff81050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 --2- 172.21.15.7:0/3383779457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8103a60 0x7efff8103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 -- 172.21.15.7:0/3383779457 >> 172.21.15.7:0/3383779457 conn(0x7efff80fefd0 msgr2=0x7efff8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 -- 172.21.15.7:0/3383779457 shutdown_connections 2024-06-13T06:57:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.056+0000 7efffe200640 1 -- 172.21.15.7:0/3383779457 wait complete. 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.057+0000 7efffe200640 1 Processor -- start 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 -- start start 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 0x7efff8079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 0x7efff80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efff810f6c0 con 0x7efff8105610 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efff810f540 con 0x7efff8103a60 2024-06-13T06:57:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efffe200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efff810f840 con 0x7efff8104c50 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efff7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efff7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35500/0 (socket says 172.21.15.7:35500) 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.058+0000 7efff7e00640 1 -- 172.21.15.7:0/1190097488 learned_addr learned my addr 172.21.15.7:0/1190097488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.059+0000 7efffce00640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 0x7efff8079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.059+0000 7efff7e00640 1 -- 172.21.15.7:0/1190097488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 msgr2=0x7efff8079750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.059+0000 7efff7e00640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 0x7efff8079750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.059+0000 7efff7e00640 1 -- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 msgr2=0x7efff80762c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:57:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.060+0000 7efffd800640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 0x7efff80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.060+0000 7efff7e00640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 0x7efff80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.060+0000 7efff7e00640 1 -- 172.21.15.7:0/1190097488 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff8076b80 con 0x7efff8104c50 2024-06-13T06:57:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.060+0000 7efffce00640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 0x7efff8079750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.060+0000 7efffd800640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 0x7efff80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.061+0000 7efff7e00640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.062+0000 7efff5600640 1 -- 172.21.15.7:0/1190097488 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00cd80 con 0x7efff8104c50 2024-06-13T06:57:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.062+0000 7efff5600640 1 -- 172.21.15.7:0/1190097488 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec010910 con 0x7efff8104c50 2024-06-13T06:57:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.062+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81a6130 con 0x7efff8104c50 2024-06-13T06:57:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.062+0000 7efff5600640 1 -- 172.21.15.7:0/1190097488 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec018990 con 0x7efff8104c50 2024-06-13T06:57:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.062+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a6590 con 0x7efff8104c50 2024-06-13T06:57:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.067+0000 7f0002617740 1 Processor -- start 2024-06-13T06:57:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7efff5600640 1 -- 172.21.15.7:0/1190097488 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7effec010ac0 con 0x7efff8104c50 2024-06-13T06:57:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 -- start start 2024-06-13T06:57:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78882dc0 0x560b788831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b788837b0 0x560b788882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b787e6420 con 0x560b78888920 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b7878dc50 con 0x560b78882dc0 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.068+0000 7f0002617740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b787e23e0 con 0x560b788837b0 2024-06-13T06:57:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.069+0000 7efffd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78882dc0 0x560b788831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.069+0000 7efffce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b788837b0 0x560b788882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.069+0000 7efff7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.069+0000 7efff7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52578/0 (socket says 172.21.15.7:52578) 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.069+0000 7efff7e00640 1 -- 172.21.15.7:0/1289057019 learned_addr learned my addr 172.21.15.7:0/1289057019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.070+0000 7efff7e00640 1 -- 172.21.15.7:0/1289057019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b788837b0 msgr2=0x560b788882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.070+0000 7efff7e00640 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b788837b0 0x560b788882b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.070+0000 7efff7e00640 1 -- 172.21.15.7:0/1289057019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78882dc0 msgr2=0x560b788831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.070+0000 7efff7e00640 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78882dc0 0x560b788831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.070+0000 7efff7e00640 1 -- 172.21.15.7:0/1289057019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b784fa210 con 0x560b78888920 2024-06-13T06:57:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.071+0000 7efff7e00640 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7effec0d5c20 tx=0x7effec0d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854cf9eb6c2614da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.073+0000 7efff5600640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.073+0000 7effe6000640 1 -- 172.21.15.7:0/1289057019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0de070 con 0x560b78888920 2024-06-13T06:57:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.073+0000 7effe6000640 1 -- 172.21.15.7:0/1289057019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec09c3b0 con 0x560b78888920 2024-06-13T06:57:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.073+0000 7effe6000640 1 -- 172.21.15.7:0/1289057019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec09b610 con 0x560b78888920 2024-06-13T06:57:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.074+0000 7efff5600640 1 -- 172.21.15.7:0/1190097488 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7effec0cf7f0 con 0x7efff8104c50 2024-06-13T06:57:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.075+0000 7efffd800640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.076+0000 7f0002617740 1 -- 172.21.15.7:0/1289057019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 msgr2=0x560b7888adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.076+0000 7f0002617740 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7effec0d5c20 tx=0x7effec0d7670 comp rx=0 tx=0).stop 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 -- 172.21.15.7:0/1289057019 shutdown_connections 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b7888adb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b788837b0 0x560b788882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 --2- 172.21.15.7:0/1289057019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78882dc0 0x560b788831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 -- 172.21.15.7:0/1289057019 >> 172.21.15.7:0/1289057019 conn(0x560b7884afb0 msgr2=0x560b7887e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7efffd800640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 0x7effe00a5140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efff0030960 tx=0x7efff0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.077+0000 7f0002617740 1 -- 172.21.15.7:0/1289057019 shutdown_connections 2024-06-13T06:57:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.078+0000 7f0002617740 1 -- 172.21.15.7:0/1289057019 wait complete. 2024-06-13T06:57:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.078+0000 7f0002617740 1 -- start start 2024-06-13T06:57:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.079+0000 7f0002617740 1 Processor -- start 2024-06-13T06:57:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.080+0000 7f0002617740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.080+0000 7f0002617740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b788837b0 0x560b7888d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.080+0000 7f0002617740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78894710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.080+0000 7efffce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.080+0000 7efffce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35510/0 (socket says 172.21.15.7:35510) 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 -- 172.21.15.7:0/3464343926 learned_addr learned my addr 172.21.15.7:0/3464343926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b7888b8d0 con 0x560b788837b0 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b787a10b0 con 0x560b78888920 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b7888ba50 con 0x560b78882dc0 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffd800640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78894710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efff7e00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b788837b0 0x560b7888d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7f0002617740 10 client.0 _ll_register_callbacks cb 0x560b7878cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 -- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 msgr2=0x560b78894710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78894710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 -- 172.21.15.7:0/3464343926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b788837b0 msgr2=0x560b7888d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b788837b0 0x560b7888d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.081+0000 7efffce00640 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b785c6620 con 0x560b78882dc0 2024-06-13T06:57:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.082+0000 7effdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.082+0000 7efffce00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effe8013760 tx=0x7effe8013c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831b37ee537f46d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.083+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8004440 con 0x560b78882dc0 2024-06-13T06:57:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.083+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7effe8011960 con 0x560b78882dc0 2024-06-13T06:57:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.083+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec09c3b0 con 0x560b78882dc0 2024-06-13T06:57:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.084+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effe8011b60 con 0x560b78882dc0 2024-06-13T06:57:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.084+0000 7f0002617740 10 client.25555 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.084+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b78897990 con 0x560b78882dc0 2024-06-13T06:57:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.085+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec09c3b0 con 0x560b78882dc0 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7effe801a070 con 0x560b78882dc0 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 10 client.25555 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 20 client.25555 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78899080 0x560b7889b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 4 client.25555 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b787f3b50 con 0x560b78899080 2024-06-13T06:57:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.086+0000 7efff7e00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78899080 0x560b7889b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.087+0000 7effdaa00640 10 client.25555 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.087+0000 7efff7e00640 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78899080 0x560b7889b500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc2c2d52400a6ee server_cookie=6bb04fe5d9db8131 in_seq=0 out_seq=0 2024-06-13T06:57:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.088+0000 7effdaa00640 1 -- 172.21.15.7:0/3464343926 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b787f3b50 con 0x560b78899080 2024-06-13T06:57:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7effdaa00640 10 client.25555 handle_command_reply: tid=0 2024-06-13T06:57:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 client.25555 shutdown 2024-06-13T06:57:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 msgr2=0x560b7888d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7effe8013760 tx=0x7effe8013c20 comp rx=0 tx=0).stop 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 shutdown_connections 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78899080 0x560b7889b500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78894710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b788837b0 0x560b7888d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 --2- 172.21.15.7:0/3464343926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7888d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.089+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 >> 172.21.15.7:0/3464343926 conn(0x560b7884afb0 msgr2=0x560b7887d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.090+0000 7effdaa00640 0 client.25555 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.090+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 shutdown_connections 2024-06-13T06:57:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.090+0000 7f0002617740 1 -- 172.21.15.7:0/3464343926 wait complete. 2024-06-13T06:57:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.090+0000 7f0002617740 20 client.25555 trim_cache size 0 max 16384 2024-06-13T06:57:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 Processor -- start 2024-06-13T06:57:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 -- start start 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7887d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b788946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b33a0 con 0x560b78888920 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b0c40 con 0x560b788837b0 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.136+0000 7f0002617740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efff81aea90 con 0x560b78882dc0 2024-06-13T06:57:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.137+0000 7efff7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.137+0000 7efff7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:45818/0 (socket says 172.21.15.7:45818) 2024-06-13T06:57:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.137+0000 7efff7e00640 1 -- 172.21.15.7:0/3811851120 learned_addr learned my addr 172.21.15.7:0/3811851120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efffce00640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7887d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efffd800640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b788946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efff7e00640 1 -- 172.21.15.7:0/3811851120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 msgr2=0x560b7887d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efff7e00640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7887d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efff7e00640 1 -- 172.21.15.7:0/3811851120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 msgr2=0x560b788946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efff7e00640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b788946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efff7e00640 1 -- 172.21.15.7:0/3811851120 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7887eb50 con 0x560b788837b0 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efffd800640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b788946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.138+0000 7efffce00640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7887d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.139+0000 7efff7e00640 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7effec098110 tx=0x7effec0d5300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2412a0fff60756 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.140+0000 7effe5600640 1 -- 172.21.15.7:0/3811851120 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec0de040 con 0x560b788837b0 2024-06-13T06:57:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.140+0000 7effe5600640 1 -- 172.21.15.7:0/3811851120 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec09c3b0 con 0x560b788837b0 2024-06-13T06:57:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.140+0000 7effe5600640 1 -- 172.21.15.7:0/3811851120 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec09bcb0 con 0x560b788837b0 2024-06-13T06:57:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 -- 172.21.15.7:0/3811851120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 msgr2=0x560b7887dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7effec098110 tx=0x7effec0d5300 comp rx=0 tx=0).stop 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 -- 172.21.15.7:0/3811851120 shutdown_connections 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78888920 0x560b788946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b788837b0 0x560b7887dd60 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7effec098110 tx=0x7effec0d5300 comp rx=0 tx=0).stop 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 --2- 172.21.15.7:0/3811851120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78882dc0 0x560b7887d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 -- 172.21.15.7:0/3811851120 >> 172.21.15.7:0/3811851120 conn(0x560b7884afb0 msgr2=0x560b7887d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.141+0000 7f0002617740 1 -- 172.21.15.7:0/3811851120 shutdown_connections 2024-06-13T06:57:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.142+0000 7f0002617740 1 -- 172.21.15.7:0/3811851120 wait complete. 2024-06-13T06:57:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.143+0000 7f0002617740 1 -- start start 2024-06-13T06:57:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.143+0000 7f0002617740 1 Processor -- start 2024-06-13T06:57:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.145+0000 7f0002617740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78882dc0 0x560b78891ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.145+0000 7f0002617740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78892530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.145+0000 7f0002617740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7f0002617740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efff81af100 con 0x560b78882dc0 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7f0002617740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b0570 con 0x560b78888920 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7f0002617740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efff81af7d0 con 0x560b78893540 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7f0002617740 10 client.0 _ll_register_callbacks cb 0x7effe8009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7efffd800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7efffd800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35524/0 (socket says 172.21.15.7:35524) 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.146+0000 7efffd800640 1 -- 172.21.15.7:0/1379233589 learned_addr learned my addr 172.21.15.7:0/1379233589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efff7e00640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78892530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efffd800640 1 -- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 msgr2=0x560b78892530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efffd800640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78892530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efffd800640 1 -- 172.21.15.7:0/1379233589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78882dc0 msgr2=0x560b78891ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efffd800640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78882dc0 0x560b78891ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.147+0000 7efffd800640 1 -- 172.21.15.7:0/1379233589 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b78893f00 con 0x560b78893540 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.148+0000 7efffd800640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efff0009be0 tx=0x7efff002a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de650a7dfda9f11 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.148+0000 7effdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.149+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0044070 con 0x560b78893540 2024-06-13T06:57:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.149+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efff0004450 con 0x560b78893540 2024-06-13T06:57:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.149+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec09c3b0 con 0x560b78893540 2024-06-13T06:57:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.150+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efff0036050 con 0x560b78893540 2024-06-13T06:57:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.150+0000 7f0002617740 10 client.25561 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.150+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b78904370 con 0x560b78893540 2024-06-13T06:57:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec09c3b0 con 0x560b78893540 2024-06-13T06:57:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7efff003c040 con 0x560b78893540 2024-06-13T06:57:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7f0002617740 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7f0002617740 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7f0002617740 20 client.25561 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.152+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78904de0 0x560b78907200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.153+0000 7efffce00640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78904de0 0x560b78907200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.153+0000 7f0002617740 4 client.25561 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:57:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.153+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b787edea0 con 0x560b78904de0 2024-06-13T06:57:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.154+0000 7efffce00640 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78904de0 0x560b78907200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5bb99f2280a559 server_cookie=3f87c5e55784694e in_seq=0 out_seq=0 2024-06-13T06:57:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.154+0000 7effdaa00640 10 client.25561 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.243+0000 7effdaa00640 1 -- 172.21.15.7:0/1379233589 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560b787edea0 con 0x560b78904de0 2024-06-13T06:57:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.243+0000 7effdaa00640 10 client.25561 handle_command_reply: tid=0 2024-06-13T06:57:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.243+0000 7f0002617740 1 client.25561 shutdown 2024-06-13T06:57:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 msgr2=0x560b788939c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efff0009be0 tx=0x7efff002a6c0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 shutdown_connections 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b78904de0 0x560b78907200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b78893540 0x560b788939c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b78888920 0x560b78892530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 --2- 172.21.15.7:0/1379233589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b78882dc0 0x560b78891ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 >> 172.21.15.7:0/1379233589 conn(0x560b7884afb0 msgr2=0x560b7887d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.244+0000 7effdaa00640 0 client.25561 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.245+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 shutdown_connections 2024-06-13T06:57:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.245+0000 7f0002617740 1 -- 172.21.15.7:0/1379233589 wait complete. 2024-06-13T06:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.245+0000 7f0002617740 20 client.25561 trim_cache size 0 max 16384 2024-06-13T06:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:57:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.246+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 msgr2=0x7effe00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.246+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 0x7effe00a5140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efff0030960 tx=0x7efff0005fd0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.246+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 msgr2=0x7efff8079c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00eeb0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 shutdown_connections 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7effe00a2c80 0x7effe00a5140 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff8105610 0x7efff80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efff8104c50 0x7efff8079c90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 --2- 172.21.15.7:0/1190097488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efff8103a60 0x7efff8079750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 >> 172.21.15.7:0/1190097488 conn(0x7efff80fefd0 msgr2=0x7efff8107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 shutdown_connections 2024-06-13T06:57:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:57:48.247+0000 7efffe200640 1 -- 172.21.15.7:0/1190097488 wait complete. 2024-06-13T06:57:49.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:49 smithi007 ceph-mon[32077]: pgmap v562: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-06-13T06:57:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:49 smithi189 ceph-mon[36616]: pgmap v562: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-06-13T06:57:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:49 smithi175 ceph-mon[36886]: pgmap v562: 129 pgs: 129 active+clean; 1020 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 930 op/s 2024-06-13T06:57:51.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:51 smithi007 ceph-mon[32077]: pgmap v563: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 720 op/s 2024-06-13T06:57:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 -- 172.21.15.7:0/1459714855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 msgr2=0x7f9d141015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 --2- 172.21.15.7:0/1459714855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d141015a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009990 tx=0x7f9cfc009ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 -- 172.21.15.7:0/1459714855 shutdown_connections 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 --2- 172.21.15.7:0/1459714855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d1410ece0 0x7f9d14111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 --2- 172.21.15.7:0/1459714855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d141015a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 --2- 172.21.15.7:0/1459714855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d14101b70 0x7f9d1410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.054+0000 7f9d13200700 1 -- 172.21.15.7:0/1459714855 >> 172.21.15.7:0/1459714855 conn(0x7f9d140fcd70 msgr2=0x7f9d140ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.056+0000 7f9d13200700 1 -- 172.21.15.7:0/1459714855 shutdown_connections 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.056+0000 7f9d13200700 1 -- 172.21.15.7:0/1459714855 wait complete. 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.057+0000 7f9d13200700 1 Processor -- start 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 -- start start 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d14107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14101b70 0x7f9d141080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 0x7f9d14103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14114a30 con 0x7f9d1410ece0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141148b0 con 0x7f9d14101b70 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.058+0000 7f9d13200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14114bb0 con 0x7f9d141011c0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.059+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d14107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.059+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d14107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34966/0 (socket says 172.21.15.7:34966) 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.059+0000 7f9d11e00700 1 -- 172.21.15.7:0/2682451567 learned_addr learned my addr 172.21.15.7:0/2682451567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.059+0000 7f9d12800700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 0x7f9d14103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d11400700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14101b70 0x7f9d141080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d12800700 1 -- 172.21.15.7:0/2682451567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 msgr2=0x7f9d14107bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d12800700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d14107bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d12800700 1 -- 172.21.15.7:0/2682451567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14101b70 msgr2=0x7f9d141080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d12800700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14101b70 0x7f9d141080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.062+0000 7f9d12800700 1 -- 172.21.15.7:0/2682451567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d14103a70 con 0x7f9d1410ece0 2024-06-13T06:57:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.063+0000 7f9d12800700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 0x7f9d14103320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00cc50 tx=0x7f9d0c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.063+0000 7f9d0aa00700 1 -- 172.21.15.7:0/2682451567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c016020 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.064+0000 7f9d0aa00700 1 -- 172.21.15.7:0/2682451567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c00ab40 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.064+0000 7f9d0aa00700 1 -- 172.21.15.7:0/2682451567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0115e0 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.067+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d14103d60 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.068+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1410dad0 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.074+0000 7f9d0aa00700 1 -- 172.21.15.7:0/2682451567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9d0c019020 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.076+0000 7f9d23699b80 1 Processor -- start 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.076+0000 7f9d23699b80 1 -- start start 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.077+0000 7f9d0aa00700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 0x7f9cf80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.078+0000 7f9d0aa00700 1 -- 172.21.15.7:0/2682451567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9d0c0cef10 con 0x7f9d1410ece0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.078+0000 7f9d11400700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 0x7f9cf80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 0x5644fdf86ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf89920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d11400700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 0x7f9cf80a52b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9d04006fd0 tx=0x7f9d04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644fde8ba10 con 0x5644fdf7c680 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34974/0 (socket says 172.21.15.7:34974) 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d11e00700 1 -- 172.21.15.7:0/1123427328 learned_addr learned my addr 172.21.15.7:0/1123427328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5644fdeceb00 con 0x5644fdf87530 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644fde898f0 con 0x5644fdf7bd60 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.079+0000 7f9d12800700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11400700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 0x5644fdf86ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 -- 172.21.15.7:0/1123427328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 msgr2=0x5644fdf89920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf89920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 -- 172.21.15.7:0/1123427328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 msgr2=0x5644fdf86ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 0x5644fdf86ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 -- 172.21.15.7:0/1123427328 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fde76f90 con 0x5644fdf7bd60 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d12800700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf89920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11400700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 0x5644fdf86ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.080+0000 7f9d11e00700 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc003680 tx=0x7f9cfc0036b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7f3d204511b6ea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.081+0000 7f9d03400700 1 -- 172.21.15.7:0/1123427328 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc028070 con 0x5644fdf7bd60 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.081+0000 7f9d03400700 1 -- 172.21.15.7:0/1123427328 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc0055b0 con 0x5644fdf7bd60 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.081+0000 7f9d03400700 1 -- 172.21.15.7:0/1123427328 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00bdf0 con 0x5644fdf7bd60 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.082+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 msgr2=0x5644fdf7c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.082+0000 7f9d23699b80 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc003680 tx=0x7f9cfc0036b0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 shutdown_connections 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf89920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7bd60 0x5644fdf7c140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 --2- 172.21.15.7:0/1123427328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7c680 0x5644fdf86ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 >> 172.21.15.7:0/1123427328 conn(0x5644fdea9170 msgr2=0x5644fdf78280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 shutdown_connections 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 -- 172.21.15.7:0/1123427328 wait complete. 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 -- start start 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.084+0000 7f9d23699b80 1 Processor -- start 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.085+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf86d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.085+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 0x5644fdf84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.087+0000 7f9d11400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 0x5644fdf84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf84560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d11400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 0x5644fdf84020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55058/0 (socket says 172.21.15.7:55058) 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d11400700 1 -- 172.21.15.7:0/3393221845 learned_addr learned my addr 172.21.15.7:0/3393221845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d23699b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644fde6af60 con 0x5644fdf7bd60 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5644fdecaac0 con 0x5644fdf7c680 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644fdf89f60 con 0x5644fdf87530 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.089+0000 7f9d23699b80 10 client.0 _ll_register_callbacks cb 0x5644fde64380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.090+0000 7f9d12800700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf84560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.092+0000 7f9d12800700 1 -- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 msgr2=0x5644fdf84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.092+0000 7f9d12800700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 0x5644fdf84020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.092+0000 7f9d12800700 1 -- 172.21.15.7:0/3393221845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 msgr2=0x5644fdf86d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.092+0000 7f9d12800700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf86d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.092+0000 7f9d12800700 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fde1d8d0 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.097+0000 7f9d12800700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf84560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00afd0 tx=0x7f9d0c019400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5aac89ee80b2917 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.097+0000 7f9cf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.097+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b3070 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.098+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9d0c0b4260 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.099+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cfc0055b0 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.100+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d0c09b460 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.101+0000 7f9d23699b80 10 client.25573 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.101+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644fdf946d0 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.102+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc0055b0 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.102+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9d0c023060 con 0x5644fdf87530 2024-06-13T06:57:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.102+0000 7f9d23699b80 10 client.25573 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.102+0000 7f9d23699b80 10 client.25573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.103+0000 7f9d23699b80 20 client.25573 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.103+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fdf96f40 0x5644fdf99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.103+0000 7f9d23699b80 4 client.25573 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.103+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644fdedc000 con 0x5644fdf96f40 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.103+0000 7f9d11e00700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fdf96f40 0x5644fdf99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.104+0000 7f9cf7e00700 10 client.25573 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.105+0000 7f9d11e00700 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fdf96f40 0x5644fdf99360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb236e09d9a7622 server_cookie=ead41caf6c01dc3e in_seq=0 out_seq=0 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.106+0000 7f9cf7e00700 1 -- 172.21.15.7:0/3393221845 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644fdedc000 con 0x5644fdf96f40 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.106+0000 7f9cf7e00700 10 client.25573 handle_command_reply: tid=0 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.107+0000 7f9d23699b80 1 client.25573 shutdown 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.108+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 msgr2=0x5644fdf84560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.108+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf84560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00afd0 tx=0x7f9d0c019400 comp rx=0 tx=0).stop 2024-06-13T06:57:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 shutdown_connections 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fdf96f40 0x5644fdf99360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf84560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf86d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 --2- 172.21.15.7:0/3393221845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7c680 0x5644fdf84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.110+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 >> 172.21.15.7:0/3393221845 conn(0x5644fdea9170 msgr2=0x5644fdf77e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.111+0000 7f9cf7e00700 0 client.25573 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.111+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 shutdown_connections 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.112+0000 7f9d23699b80 1 -- 172.21.15.7:0/3393221845 wait complete. 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.112+0000 7f9d23699b80 20 client.25573 trim_cache size 0 max 16384 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.145+0000 7f9d23699b80 1 Processor -- start 2024-06-13T06:57:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.145+0000 7f9d23699b80 1 -- start start 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.146+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf83fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.146+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.146+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d23699b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141c7e40 con 0x5644fdf7bd60 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d23699b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14073620 con 0x5644fdf87530 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d23699b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14071430 con 0x5644fdf7c680 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34994/0 (socket says 172.21.15.7:34994) 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 -- 172.21.15.7:0/2482931162 learned_addr learned my addr 172.21.15.7:0/2482931162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d12800700 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 -- 172.21.15.7:0/2482931162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 msgr2=0x5644fdf9ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf9ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 -- 172.21.15.7:0/2482931162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 msgr2=0x5644fdf83fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf83fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.147+0000 7f9d11400700 1 -- 172.21.15.7:0/2482931162 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fdf796c0 con 0x5644fdf7c680 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.148+0000 7f9d12800700 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf9ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.148+0000 7f9d11400700 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d04013b80 tx=0x7f9d0400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47cb298836fbe98d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.151+0000 7f9d02a00700 1 -- 172.21.15.7:0/2482931162 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04024030 con 0x5644fdf7c680 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.151+0000 7f9d02a00700 1 -- 172.21.15.7:0/2482931162 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d04011d40 con 0x5644fdf7c680 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.151+0000 7f9d02a00700 1 -- 172.21.15.7:0/2482931162 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04019040 con 0x5644fdf7c680 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 -- 172.21.15.7:0/2482931162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 msgr2=0x5644fdf84520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d04013b80 tx=0x7f9d0400f000 comp rx=0 tx=0).stop 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 -- 172.21.15.7:0/2482931162 shutdown_connections 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf87530 0x5644fdf9ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf7bd60 0x5644fdf83fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 --2- 172.21.15.7:0/2482931162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf7c680 0x5644fdf84520 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d04013b80 tx=0x7f9d0400f000 comp rx=0 tx=0).stop 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.152+0000 7f9d23699b80 1 -- 172.21.15.7:0/2482931162 >> 172.21.15.7:0/2482931162 conn(0x5644fdea9170 msgr2=0x5644fdf838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.153+0000 7f9d23699b80 1 -- 172.21.15.7:0/2482931162 shutdown_connections 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.153+0000 7f9d23699b80 1 -- 172.21.15.7:0/2482931162 wait complete. 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.154+0000 7f9d23699b80 1 -- start start 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.154+0000 7f9d23699b80 1 Processor -- start 2024-06-13T06:57:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.157+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 0x5644fdf8cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.157+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf8d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.157+0000 7f9d23699b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.157+0000 7f9d11400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf8d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d23699b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14071ae0 con 0x5644fdf8d960 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d23699b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14072f50 con 0x5644fdf7bd60 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d23699b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d140721b0 con 0x5644fdf87530 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d11e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 0x5644fdf8cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d12800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d12800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50024/0 (socket says 172.21.15.7:50024) 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.158+0000 7f9d12800700 1 -- 172.21.15.7:0/777090560 learned_addr learned my addr 172.21.15.7:0/777090560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d23699b80 10 client.0 _ll_register_callbacks cb 0x5644fde64380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11400700 1 -- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 msgr2=0x5644fdf8cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11400700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 0x5644fdf8cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11400700 1 -- 172.21.15.7:0/777090560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 msgr2=0x5644fe000bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11400700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11400700 1 -- 172.21.15.7:0/777090560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fe001130 con 0x5644fdf87530 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d04011d40 con 0x5644fdf87530 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d12800700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.159+0000 7f9d11e00700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 0x5644fdf8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.160+0000 7f9d11400700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf8d420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5644fdf723b0 tx=0x7f9d04011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f90c411ac2df7467 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.160+0000 7f9cf7e00700 10 client.25579 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.161+0000 7f9cf7e00700 1 -- 172.21.15.7:0/777090560 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04024040 con 0x5644fdf87530 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.162+0000 7f9cf7e00700 1 -- 172.21.15.7:0/777090560 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9d04020b00 con 0x5644fdf87530 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.162+0000 7f9cf7e00700 1 -- 172.21.15.7:0/777090560 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d04020da0 con 0x5644fdf87530 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.162+0000 7f9d23699b80 10 client.25579 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.162+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5644fe002190 con 0x5644fdf87530 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9cf7e00700 1 -- 172.21.15.7:0/777090560 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9d04020460 con 0x5644fdf87530 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 10 client.25579 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 20 client.25579 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fe002630 0x5644fe004a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 4 client.25579 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644fded6580 con 0x5644fe002630 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.163+0000 7f9d11e00700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fe002630 0x5644fe004a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.165+0000 7f9cf7e00700 10 client.25579 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.165+0000 7f9d11e00700 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fe002630 0x5644fe004a50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a316fd78e85ca2cf server_cookie=67e8ff4f4400ecaa in_seq=0 out_seq=0 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9cf7e00700 1 -- 172.21.15.7:0/777090560 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644fded6580 con 0x5644fe002630 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9cf7e00700 10 client.25579 handle_command_reply: tid=0 2024-06-13T06:57:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 client.25579 shutdown 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 msgr2=0x5644fdf8d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf8d420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5644fdf723b0 tx=0x7f9d04011f60 comp rx=0 tx=0).stop 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 shutdown_connections 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644fe002630 0x5644fe004a50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644fdf8d960 0x5644fe000bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644fdf7bd60 0x5644fdf8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 --2- 172.21.15.7:0/777090560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644fdf87530 0x5644fdf8d420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9cf7e00700 0 client.25579 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.166+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 >> 172.21.15.7:0/777090560 conn(0x5644fdea9170 msgr2=0x5644fdf84160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.167+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 shutdown_connections 2024-06-13T06:57:51.464 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.167+0000 7f9d23699b80 1 -- 172.21.15.7:0/777090560 wait complete. 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.167+0000 7f9d23699b80 20 client.25579 trim_cache size 0 max 16384 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 msgr2=0x7f9cf80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 0x7f9cf80a52b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9d04006fd0 tx=0x7f9d04008040 comp rx=0 tx=0).stop 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 msgr2=0x7f9d14103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 0x7f9d14103320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00cc50 tx=0x7f9d0c00cf60 comp rx=0 tx=0).stop 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 shutdown_connections 2024-06-13T06:57:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d1410ece0 0x7f9d14103320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d141011c0 0x7f9d14107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9cf80a2df0 0x7f9cf80a52b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 --2- 172.21.15.7:0/2682451567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14101b70 0x7f9d141080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 >> 172.21.15.7:0/2682451567 conn(0x7f9d140fcd70 msgr2=0x7f9d1410fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 shutdown_connections 2024-06-13T06:57:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:51.168+0000 7f9d13200700 1 -- 172.21.15.7:0/2682451567 wait complete. 2024-06-13T06:57:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:51 smithi189 ceph-mon[36616]: pgmap v563: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 720 op/s 2024-06-13T06:57:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:51 smithi175 ceph-mon[36886]: pgmap v563: 129 pgs: 129 active+clean; 1020 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 MiB/s wr, 720 op/s 2024-06-13T06:57:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 110.0 s, 1254.5 tps, lat 79.371 ms stddev 95.043 2024-06-13T06:57:53.369 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T06:57:53.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T06:57:53.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:53 smithi007 ceph-mon[32077]: pgmap v564: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-06-13T06:57:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:53 smithi189 ceph-mon[36616]: pgmap v564: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-06-13T06:57:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:53 smithi175 ceph-mon[36886]: pgmap v564: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-06-13T06:57:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:55 smithi189 ceph-mon[36616]: pgmap v565: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 601 op/s 2024-06-13T06:57:55.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:55 smithi007 ceph-mon[32077]: pgmap v565: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 601 op/s 2024-06-13T06:57:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:55 smithi175 ceph-mon[36886]: pgmap v565: 129 pgs: 129 active+clean; 1020 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 601 op/s 2024-06-13T06:57:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.124+0000 7f31e5c00700 1 -- 172.21.15.7:0/981368087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e006a790 msgr2=0x7f31e006ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.124+0000 7f31e5c00700 1 --2- 172.21.15.7:0/981368087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e006a790 0x7f31e006ac40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b0a0 tx=0x7f31d800b3b0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.126+0000 7f31e5c00700 1 -- 172.21.15.7:0/981368087 shutdown_connections 2024-06-13T06:57:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.126+0000 7f31e5c00700 1 --2- 172.21.15.7:0/981368087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e006a790 0x7f31e006ac40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.126+0000 7f31e5c00700 1 --2- 172.21.15.7:0/981368087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e01084e0 0x7f31e01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.126+0000 7f31e5c00700 1 --2- 172.21.15.7:0/981368087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e0108e00 0x7f31e006a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.126+0000 7f31e5c00700 1 -- 172.21.15.7:0/981368087 >> 172.21.15.7:0/981368087 conn(0x7f31e0078340 msgr2=0x7f31e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.127+0000 7f31e5c00700 1 -- 172.21.15.7:0/981368087 shutdown_connections 2024-06-13T06:57:57.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.127+0000 7f31e5c00700 1 -- 172.21.15.7:0/981368087 wait complete. 2024-06-13T06:57:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.130+0000 7f31e5c00700 1 Processor -- start 2024-06-13T06:57:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.130+0000 7f31e5c00700 1 -- start start 2024-06-13T06:57:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 0x7f31e019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 0x7f31e01a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31e010c420 con 0x7f31e01084e0 2024-06-13T06:57:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31e010c2a0 con 0x7f31e006a790 2024-06-13T06:57:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31e5c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31e010c5a0 con 0x7f31e0108e00 2024-06-13T06:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31de000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 0x7f31e019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31df400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 0x7f31e01a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31de000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50028/0 (socket says 172.21.15.7:50028) 2024-06-13T06:57:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.131+0000 7f31de000700 1 -- 172.21.15.7:0/2653170023 learned_addr learned my addr 172.21.15.7:0/2653170023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 -- 172.21.15.7:0/2653170023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 msgr2=0x7f31e019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 0x7f31e019f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 -- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 msgr2=0x7f31e019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 -- 172.21.15.7:0/2653170023 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e01a41e0 con 0x7f31e0108e00 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31dea00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 0x7f31e019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31de000700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.132+0000 7f31df400700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 0x7f31e01a3aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b0a0 tx=0x7f31d8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.133+0000 7f31d7400700 1 -- 172.21.15.7:0/2653170023 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8021070 con 0x7f31e0108e00 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.133+0000 7f31d7400700 1 -- 172.21.15.7:0/2653170023 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d8007b80 con 0x7f31e0108e00 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.133+0000 7f31d7400700 1 -- 172.21.15.7:0/2653170023 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8005040 con 0x7f31e0108e00 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.133+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e01a4470 con 0x7f31e0108e00 2024-06-13T06:57:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.133+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e01abd50 con 0x7f31e0108e00 2024-06-13T06:57:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.140+0000 7f31d7400700 1 -- 172.21.15.7:0/2653170023 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f31d8024070 con 0x7f31e0108e00 2024-06-13T06:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.141+0000 7f31d7400700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 0x7f31c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.141+0000 7f31d7400700 1 -- 172.21.15.7:0/2653170023 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f31d80d0170 con 0x7f31e0108e00 2024-06-13T06:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31f0620b80 1 Processor -- start 2024-06-13T06:57:57.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31f0620b80 1 -- start start 2024-06-13T06:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31dea00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 0x7f31c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.142+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31de000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31de000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55092/0 (socket says 172.21.15.7:55092) 2024-06-13T06:57:57.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31dea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31dea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50042/0 (socket says 172.21.15.7:50042) 2024-06-13T06:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31df400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31df400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35016/0 (socket says 172.21.15.7:35016) 2024-06-13T06:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31de000700 1 -- 172.21.15.7:0/1024777603 learned_addr learned my addr 172.21.15.7:0/1024777603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:57.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558755302a90 con 0x55875536fec0 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558755345b80 con 0x5587553f3630 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.143+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558755300970 con 0x5587553fe380 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 0x7f31c40a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f31d00138e0 tx=0x7f31d0013ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 -- 172.21.15.7:0/1024777603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 msgr2=0x558755400770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 -- 172.21.15.7:0/1024777603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 msgr2=0x5587553f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31dea00700 1 -- 172.21.15.7:0/1024777603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587552ee010 con 0x55875536fec0 2024-06-13T06:57:57.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.144+0000 7f31df400700 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.145+0000 7f31dea00700 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31d000f6a0 tx=0x7f31d0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e2bc0fe7c8ecc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.146+0000 7f31cbe00700 1 -- 172.21.15.7:0/1024777603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d00120d0 con 0x55875536fec0 2024-06-13T06:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.146+0000 7f31cbe00700 1 -- 172.21.15.7:0/1024777603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d0024020 con 0x55875536fec0 2024-06-13T06:57:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 msgr2=0x5587553f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31d000f6a0 tx=0x7f31d0020000 comp rx=0 tx=0).stop 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 shutdown_connections 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x558755400770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55875536fec0 0x5587553f30f0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31d000f6a0 tx=0x7f31d0020000 comp rx=0 tx=0).stop 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1024777603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.147+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 >> 172.21.15.7:0/1024777603 conn(0x5587553201f0 msgr2=0x5587553ef530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.149+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 shutdown_connections 2024-06-13T06:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.149+0000 7f31f0620b80 1 -- 172.21.15.7:0/1024777603 wait complete. 2024-06-13T06:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.149+0000 7f31f0620b80 1 -- start start 2024-06-13T06:57:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.149+0000 7f31f0620b80 1 Processor -- start 2024-06-13T06:57:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x5587553fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5587552e1fe0 con 0x5587553fe380 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558755341b40 con 0x5587553f3630 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558755401020 con 0x5587553fb510 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31f0620b80 10 client.0 _ll_register_callbacks cb 0x5587552df7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31de000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31de000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50058/0 (socket says 172.21.15.7:50058) 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.150+0000 7f31de000700 1 -- 172.21.15.7:0/723735549 learned_addr learned my addr 172.21.15.7:0/723735549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 -- 172.21.15.7:0/723735549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 msgr2=0x5587553fb990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x5587553fb990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 -- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 msgr2=0x5587553fdcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31dea00700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 -- 172.21.15.7:0/723735549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558755294950 con 0x5587553fe380 2024-06-13T06:57:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31dea00700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31de000700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31cc011d10 tx=0x7f31cc00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6f8be7f8d6e84d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.151+0000 7f31c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc01a3f0 con 0x5587553fe380 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f31cc01a590 con 0x5587553fe380 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558755353080 con 0x5587553fe380 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31cc027050 con 0x5587553fe380 2024-06-13T06:57:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31f0620b80 10 client.15366 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.152+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55875541cea0 con 0x5587553fe380 2024-06-13T06:57:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.154+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558755353080 con 0x5587553fe380 2024-06-13T06:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.154+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f31cc022030 con 0x5587553fe380 2024-06-13T06:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 20 client.15366 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875541f4e0 0x558755421930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 4 client.15366 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:57:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.155+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55875534d600 con 0x55875541f4e0 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.156+0000 7f31dea00700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875541f4e0 0x558755421930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.156+0000 7f31dea00700 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875541f4e0 0x558755421930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f8fb65429df408 server_cookie=c1d188fb3fad60da in_seq=0 out_seq=0 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.156+0000 7f31c8c00700 10 client.15366 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.157+0000 7f31c8c00700 1 -- 172.21.15.7:0/723735549 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55875534d600 con 0x55875541f4e0 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.157+0000 7f31c8c00700 10 client.15366 handle_command_reply: tid=0 2024-06-13T06:57:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.157+0000 7f31f0620b80 1 client.15366 shutdown 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 msgr2=0x5587553fafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31cc011d10 tx=0x7f31cc00f6a0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 shutdown_connections 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875541f4e0 0x558755421930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x5587553fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31c8c00700 0 client.15366 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 --2- 172.21.15.7:0/723735549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x5587553fafd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 >> 172.21.15.7:0/723735549 conn(0x5587553201f0 msgr2=0x5587553ee7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 shutdown_connections 2024-06-13T06:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 1 -- 172.21.15.7:0/723735549 wait complete. 2024-06-13T06:57:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.158+0000 7f31f0620b80 20 client.15366 trim_cache size 0 max 16384 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.197+0000 7f31f0620b80 1 Processor -- start 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.197+0000 7f31f0620b80 1 -- start start 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 0x5587553ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x5587553ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01ae8d0 con 0x5587553fb510 2024-06-13T06:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01ac130 con 0x5587553f3630 2024-06-13T06:57:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.198+0000 7f31f0620b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0072720 con 0x5587553fe380 2024-06-13T06:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.199+0000 7f31dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.199+0000 7f31dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55104/0 (socket says 172.21.15.7:55104) 2024-06-13T06:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.199+0000 7f31dea00700 1 -- 172.21.15.7:0/1998317154 learned_addr learned my addr 172.21.15.7:0/1998317154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 0x5587553ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31df400700 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x5587553ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 -- 172.21.15.7:0/1998317154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 msgr2=0x5587553ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x5587553ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 -- 172.21.15.7:0/1998317154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 msgr2=0x5587553fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.200+0000 7f31de000700 1 -- 172.21.15.7:0/1998317154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55875540af30 con 0x5587553fb510 2024-06-13T06:57:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.201+0000 7f31de000700 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 0x5587553ff770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31cc019840 tx=0x7f31cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb0844914baa776 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.202+0000 7f31cb400700 1 -- 172.21.15.7:0/1998317154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0182f0 con 0x5587553fb510 2024-06-13T06:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.202+0000 7f31cb400700 1 -- 172.21.15.7:0/1998317154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc030070 con 0x5587553fb510 2024-06-13T06:57:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31cb400700 1 -- 172.21.15.7:0/1998317154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc019390 con 0x5587553fb510 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 -- 172.21.15.7:0/1998317154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 msgr2=0x5587553ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 0x5587553ff770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31cc019840 tx=0x7f31cc001e70 comp rx=0 tx=0).stop 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 -- 172.21.15.7:0/1998317154 shutdown_connections 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fe380 0x5587553ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587553fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 --2- 172.21.15.7:0/1998317154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fb510 0x5587553ff770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 -- 172.21.15.7:0/1998317154 >> 172.21.15.7:0/1998317154 conn(0x5587553201f0 msgr2=0x5587553f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.204+0000 7f31f0620b80 1 -- 172.21.15.7:0/1998317154 shutdown_connections 2024-06-13T06:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.205+0000 7f31f0620b80 1 -- 172.21.15.7:0/1998317154 wait complete. 2024-06-13T06:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.205+0000 7f31f0620b80 1 -- start start 2024-06-13T06:57:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.206+0000 7f31f0620b80 1 Processor -- start 2024-06-13T06:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x558755406b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x558755426850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0072d90 con 0x5587553fe380 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0074200 con 0x5587553f3630 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0073460 con 0x5587553fb510 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.209+0000 7f31f0620b80 10 client.0 _ll_register_callbacks cb 0x5587552df7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.210+0000 7f31dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.210+0000 7f31dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55118/0 (socket says 172.21.15.7:55118) 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.210+0000 7f31dea00700 1 -- 172.21.15.7:0/3665313445 learned_addr learned my addr 172.21.15.7:0/3665313445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.210+0000 7f31df400700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x558755426850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.210+0000 7f31de000700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x558755406b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.211+0000 7f31dea00700 1 -- 172.21.15.7:0/3665313445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 msgr2=0x558755406b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31dea00700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x558755406b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31dea00700 1 -- 172.21.15.7:0/3665313445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 msgr2=0x558755426850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31dea00700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x558755426850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31dea00700 1 -- 172.21.15.7:0/3665313445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558755427090 con 0x5587553f3630 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31de000700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x558755406b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.213+0000 7f31df400700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x558755426850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.214+0000 7f31dea00700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31d000f6a0 tx=0x7f31d00058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b108d636b2378bc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:57:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.214+0000 7f31c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.218+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0037070 con 0x5587553f3630 2024-06-13T06:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.218+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f31d00052c0 con 0x5587553f3630 2024-06-13T06:57:57.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.219+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31cc030070 con 0x5587553f3630 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.227+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31d0041050 con 0x5587553f3630 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.227+0000 7f31f0620b80 10 client.25598 fetch_fsmap learned FSMap version 29 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.227+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558755405f00 con 0x5587553f3630 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc030070 con 0x5587553f3630 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f31d001f070 con 0x5587553f3630 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 20 client.25598 populate_metadata read hostname 'smithi007' 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875548a210 0x55875548c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 4 client.25598 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T06:57:57.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.228+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587551c72d0 con 0x55875548a210 2024-06-13T06:57:57.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.230+0000 7f31de000700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875548a210 0x55875548c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:57:57.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.232+0000 7f31c8c00700 10 client.25598 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:57.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.232+0000 7f31de000700 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875548a210 0x55875548c600 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbd9cc3c00a1a6a server_cookie=541c670a58f18569 in_seq=0 out_seq=0 2024-06-13T06:57:57.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.233+0000 7f31c8c00700 1 -- 172.21.15.7:0/3665313445 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587551c72d0 con 0x55875548a210 2024-06-13T06:57:57.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.233+0000 7f31c8c00700 10 client.25598 handle_command_reply: tid=0 2024-06-13T06:57:57.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.233+0000 7f31f0620b80 1 client.25598 shutdown 2024-06-13T06:57:57.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.234+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 msgr2=0x5587554065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.234+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31d000f6a0 tx=0x7f31d00058e0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 shutdown_connections 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55875548a210 0x55875548c600 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587553fe380 0x558755426850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5587553f3630 0x5587554065f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 --2- 172.21.15.7:0/3665313445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5587553fb510 0x558755406b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 >> 172.21.15.7:0/3665313445 conn(0x5587553201f0 msgr2=0x5587553ee7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.235+0000 7f31c8c00700 0 client.25598 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:57:57.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.236+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 shutdown_connections 2024-06-13T06:57:57.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.236+0000 7f31f0620b80 1 -- 172.21.15.7:0/3665313445 wait complete. 2024-06-13T06:57:57.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.236+0000 7f31f0620b80 20 client.25598 trim_cache size 0 max 16384 2024-06-13T06:57:57.237 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:57:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T06:57:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f7263072-14be-42e7-9cf0-3f7aa3e25e5b", 2024-06-13T06:57:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T06:57:57.237 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 msgr2=0x7f31c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 0x7f31c40a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f31d00138e0 tx=0x7f31d0013ca0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 msgr2=0x7f31e01a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 0x7f31e01a3aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b0a0 tx=0x7f31d8004d00 comp rx=0 tx=0).stop 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 shutdown_connections 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31e0108e00 0x7f31e01a3aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f31c40a2ca0 0x7f31c40a5160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31e006a790 0x7f31e019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 --2- 172.21.15.7:0/2653170023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e01084e0 0x7f31e019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:57:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.238+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 >> 172.21.15.7:0/2653170023 conn(0x7f31e0078340 msgr2=0x7f31e0106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:57:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.239+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 shutdown_connections 2024-06-13T06:57:57.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:57:57.239+0000 7f31e5c00700 1 -- 172.21.15.7:0/2653170023 wait complete. 2024-06-13T06:57:57.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:57 smithi007 ceph-mon[32077]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-06-13T06:57:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:57 smithi189 ceph-mon[36616]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-06-13T06:57:57.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:57 smithi175 ceph-mon[36886]: pgmap v566: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 611 op/s 2024-06-13T06:57:57.617 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 06:57:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T06:57:57.318+0000 7f97ae548700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a7 [...2,head] /volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494/ auth{1=2,2=1} fragtree_t(*^3) v4090 ap=2 f(v2 m2024-06-13T06:56:50.264644+0000 310=310+0) n(v10 rc2024-06-13T06:57:35.460922+0000 b528192047 311=310+1) (inest sync->mix g=1,2) caps={25141=pAsLsXsFs/pAsLsXsFsx@408} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55d84196bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a7:[<0x10000000014/13494 v1840>,<0x10000000006/base v351>,<0x10000000005/data v79>,<0x100000001fb/postgres v17>,<0x100000001fa/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 v33>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v39>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x100000004a7:[<0x10000000014/13494 v4090>,<0x10000000006/base v1222>,<0x10000000005/data v251>,<0x100000001fb/postgres v138>,<0x100000001fa/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5 v158>,<0x10000000001/sv_1 v174>,<0x10000000000/qa v163>,<0x1/volumes v164>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-13T06:56:03.163396+0000 309=309+0)","ondisk_value.rstat":"n(v0 rc2024-06-13T06:57:35.460922+0000 b528192047 311=310+1)","memory_value.dirstat":"f(v2 m2024-06-13T06:56:50.264644+0000 310=310+0)","memory_value.rstat":"n(v10 rc2024-06-13T06:57:35.460922+0000 b528192047 311=310+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-13T06:57:57.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 115.0 s, 1340.0 tps, lat 74.466 ms stddev 88.896 2024-06-13T06:57:58.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:57:58.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-06-13T06:57:58.565 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.565 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.565 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:58 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:58 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:57:58.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-06-13T06:57:58.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:58.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:58 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:57:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:57:59.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:57:59 smithi007 ceph-mon[32077]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-06-13T06:57:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:57:59 smithi189 ceph-mon[36616]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-06-13T06:57:59.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:57:59 smithi175 ceph-mon[36886]: pgmap v567: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.473+0000 7f6845400640 1 -- 172.21.15.7:0/1106291590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840101860 msgr2=0x7f6840101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.473+0000 7f6845400640 1 --2- 172.21.15.7:0/1106291590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840101860 0x7f6840101c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f682c009860 tx=0x7f682c02a340 comp rx=0 tx=0).stop 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.474+0000 7f6845400640 1 -- 172.21.15.7:0/1106291590 shutdown_connections 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.474+0000 7f6845400640 1 --2- 172.21.15.7:0/1106291590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840103410 0x7f6840109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.474+0000 7f6845400640 1 --2- 172.21.15.7:0/1106291590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.474+0000 7f6845400640 1 --2- 172.21.15.7:0/1106291590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840101860 0x7f6840101c80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.474+0000 7f6845400640 1 -- 172.21.15.7:0/1106291590 >> 172.21.15.7:0/1106291590 conn(0x7f68400fcdd0 msgr2=0x7f68400ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.475+0000 7f6845400640 1 -- 172.21.15.7:0/1106291590 shutdown_connections 2024-06-13T06:58:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.475+0000 7f6845400640 1 -- 172.21.15.7:0/1106291590 wait complete. 2024-06-13T06:58:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.480+0000 7f6845400640 1 Processor -- start 2024-06-13T06:58:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.481+0000 7f6845400640 1 -- start start 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 0x7f6840075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d500 con 0x7f6840102a50 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d380 con 0x7f6840103410 2024-06-13T06:58:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.482+0000 7f6845400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d680 con 0x7f6840101860 2024-06-13T06:58:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.483+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 0x7f6840075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.483+0000 7f6844a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.483+0000 7f6844a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59466/0 (socket says 172.21.15.7:59466) 2024-06-13T06:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.483+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 0x7f6840075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55918/0 (socket says 172.21.15.7:55918) 2024-06-13T06:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.483+0000 7f683fe00640 1 -- 172.21.15.7:0/2082040734 learned_addr learned my addr 172.21.15.7:0/2082040734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.484+0000 7f683f400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.485+0000 7f6844a00640 1 -- 172.21.15.7:0/2082040734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 msgr2=0x7f6840075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.485+0000 7f6844a00640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 0x7f6840075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.485+0000 7f6844a00640 1 -- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 msgr2=0x7f6840073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.485+0000 7f6844a00640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.485+0000 7f6844a00640 1 -- 172.21.15.7:0/2082040734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840074b60 con 0x7f6840103410 2024-06-13T06:58:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.487+0000 7f683f400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.487+0000 7f6844a00640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f683800ef00 tx=0x7f683800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.488+0000 7f683cc00640 1 -- 172.21.15.7:0/2082040734 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838004b20 con 0x7f6840103410 2024-06-13T06:58:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.488+0000 7f683cc00640 1 -- 172.21.15.7:0/2082040734 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6838003e50 con 0x7f6840103410 2024-06-13T06:58:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.488+0000 7f683cc00640 1 -- 172.21.15.7:0/2082040734 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800b570 con 0x7f6840103410 2024-06-13T06:58:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.488+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68401a1cc0 con 0x7f6840103410 2024-06-13T06:58:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.489+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68401a21c0 con 0x7f6840103410 2024-06-13T06:58:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.505+0000 7f6849819740 1 Processor -- start 2024-06-13T06:58:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.506+0000 7f683cc00640 1 -- 172.21.15.7:0/2082040734 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f683801d020 con 0x7f6840103410 2024-06-13T06:58:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.508+0000 7f683cc00640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 0x7f68240a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.509+0000 7f683f400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 0x7f68240a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.510+0000 7f683cc00640 1 -- 172.21.15.7:0/2082040734 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6838099730 con 0x7f6840103410 2024-06-13T06:58:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.511+0000 7f6849819740 1 -- start start 2024-06-13T06:58:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.511+0000 7f6849819740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c96904dc0 0x559c969051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.511+0000 7f683f400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 0x7f68240a51d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f68400754c0 tx=0x7f6830005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6849819740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c9690a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6849819740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6849819740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c96868420 con 0x559c96904dc0 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6849819740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c9680fc50 con 0x559c969057b0 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6849819740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c968643e0 con 0x559c9690a920 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6844a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6844a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55922/0 (socket says 172.21.15.7:55922) 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.512+0000 7f6844a00640 1 -- 172.21.15.7:0/1698621918 learned_addr learned my addr 172.21.15.7:0/1698621918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f683fe00640 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c96904dc0 0x559c969051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f6844a00640 1 -- 172.21.15.7:0/1698621918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 msgr2=0x559c9690a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f6844a00640 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c9690a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f6844a00640 1 -- 172.21.15.7:0/1698621918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c96904dc0 msgr2=0x559c969051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f6844a00640 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c96904dc0 0x559c969051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.513+0000 7f6844a00640 1 -- 172.21.15.7:0/1698621918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9657c210 con 0x559c9690a920 2024-06-13T06:58:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.515+0000 7f6844a00640 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6838005fd0 tx=0x7f68380a2d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8719869f1d71706 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.515+0000 7f6835200640 1 -- 172.21.15.7:0/1698621918 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380980c0 con 0x559c9690a920 2024-06-13T06:58:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.515+0000 7f6835200640 1 -- 172.21.15.7:0/1698621918 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f683801de50 con 0x559c9690a920 2024-06-13T06:58:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.515+0000 7f6835200640 1 -- 172.21.15.7:0/1698621918 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838098260 con 0x559c9690a920 2024-06-13T06:58:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.516+0000 7f6849819740 1 -- 172.21.15.7:0/1698621918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 msgr2=0x559c9690cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.516+0000 7f6849819740 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6838005fd0 tx=0x7f68380a2d50 comp rx=0 tx=0).stop 2024-06-13T06:58:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.517+0000 7f6849819740 1 -- 172.21.15.7:0/1698621918 shutdown_connections 2024-06-13T06:58:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.517+0000 7f6849819740 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c9690a920 0x559c9690cdb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.517+0000 7f6849819740 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c9690a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.517+0000 7f6849819740 1 --2- 172.21.15.7:0/1698621918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c96904dc0 0x559c969051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.517+0000 7f6849819740 1 -- 172.21.15.7:0/1698621918 >> 172.21.15.7:0/1698621918 conn(0x559c968ccfb0 msgr2=0x559c96900840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.518+0000 7f6849819740 1 -- 172.21.15.7:0/1698621918 shutdown_connections 2024-06-13T06:58:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.518+0000 7f6849819740 1 -- 172.21.15.7:0/1698621918 wait complete. 2024-06-13T06:58:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.519+0000 7f6849819740 1 -- start start 2024-06-13T06:58:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.519+0000 7f6849819740 1 Processor -- start 2024-06-13T06:58:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.520+0000 7f6849819740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c9690f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.520+0000 7f6849819740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 0x559c9690f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.520+0000 7f6849819740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 0x559c969166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f683f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 0x559c9690f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6849819740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c9690d840 con 0x559c969057b0 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6849819740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c968230b0 con 0x559c9690a920 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6849819740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c9690d9c0 con 0x559c96904dc0 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c9690f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c9690f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55932/0 (socket says 172.21.15.7:55932) 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f683fe00640 1 -- 172.21.15.7:0/272139158 learned_addr learned my addr 172.21.15.7:0/272139158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6844a00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 0x559c969166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6849819740 10 client.0 _ll_register_callbacks cb 0x559c9680ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.521+0000 7f6844a00640 1 -- 172.21.15.7:0/272139158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 msgr2=0x559c9690f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6844a00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c9690f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6844a00640 1 -- 172.21.15.7:0/272139158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 msgr2=0x559c9690f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6844a00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 0x559c9690f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6844a00640 1 -- 172.21.15.7:0/272139158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c96648620 con 0x559c9690a920 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f683f400640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 0x559c9690f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6844a00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 0x559c969166a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6838097af0 tx=0x7f683800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97837bd89463fd0a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.522+0000 7f6829600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.523+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380b3070 con 0x559c9690a920 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.523+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f68380a4280 con 0x559c9690a920 2024-06-13T06:58:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.524+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f683801de50 con 0x559c9690a920 2024-06-13T06:58:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.524+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68380ba050 con 0x559c9690a920 2024-06-13T06:58:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.525+0000 7f6849819740 10 client.25607 fetch_fsmap learned FSMap version 29 2024-06-13T06:58:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.525+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c9690d840 con 0x559c9690a920 2024-06-13T06:58:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.525+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f683801de50 con 0x559c9690a920 2024-06-13T06:58:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.527+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f683800b040 con 0x559c9690a920 2024-06-13T06:58:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.527+0000 7f6849819740 10 client.25607 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:58:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.527+0000 7f6849819740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.528+0000 7f6849819740 20 client.25607 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.528+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9691ab50 0x559c9691cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.529+0000 7f6849819740 4 client.25607 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.529+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c96875b50 con 0x559c9691ab50 2024-06-13T06:58:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.529+0000 7f683fe00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9691ab50 0x559c9691cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.530+0000 7f683fe00640 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9691ab50 0x559c9691cf40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9375e61274d25ba6 server_cookie=7acd3f0f069aad8b in_seq=0 out_seq=0 2024-06-13T06:58:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.530+0000 7f6829600640 10 client.25607 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.531+0000 7f6829600640 1 -- 172.21.15.7:0/272139158 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c96875b50 con 0x559c9691ab50 2024-06-13T06:58:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.531+0000 7f6829600640 10 client.25607 handle_command_reply: tid=0 2024-06-13T06:58:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.531+0000 7f6849819740 1 client.25607 shutdown 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 msgr2=0x559c969166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 0x559c969166a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6838097af0 tx=0x7f683800ccc0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 shutdown_connections 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9691ab50 0x559c9691cf40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c9690a920 0x559c969166a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c969057b0 0x559c9690f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 --2- 172.21.15.7:0/272139158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c9690f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 >> 172.21.15.7:0/272139158 conn(0x559c968ccfb0 msgr2=0x559c968ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.532+0000 7f6829600640 0 client.25607 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.533+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 shutdown_connections 2024-06-13T06:58:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.536+0000 7f6849819740 1 -- 172.21.15.7:0/272139158 wait complete. 2024-06-13T06:58:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.536+0000 7f6849819740 20 client.25607 trim_cache size 0 max 16384 2024-06-13T06:58:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 Processor -- start 2024-06-13T06:58:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 -- start start 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96916660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c969070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68401aab20 con 0x559c9690a920 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.569+0000 7f6849819740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a83c0 con 0x559c969057b0 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f6849819740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a6210 con 0x559c96904dc0 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96916660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55944/0 (socket says 172.21.15.7:55944) 2024-06-13T06:58:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683fe00640 1 -- 172.21.15.7:0/153891812 learned_addr learned my addr 172.21.15.7:0/153891812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f6844a00640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c969070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683f400640 1 -- 172.21.15.7:0/153891812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 msgr2=0x559c96906b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683f400640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.570+0000 7f683f400640 1 -- 172.21.15.7:0/153891812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 msgr2=0x559c969070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f683f400640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c969070b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f683f400640 1 -- 172.21.15.7:0/153891812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c96909e50 con 0x559c969057b0 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f683fe00640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f6844a00640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c969070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f683f400640 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96916660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x559c96905bb0 tx=0x7f6830004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84cafa5140c6222b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f682be00640 1 -- 172.21.15.7:0/153891812 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830022030 con 0x559c969057b0 2024-06-13T06:58:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.571+0000 7f682be00640 1 -- 172.21.15.7:0/153891812 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6830017030 con 0x559c969057b0 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 -- 172.21.15.7:0/153891812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 msgr2=0x559c96916660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96916660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x559c96905bb0 tx=0x7f6830004730 comp rx=0 tx=0).stop 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 -- 172.21.15.7:0/153891812 shutdown_connections 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c969070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96916660 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 --2- 172.21.15.7:0/153891812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96906b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 -- 172.21.15.7:0/153891812 >> 172.21.15.7:0/153891812 conn(0x559c968ccfb0 msgr2=0x559c9690f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.572+0000 7f6849819740 1 -- 172.21.15.7:0/153891812 shutdown_connections 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.573+0000 7f6849819740 1 -- 172.21.15.7:0/153891812 wait complete. 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.573+0000 7f6849819740 1 -- start start 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.573+0000 7f6849819740 1 Processor -- start 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96911a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c96994760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a6880 con 0x559c9690a920 2024-06-13T06:58:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a7cf0 con 0x559c969057b0 2024-06-13T06:58:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68401a6f50 con 0x559c96904dc0 2024-06-13T06:58:01.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.574+0000 7f6849819740 10 client.0 _ll_register_callbacks cb 0x7f68380669d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.575+0000 7f683f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96911a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.575+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.575+0000 7f683fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55952/0 (socket says 172.21.15.7:55952) 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.575+0000 7f683fe00640 1 -- 172.21.15.7:0/1408331692 learned_addr learned my addr 172.21.15.7:0/1408331692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f6844a00640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c96994760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 -- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 msgr2=0x559c96911a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96911a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 -- 172.21.15.7:0/1408331692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 msgr2=0x559c96994760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c96994760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 -- 172.21.15.7:0/1408331692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c969a5cb0 con 0x559c96904dc0 2024-06-13T06:58:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f683fe00640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f682c035040 tx=0x7f682c0094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c26a09b875bb6a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.576+0000 7f6829600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.577+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c037040 con 0x559c96904dc0 2024-06-13T06:58:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.577+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f682c030910 con 0x559c96904dc0 2024-06-13T06:58:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.577+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6830017030 con 0x559c96904dc0 2024-06-13T06:58:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.578+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f682c004040 con 0x559c96904dc0 2024-06-13T06:58:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.578+0000 7f6849819740 10 client.25612 fetch_fsmap learned FSMap version 29 2024-06-13T06:58:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.578+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c969a6fe0 con 0x559c96904dc0 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6830017030 con 0x559c96904dc0 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f682c04e030 con 0x559c96904dc0 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 10 client.25612 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 20 client.25612 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c969a7a00 0x559c969a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 4 client.25612 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:58:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.579+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c9686fea0 con 0x559c969a7a00 2024-06-13T06:58:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.580+0000 7f683f400640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c969a7a00 0x559c969a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.580+0000 7f6829600640 10 client.25612 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:01.580+0000 7f683f400640 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c969a7a00 0x559c969a9e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ce0a3ccacf934e server_cookie=e15c584b65e35e6 in_seq=0 out_seq=0 2024-06-13T06:58:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:01 smithi189 ceph-mon[36616]: pgmap v568: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-06-13T06:58:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:01 smithi175 ceph-mon[36886]: pgmap v568: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-06-13T06:58:01.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:01 smithi007 ceph-mon[32077]: pgmap v568: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-06-13T06:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.009+0000 7f6829600640 1 -- 172.21.15.7:0/1408331692 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559c9686fea0 con 0x559c969a7a00 2024-06-13T06:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.009+0000 7f6829600640 10 client.25612 handle_command_reply: tid=0 2024-06-13T06:58:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.009+0000 7f6849819740 1 client.25612 shutdown 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.009+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 msgr2=0x559c96911500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.009+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f682c035040 tx=0x7f682c0094a0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 shutdown_connections 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c969a7a00 0x559c969a9e20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c9690a920 0x559c96994760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c969057b0 0x559c96911a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 --2- 172.21.15.7:0/1408331692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c96904dc0 0x559c96911500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 >> 172.21.15.7:0/1408331692 conn(0x559c968ccfb0 msgr2=0x559c9690f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6829600640 0 client.25612 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 shutdown_connections 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 1 -- 172.21.15.7:0/1408331692 wait complete. 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.010+0000 7f6849819740 20 client.25612 trim_cache size 0 max 16384 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:58:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 msgr2=0x7f68240a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 0x7f68240a51d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f68400754c0 tx=0x7f6830005e50 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 msgr2=0x7f68400742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f683800ef00 tx=0x7f683800c740 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 shutdown_connections 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68240a2d10 0x7f68240a51d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6840103410 0x7f68400742d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6840102a50 0x7f6840073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 --2- 172.21.15.7:0/2082040734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6840101860 0x7f6840075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 >> 172.21.15.7:0/2082040734 conn(0x7f68400fcdd0 msgr2=0x7f6840105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 shutdown_connections 2024-06-13T06:58:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:02.013+0000 7f6845400640 1 -- 172.21.15.7:0/2082040734 wait complete. 2024-06-13T06:58:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 120.0 s, 1201.6 tps, lat 83.725 ms stddev 111.604 2024-06-13T06:58:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.397+0000 7f8115c00700 1 -- 172.21.15.7:0/1138434348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 msgr2=0x7f8110109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.397+0000 7f8115c00700 1 --2- 172.21.15.7:0/1138434348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f8110109ad0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8100009990 tx=0x7f8100009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.398+0000 7f8115c00700 1 -- 172.21.15.7:0/1138434348 shutdown_connections 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.398+0000 7f8115c00700 1 --2- 172.21.15.7:0/1138434348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010e220 0x7f8110115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.398+0000 7f8115c00700 1 --2- 172.21.15.7:0/1138434348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f8110109ad0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.398+0000 7f8115c00700 1 --2- 172.21.15.7:0/1138434348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010a0a0 0x7f811010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.398+0000 7f8115c00700 1 -- 172.21.15.7:0/1138434348 >> 172.21.15.7:0/1138434348 conn(0x7f811006d600 msgr2=0x7f811006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.399+0000 7f8115c00700 1 -- 172.21.15.7:0/1138434348 shutdown_connections 2024-06-13T06:58:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.399+0000 7f8115c00700 1 -- 172.21.15.7:0/1138434348 wait complete. 2024-06-13T06:58:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.401+0000 7f8115c00700 1 Processor -- start 2024-06-13T06:58:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.401+0000 7f8115c00700 1 -- start start 2024-06-13T06:58:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f81101a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010e220 0x7f811011a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8110119280 con 0x7f811010e220 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8110119100 con 0x7f811010a0a0 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f8115c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8110119400 con 0x7f81101096f0 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f810e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f810e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59510/0 (socket says 172.21.15.7:59510) 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f810e000700 1 -- 172.21.15.7:0/1112363645 learned_addr learned my addr 172.21.15.7:0/1112363645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.403+0000 7f810f400700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010e220 0x7f811011a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.404+0000 7f810ea00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f81101a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.405+0000 7f810e000700 1 -- 172.21.15.7:0/1112363645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 msgr2=0x7f81101a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.405+0000 7f810e000700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f81101a6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.405+0000 7f810e000700 1 -- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010e220 msgr2=0x7f811011a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.405+0000 7f810e000700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010e220 0x7f811011a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.405+0000 7f810e000700 1 -- 172.21.15.7:0/1112363645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811011b160 con 0x7f811010a0a0 2024-06-13T06:58:03.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:03 smithi007 ceph-mon[32077]: pgmap v569: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-06-13T06:58:03.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:03.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:03.406 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.407+0000 7f810e000700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8108007f10 tx=0x7f810800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.408+0000 7f8107400700 1 -- 172.21.15.7:0/1112363645 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810800ddd0 con 0x7f811010a0a0 2024-06-13T06:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.408+0000 7f8107400700 1 -- 172.21.15.7:0/1112363645 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8108018070 con 0x7f811010a0a0 2024-06-13T06:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.408+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81101a3b90 con 0x7f811010a0a0 2024-06-13T06:58:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.408+0000 7f8107400700 1 -- 172.21.15.7:0/1112363645 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810800ddd0 con 0x7f811010a0a0 2024-06-13T06:58:03.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.409+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81101a4080 con 0x7f811010a0a0 2024-06-13T06:58:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.412+0000 7f8107400700 1 -- 172.21.15.7:0/1112363645 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8108040b30 con 0x7f811010a0a0 2024-06-13T06:58:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.414+0000 7f8107400700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 0x7f80f40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.415+0000 7f8107400700 1 -- 172.21.15.7:0/1112363645 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f81080ce460 con 0x7f811010a0a0 2024-06-13T06:58:03.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.415+0000 7f810ea00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 0x7f80f40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.416+0000 7f810ea00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 0x7f80f40a5310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f81000098e0 tx=0x7f810000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.421+0000 7f812053eb80 1 Processor -- start 2024-06-13T06:58:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.421+0000 7f812053eb80 1 -- start start 2024-06-13T06:58:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.421+0000 7f812053eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.421+0000 7f812053eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 0x556573c3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.421+0000 7f812053eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 0x556573c3f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f812053eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556573b4da10 con 0x556573c44b90 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f812053eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556573b90b00 con 0x556573c454b0 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f812053eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556573b4b8f0 con 0x556573c3f240 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f810e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 0x556573c3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f810e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 0x556573c3ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59522/0 (socket says 172.21.15.7:59522) 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f810e000700 1 -- 172.21.15.7:0/3895528095 learned_addr learned my addr 172.21.15.7:0/3895528095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.422+0000 7f810f400700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 0x556573c3f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810ea00700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 -- 172.21.15.7:0/3895528095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 msgr2=0x556573c3ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 0x556573c3ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 -- 172.21.15.7:0/3895528095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 msgr2=0x556573c44f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c44f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 -- 172.21.15.7:0/3895528095 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556573b38f90 con 0x556573c3f240 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810ea00700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.423+0000 7f810f400700 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 0x556573c3f680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007b00 tx=0x7f80fc007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a060100f83ff44ad server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.424+0000 7f80fbe00700 1 -- 172.21.15.7:0/3895528095 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004310 con 0x556573c3f240 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.424+0000 7f80fbe00700 1 -- 172.21.15.7:0/3895528095 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc015070 con 0x556573c3f240 2024-06-13T06:58:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.424+0000 7f80fbe00700 1 -- 172.21.15.7:0/3895528095 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc01a470 con 0x556573c3f240 2024-06-13T06:58:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 -- 172.21.15.7:0/3895528095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 msgr2=0x556573c3f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 0x556573c3f680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007b00 tx=0x7f80fc007e10 comp rx=0 tx=0).stop 2024-06-13T06:58:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 -- 172.21.15.7:0/3895528095 shutdown_connections 2024-06-13T06:58:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c3f240 0x556573c3f680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 --2- 172.21.15.7:0/3895528095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c454b0 0x556573c3ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 -- 172.21.15.7:0/3895528095 >> 172.21.15.7:0/3895528095 conn(0x556573b6b170 msgr2=0x556573c39ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 -- 172.21.15.7:0/3895528095 shutdown_connections 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.425+0000 7f812053eb80 1 -- 172.21.15.7:0/3895528095 wait complete. 2024-06-13T06:58:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.426+0000 7f812053eb80 1 -- start start 2024-06-13T06:58:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.426+0000 7f812053eb80 1 Processor -- start 2024-06-13T06:58:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.427+0000 7f812053eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.427+0000 7f812053eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c48540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.427+0000 7f812053eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.427+0000 7f810e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c48540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.427+0000 7f810e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c48540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50972/0 (socket says 172.21.15.7:50972) 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f810ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f810f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f810f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55994/0 (socket says 172.21.15.7:55994) 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f810ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59532/0 (socket says 172.21.15.7:59532) 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f810e000700 1 -- 172.21.15.7:0/1688407028 learned_addr learned my addr 172.21.15.7:0/1688407028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556573b2cf60 con 0x556573c44b90 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556573b8cac0 con 0x556573c3f240 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.428+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556573c4be90 con 0x556573c454b0 2024-06-13T06:58:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f812053eb80 10 client.0 _ll_register_callbacks cb 0x556573b26380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810f400700 1 -- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 msgr2=0x556573c4b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810f400700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810f400700 1 -- 172.21.15.7:0/1688407028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 msgr2=0x556573c48540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810f400700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c48540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810f400700 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556573adf8d0 con 0x556573c454b0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.429+0000 7f810ea00700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.430+0000 7f810f400700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007b00 tx=0x7f80fc003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db31a601c45073e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.430+0000 7f80f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.430+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0055a0 con 0x556573c454b0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.430+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80fc0108c0 con 0x556573c454b0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.430+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc015070 con 0x556573c454b0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80fc030050 con 0x556573c454b0 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 10 client.25624 fetch_fsmap learned FSMap version 29 2024-06-13T06:58:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556573c565d0 con 0x556573c454b0 2024-06-13T06:58:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc015070 con 0x556573c454b0 2024-06-13T06:58:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f80fc02b030 con 0x556573c454b0 2024-06-13T06:58:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:58:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 20 client.25624 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.431+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c58aa0 0x556573c5ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.432+0000 7f812053eb80 4 client.25624 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.432+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556573b9e000 con 0x556573c58aa0 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.432+0000 7f810ea00700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c58aa0 0x556573c5ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.432+0000 7f810ea00700 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c58aa0 0x556573c5ae90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f12be4b52cc6852 server_cookie=ade1540111218ad6 in_seq=0 out_seq=0 2024-06-13T06:58:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.432+0000 7f80f8c00700 10 client.25624 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.433+0000 7f80f8c00700 1 -- 172.21.15.7:0/1688407028 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556573b9e000 con 0x556573c58aa0 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.433+0000 7f80f8c00700 10 client.25624 handle_command_reply: tid=0 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.433+0000 7f812053eb80 1 client.25624 shutdown 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 msgr2=0x556573c48a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007b00 tx=0x7f80fc003c00 comp rx=0 tx=0).stop 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 shutdown_connections 2024-06-13T06:58:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c58aa0 0x556573c5ae90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573c454b0 0x556573c48a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.435+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c3f240 0x556573c4b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.435+0000 7f812053eb80 1 --2- 172.21.15.7:0/1688407028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c44b90 0x556573c48540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.435+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 >> 172.21.15.7:0/1688407028 conn(0x556573b6b170 msgr2=0x556573c33790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.434+0000 7f80f8c00700 0 client.25624 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.435+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 shutdown_connections 2024-06-13T06:58:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.436+0000 7f812053eb80 1 -- 172.21.15.7:0/1688407028 wait complete. 2024-06-13T06:58:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.436+0000 7f812053eb80 20 client.25624 trim_cache size 0 max 16384 2024-06-13T06:58:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 Processor -- start 2024-06-13T06:58:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 -- start start 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81101c8170 con 0x556573c3f240 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81101ae980 con 0x556573c44b90 2024-06-13T06:58:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.476+0000 7f812053eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81101ac7d0 con 0x556573b6b170 2024-06-13T06:58:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.477+0000 7f810f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.477+0000 7f810e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.477+0000 7f810f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59548/0 (socket says 172.21.15.7:59548) 2024-06-13T06:58:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.477+0000 7f810e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50988/0 (socket says 172.21.15.7:50988) 2024-06-13T06:58:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.477+0000 7f810f400700 1 -- 172.21.15.7:0/3886567406 learned_addr learned my addr 172.21.15.7:0/3886567406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.479+0000 7f810ea00700 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810f400700 1 -- 172.21.15.7:0/3886567406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 msgr2=0x556573c3ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810f400700 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c3ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810f400700 1 -- 172.21.15.7:0/3886567406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 msgr2=0x556573c48c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810f400700 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810f400700 1 -- 172.21.15.7:0/3886567406 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556573c48310 con 0x556573c44b90 2024-06-13T06:58:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.480+0000 7f810e000700 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.481+0000 7f810f400700 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007700 tx=0x7f80fc0071c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5444bfe38c8d73 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.482+0000 7f80fb400700 1 -- 172.21.15.7:0/3886567406 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0053a0 con 0x556573c44b90 2024-06-13T06:58:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.483+0000 7f80fb400700 1 -- 172.21.15.7:0/3886567406 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc015040 con 0x556573c44b90 2024-06-13T06:58:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.483+0000 7f80fb400700 1 -- 172.21.15.7:0/3886567406 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc010d10 con 0x556573c44b90 2024-06-13T06:58:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.484+0000 7f812053eb80 1 -- 172.21.15.7:0/3886567406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 msgr2=0x556573c34130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.484+0000 7f812053eb80 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007700 tx=0x7f80fc0071c0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 -- 172.21.15.7:0/3886567406 shutdown_connections 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c34130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c3ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 --2- 172.21.15.7:0/3886567406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c48c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 -- 172.21.15.7:0/3886567406 >> 172.21.15.7:0/3886567406 conn(0x556573c3fbd0 msgr2=0x556573c48500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 -- 172.21.15.7:0/3886567406 shutdown_connections 2024-06-13T06:58:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.486+0000 7f812053eb80 1 -- 172.21.15.7:0/3886567406 wait complete. 2024-06-13T06:58:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.489+0000 7f812053eb80 1 -- start start 2024-06-13T06:58:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.489+0000 7f812053eb80 1 Processor -- start 2024-06-13T06:58:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81101ace40 con 0x556573c3f240 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81101ae2b0 con 0x556573c44b90 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f812053eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81101ad510 con 0x556573b6b170 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f810ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f810f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f810f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59556/0 (socket says 172.21.15.7:59556) 2024-06-13T06:58:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f810f400700 1 -- 172.21.15.7:0/4243600888 learned_addr learned my addr 172.21.15.7:0/4243600888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.490+0000 7f810ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c51390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56010/0 (socket says 172.21.15.7:56010) 2024-06-13T06:58:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.491+0000 7f810e000700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.491+0000 7f812053eb80 10 client.0 _ll_register_callbacks cb 0x556573b26380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 -- 172.21.15.7:0/4243600888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 msgr2=0x556573c51390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c51390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 -- 172.21.15.7:0/4243600888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 msgr2=0x556573c518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 -- 172.21.15.7:0/4243600888 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556573c5fef0 con 0x556573c44b90 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f810f400700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f80fc000c00 tx=0x7f80fc003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a41d563c751cf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.492+0000 7f80f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc01a470 con 0x556573c44b90 2024-06-13T06:58:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80fc01a610 con 0x556573c44b90 2024-06-13T06:58:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc015040 con 0x556573c44b90 2024-06-13T06:58:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80fc04d050 con 0x556573c44b90 2024-06-13T06:58:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f812053eb80 10 client.25631 fetch_fsmap learned FSMap version 29 2024-06-13T06:58:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.493+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556573cc2d30 con 0x556573c44b90 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc015040 con 0x556573c44b90 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f80fc02b040 con 0x556573c44b90 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 10 client.25631 fetch_fsmap finished waiting for FSMap version 29 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 20 client.25631 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c454b0 0x556573cc53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 4 client.25631 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.494+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556573b98580 con 0x556573c454b0 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.495+0000 7f810ea00700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c454b0 0x556573cc53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f810ea00700 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c454b0 0x556573cc53e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4c4b114c790ee2 server_cookie=bb69162ff2bdc74d in_seq=0 out_seq=0 2024-06-13T06:58:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f80f8c00700 10 client.25631 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f80f8c00700 1 -- 172.21.15.7:0/4243600888 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556573b98580 con 0x556573c454b0 2024-06-13T06:58:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f80f8c00700 10 client.25631 handle_command_reply: tid=0 2024-06-13T06:58:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f812053eb80 1 client.25631 shutdown 2024-06-13T06:58:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 msgr2=0x556573c5f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.496+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f80fc000c00 tx=0x7f80fc003680 comp rx=0 tx=0).stop 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 shutdown_connections 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556573c454b0 0x556573cc53e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556573c44b90 0x556573c5f8b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556573b6b170 0x556573c51390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 --2- 172.21.15.7:0/4243600888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556573c3f240 0x556573c518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 >> 172.21.15.7:0/4243600888 conn(0x556573c3fbd0 msgr2=0x556573c45890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f80f8c00700 0 client.25631 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 shutdown_connections 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 1 -- 172.21.15.7:0/4243600888 wait complete. 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.497+0000 7f812053eb80 20 client.25631 trim_cache size 0 max 16384 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T06:58:03.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 msgr2=0x7f80f40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 0x7f80f40a5310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f81000098e0 tx=0x7f810000b540 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 msgr2=0x7f811011a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8108007f10 tx=0x7f810800d3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 shutdown_connections 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811010e220 0x7f811011a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.499+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81101096f0 0x7f81101a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.500+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80f40a2e50 0x7f80f40a5310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.500+0000 7f8115c00700 1 --2- 172.21.15.7:0/1112363645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f811010a0a0 0x7f811011a370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.500+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 >> 172.21.15.7:0/1112363645 conn(0x7f811006d600 msgr2=0x7f8110073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.500+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 shutdown_connections 2024-06-13T06:58:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:03.500+0000 7f8115c00700 1 -- 172.21.15.7:0/1112363645 wait complete. 2024-06-13T06:58:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:03 smithi189 ceph-mon[36616]: pgmap v569: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-06-13T06:58:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:03 smithi175 ceph-mon[36886]: pgmap v569: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-06-13T06:58:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:04.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T06:58:04.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:58:05.168 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:05 smithi189 ceph-mon[36616]: pgmap v570: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-06-13T06:58:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:05 smithi175 ceph-mon[36886]: pgmap v570: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-06-13T06:58:05.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:05 smithi007 ceph-mon[32077]: pgmap v570: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.5 MiB/s wr, 757 op/s 2024-06-13T06:58:07.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:07 smithi189 ceph-mon[36616]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.9 MiB/s wr, 810 op/s 2024-06-13T06:58:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:07 smithi175 ceph-mon[36886]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.9 MiB/s wr, 810 op/s 2024-06-13T06:58:07.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:07 smithi007 ceph-mon[32077]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.9 MiB/s wr, 810 op/s 2024-06-13T06:58:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 125.0 s, 1319.8 tps, lat 75.067 ms stddev 85.549 2024-06-13T06:58:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:09 smithi189 ceph-mon[36616]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 818 op/s 2024-06-13T06:58:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:09 smithi175 ceph-mon[36886]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 818 op/s 2024-06-13T06:58:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:09 smithi007 ceph-mon[32077]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.3 MiB/s wr, 818 op/s 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.285+0000 7f60e2400700 1 -- 172.21.15.7:0/2855717025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 msgr2=0x7f60dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.285+0000 7f60e2400700 1 --2- 172.21.15.7:0/2855717025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc074aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0098f0 tx=0x7f60cc009c00 comp rx=0 tx=0).stop 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.286+0000 7f60e2400700 1 -- 172.21.15.7:0/2855717025 shutdown_connections 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.286+0000 7f60e2400700 1 --2- 172.21.15.7:0/2855717025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc10eef0 0x7f60dc1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.286+0000 7f60e2400700 1 --2- 172.21.15.7:0/2855717025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc074aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.286+0000 7f60e2400700 1 --2- 172.21.15.7:0/2855717025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc074fe0 0x7f60dc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.286+0000 7f60e2400700 1 -- 172.21.15.7:0/2855717025 >> 172.21.15.7:0/2855717025 conn(0x7f60dc0795f0 msgr2=0x7f60dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.287+0000 7f60e2400700 1 -- 172.21.15.7:0/2855717025 shutdown_connections 2024-06-13T06:58:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.287+0000 7f60e2400700 1 -- 172.21.15.7:0/2855717025 wait complete. 2024-06-13T06:58:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.288+0000 7f60e2400700 1 Processor -- start 2024-06-13T06:58:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.288+0000 7f60e2400700 1 -- start start 2024-06-13T06:58:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 0x7f60dc102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc10eef0 0x7f60dc103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc114a40 con 0x7f60dc10eef0 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1148c0 con 0x7f60dc0746c0 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60e2400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc114bc0 con 0x7f60dc074fe0 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60db400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60db400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59568/0 (socket says 172.21.15.7:59568) 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60db400700 1 -- 172.21.15.7:0/2613301924 learned_addr learned my addr 172.21.15.7:0/2613301924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60daa00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 0x7f60dc102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.289+0000 7f60dbe00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc10eef0 0x7f60dc103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 -- 172.21.15.7:0/2613301924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 msgr2=0x7f60dc102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 -- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc10eef0 msgr2=0x7f60dc103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc10eef0 0x7f60dc103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 -- 172.21.15.7:0/2613301924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc19ad10 con 0x7f60dc074fe0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60db400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.290+0000 7f60daa00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 0x7f60dc102b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.291+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004220 con 0x7f60dc074fe0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.291+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c80043c0 con 0x7f60dc074fe0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.291+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8003ce0 con 0x7f60dc074fe0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.292+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc19b000 con 0x7f60dc074fe0 2024-06-13T06:58:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.292+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc19b4c0 con 0x7f60dc074fe0 2024-06-13T06:58:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.295+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc075da0 con 0x7f60dc074fe0 2024-06-13T06:58:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.297+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f60c800c7c0 con 0x7f60dc074fe0 2024-06-13T06:58:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.299+0000 7f60d3e00700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 0x7f60c00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.299+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f60c80cf550 con 0x7f60dc074fe0 2024-06-13T06:58:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.301+0000 7f60db400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 0x7f60c00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.303+0000 7f60db400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 0x7f60c00a5190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60cc005e90 tx=0x7f60cc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.304+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60c809c170 con 0x7f60dc074fe0 2024-06-13T06:58:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.777+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60dc19b7b0 con 0x7f60dc074fe0 2024-06-13T06:58:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:58:10.781 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:58:10.063919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24521,24527]},"id":1}]} 2024-06-13T06:58:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.778+0000 7f60d3e00700 1 -- 172.21.15.7:0/2613301924 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11871 (secure 0 0 0) 0x7f60c809ba70 con 0x7f60dc074fe0 2024-06-13T06:58:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.783+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 msgr2=0x7f60c00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.783+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 0x7f60c00a5190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f60cc005e90 tx=0x7f60cc019040 comp rx=0 tx=0).stop 2024-06-13T06:58:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.783+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 msgr2=0x7f60dc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.783+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 0x7f60dc102b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60c8005fd0 tx=0x7f60c800eca0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 shutdown_connections 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60dc10eef0 0x7f60dc103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f60c00a2cd0 0x7f60c00a5190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60dc0746c0 0x7f60dc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 --2- 172.21.15.7:0/2613301924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60dc074fe0 0x7f60dc102b50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.784+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 >> 172.21.15.7:0/2613301924 conn(0x7f60dc0795f0 msgr2=0x7f60dc0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.785+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 shutdown_connections 2024-06-13T06:58:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:10.785+0000 7f60e2400700 1 -- 172.21.15.7:0/2613301924 wait complete. 2024-06-13T06:58:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-06-13T06:58:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:11 smithi189 ceph-mon[36616]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 822 op/s 2024-06-13T06:58:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:11 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:58:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:11 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2613301924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:11 smithi175 ceph-mon[36886]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 822 op/s 2024-06-13T06:58:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:11 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:58:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:11 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2613301924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:11 smithi007 ceph-mon[32077]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 822 op/s 2024-06-13T06:58:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:11 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:58:11.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:11 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2613301924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:58:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 130.0 s, 1377.4 tps, lat 72.910 ms stddev 83.920 2024-06-13T06:58:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:13 smithi007 ceph-mon[32077]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.2 MiB/s wr, 798 op/s 2024-06-13T06:58:13.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T06:58:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:13 smithi189 ceph-mon[36616]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.2 MiB/s wr, 798 op/s 2024-06-13T06:58:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:13 smithi175 ceph-mon[36886]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.2 MiB/s wr, 798 op/s 2024-06-13T06:58:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 -- 172.21.15.7:0/1793436391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 msgr2=0x7fde3010bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 --2- 172.21.15.7:0/1793436391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde3010bd30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde28009ca0 tx=0x7fde2802a970 comp rx=0 tx=0).stop 2024-06-13T06:58:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 -- 172.21.15.7:0/1793436391 shutdown_connections 2024-06-13T06:58:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 --2- 172.21.15.7:0/1793436391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde3010bd30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 --2- 172.21.15.7:0/1793436391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde300743a0 0x7fde30074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 --2- 172.21.15.7:0/1793436391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde30075980 0x7fde30073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.211+0000 7fde38600640 1 -- 172.21.15.7:0/1793436391 >> 172.21.15.7:0/1793436391 conn(0x7fde300fcf90 msgr2=0x7fde300ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.212+0000 7fde38600640 1 -- 172.21.15.7:0/1793436391 shutdown_connections 2024-06-13T06:58:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.212+0000 7fde38600640 1 -- 172.21.15.7:0/1793436391 wait complete. 2024-06-13T06:58:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.213+0000 7fde38600640 1 Processor -- start 2024-06-13T06:58:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.214+0000 7fde38600640 1 -- start start 2024-06-13T06:58:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.215+0000 7fde38600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde300743a0 0x7fde3006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.215+0000 7fde38600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde30075980 0x7fde3006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.215+0000 7fde38600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde38600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde30102610 con 0x7fde300743a0 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde38600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde30102490 con 0x7fde30109890 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde38600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde30102790 con 0x7fde30075980 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34822/0 (socket says 172.21.15.7:34822) 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 -- 172.21.15.7:0/1698738214 learned_addr learned my addr 172.21.15.7:0/1698738214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde35c00640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde300743a0 0x7fde3006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 -- 172.21.15.7:0/1698738214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde30075980 msgr2=0x7fde3006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde30075980 0x7fde3006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 -- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde300743a0 msgr2=0x7fde3006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde300743a0 0x7fde3006da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.216+0000 7fde36600640 1 -- 172.21.15.7:0/1698738214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde300736a0 con 0x7fde30109890 2024-06-13T06:58:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.217+0000 7fde36600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde28009dc0 tx=0x7fde2802f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.217+0000 7fde2e000640 1 -- 172.21.15.7:0/1698738214 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28047070 con 0x7fde30109890 2024-06-13T06:58:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.217+0000 7fde2e000640 1 -- 172.21.15.7:0/1698738214 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde28030b50 con 0x7fde30109890 2024-06-13T06:58:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.217+0000 7fde2e000640 1 -- 172.21.15.7:0/1698738214 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28034070 con 0x7fde30109890 2024-06-13T06:58:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.217+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde30073930 con 0x7fde30109890 2024-06-13T06:58:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.218+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301aef70 con 0x7fde30109890 2024-06-13T06:58:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.221+0000 7fde3b37c740 1 Processor -- start 2024-06-13T06:58:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.223+0000 7fde2e000640 1 -- 172.21.15.7:0/1698738214 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fde2804a070 con 0x7fde30109890 2024-06-13T06:58:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.223+0000 7fde3b37c740 1 -- start start 2024-06-13T06:58:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293cc760 0x560b293cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b2932a420 con 0x560b293cc760 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b292d1c50 con 0x560b293c75f0 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde3b37c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b293263e0 con 0x560b293c6c00 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde35200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde35200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35236/0 (socket says 172.21.15.7:35236) 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde35200640 1 -- 172.21.15.7:0/3954868613 learned_addr learned my addr 172.21.15.7:0/3954868613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.224+0000 7fde35c00640 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 -- 172.21.15.7:0/3954868613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 msgr2=0x560b293cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 -- 172.21.15.7:0/3954868613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293cc760 msgr2=0x560b293cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293cc760 0x560b293cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 -- 172.21.15.7:0/3954868613 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2903e210 con 0x560b293c6c00 2024-06-13T06:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.225+0000 7fde35200640 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00bd50 tx=0x7fde1c00d420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b27874c33ab7b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.226+0000 7fde2e000640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 0x7fde180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.226+0000 7fde22a00640 1 -- 172.21.15.7:0/3954868613 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c019070 con 0x560b293c6c00 2024-06-13T06:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.226+0000 7fde22a00640 1 -- 172.21.15.7:0/3954868613 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1c004d50 con 0x560b293c6c00 2024-06-13T06:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.226+0000 7fde22a00640 1 -- 172.21.15.7:0/3954868613 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c014680 con 0x560b293c6c00 2024-06-13T06:58:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.226+0000 7fde2e000640 1 -- 172.21.15.7:0/1698738214 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fde280f0290 con 0x7fde30109890 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde35c00640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 0x7fde180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 -- 172.21.15.7:0/3954868613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 msgr2=0x560b293c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00bd50 tx=0x7fde1c00d420 comp rx=0 tx=0).stop 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 -- 172.21.15.7:0/3954868613 shutdown_connections 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293cc760 0x560b293cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3954868613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293c6c00 0x560b293c7020 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 -- 172.21.15.7:0/3954868613 >> 172.21.15.7:0/3954868613 conn(0x560b2938efb0 msgr2=0x560b293c2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.228+0000 7fde3b37c740 1 -- 172.21.15.7:0/3954868613 shutdown_connections 2024-06-13T06:58:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.229+0000 7fde3b37c740 1 -- 172.21.15.7:0/3954868613 wait complete. 2024-06-13T06:58:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.229+0000 7fde35c00640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 0x7fde180a50f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560b293c79f0 tx=0x7fde24009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.229+0000 7fde3b37c740 1 -- start start 2024-06-13T06:58:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.229+0000 7fde3b37c740 1 Processor -- start 2024-06-13T06:58:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560b293cf710 con 0x560b293c6c00 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560b292e50b0 con 0x560b293c75f0 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b293cf890 con 0x560b293cc760 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde36600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34840/0 (socket says 172.21.15.7:34840) 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde35200640 1 -- 172.21.15.7:0/2758654999 learned_addr learned my addr 172.21.15.7:0/2758654999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde35c00640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde3b37c740 10 client.0 _ll_register_callbacks cb 0x560b292d0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde36600640 1 -- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 msgr2=0x560b293d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde36600640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde36600640 1 -- 172.21.15.7:0/2758654999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 msgr2=0x560b293d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.231+0000 7fde36600640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde36600640 1 -- 172.21.15.7:0/2758654999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2910a620 con 0x560b293cc760 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde36600640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d8510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde280bc6e0 tx=0x7fde280bc350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ae724de6eb4f84 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde17400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280bce50 con 0x560b293cc760 2024-06-13T06:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fde280041d0 con 0x560b293cc760 2024-06-13T06:58:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.232+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c004d50 con 0x560b293cc760 2024-06-13T06:58:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.233+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde280bc750 con 0x560b293cc760 2024-06-13T06:58:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.233+0000 7fde3b37c740 10 client.25639 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.234+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b293cf710 con 0x560b293cc760 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.234+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1c004d50 con 0x560b293cc760 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.234+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fde280a5ce0 con 0x560b293cc760 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 10 client.25639 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 20 client.25639 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b293dcfc0 0x560b293df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 4 client.25639 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b29337b50 con 0x560b293dcfc0 2024-06-13T06:58:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.235+0000 7fde35c00640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b293dcfc0 0x560b293df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.236+0000 7fde17400640 10 client.25639 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.238+0000 7fde35c00640 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b293dcfc0 0x560b293df400 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc972ec520bf6b9 server_cookie=a3c68007eae78dd9 in_seq=0 out_seq=0 2024-06-13T06:58:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.239+0000 7fde17400640 1 -- 172.21.15.7:0/2758654999 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b29337b50 con 0x560b293dcfc0 2024-06-13T06:58:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.239+0000 7fde17400640 10 client.25639 handle_command_reply: tid=0 2024-06-13T06:58:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.239+0000 7fde3b37c740 1 client.25639 shutdown 2024-06-13T06:58:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 msgr2=0x560b293d8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d8510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde280bc6e0 tx=0x7fde280bc350 comp rx=0 tx=0).stop 2024-06-13T06:58:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 shutdown_connections 2024-06-13T06:58:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b293dcfc0 0x560b293df400 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde17400640 0 client.25639 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d8510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2758654999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 >> 172.21.15.7:0/2758654999 conn(0x560b2938efb0 msgr2=0x560b293c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 shutdown_connections 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.240+0000 7fde3b37c740 1 -- 172.21.15.7:0/2758654999 wait complete. 2024-06-13T06:58:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.241+0000 7fde3b37c740 20 client.25639 trim_cache size 0 max 16384 2024-06-13T06:58:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.267+0000 7fde3b37c740 1 Processor -- start 2024-06-13T06:58:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.267+0000 7fde3b37c740 1 -- start start 2024-06-13T06:58:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 0x560b293bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c75f0 0x560b293d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde301bbc10 con 0x560b293c75f0 2024-06-13T06:58:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde301b94e0 con 0x560b293c6c00 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde3b37c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde301b7330 con 0x560b293cc760 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde35200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c75f0 0x560b293d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde36600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde36600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35256/0 (socket says 172.21.15.7:35256) 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde36600640 1 -- 172.21.15.7:0/2217653804 learned_addr learned my addr 172.21.15.7:0/2217653804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.268+0000 7fde35c00640 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 0x560b293bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 -- 172.21.15.7:0/2217653804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 msgr2=0x560b293bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 0x560b293bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 -- 172.21.15.7:0/2217653804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c75f0 msgr2=0x560b293d84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c75f0 0x560b293d84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 -- 172.21.15.7:0/2217653804 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b293dfa80 con 0x560b293cc760 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde35c00640 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 0x560b293bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde36600640 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde280bc450 tx=0x7fde2802a970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf140b2f1d87ad7c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde22000640 1 -- 172.21.15.7:0/2217653804 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280bbea0 con 0x560b293cc760 2024-06-13T06:58:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde22000640 1 -- 172.21.15.7:0/2217653804 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde280bce50 con 0x560b293cc760 2024-06-13T06:58:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.269+0000 7fde22000640 1 -- 172.21.15.7:0/2217653804 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280b84a0 con 0x560b293cc760 2024-06-13T06:58:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.270+0000 7fde3b37c740 1 -- 172.21.15.7:0/2217653804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 msgr2=0x560b293c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.270+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde280bc450 tx=0x7fde2802a970 comp rx=0 tx=0).stop 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 -- 172.21.15.7:0/2217653804 shutdown_connections 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293c5570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c75f0 0x560b293d84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 --2- 172.21.15.7:0/2217653804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c6c00 0x560b293bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 -- 172.21.15.7:0/2217653804 >> 172.21.15.7:0/2217653804 conn(0x560b2938efb0 msgr2=0x560b293c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 -- 172.21.15.7:0/2217653804 shutdown_connections 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 -- 172.21.15.7:0/2217653804 wait complete. 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.271+0000 7fde3b37c740 1 -- start start 2024-06-13T06:58:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 Processor -- start 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde301b79a0 con 0x560b293c6c00 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde301b8e10 con 0x560b293c75f0 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde301b8070 con 0x560b293cc760 2024-06-13T06:58:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.272+0000 7fde3b37c740 10 client.0 _ll_register_callbacks cb 0x7fde28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde36600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde36600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35258/0 (socket says 172.21.15.7:35258) 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34858/0 (socket says 172.21.15.7:34858) 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde35200640 1 -- 172.21.15.7:0/3290820793 learned_addr learned my addr 172.21.15.7:0/3290820793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.273+0000 7fde35c00640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 -- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 msgr2=0x560b293d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 -- 172.21.15.7:0/3290820793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 msgr2=0x560b293d5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 -- 172.21.15.7:0/3290820793 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b293d7830 con 0x560b293cc760 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde36600640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde28043000 tx=0x7fde280bac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff50b2ca18da471 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.274+0000 7fde17400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.275+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28085020 con 0x560b293cc760 2024-06-13T06:58:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.275+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fde280b84a0 con 0x560b293cc760 2024-06-13T06:58:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.275+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde280bce50 con 0x560b293cc760 2024-06-13T06:58:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.277+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde280bbea0 con 0x560b293cc760 2024-06-13T06:58:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.277+0000 7fde3b37c740 10 client.25645 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.277+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b293d7830 con 0x560b293cc760 2024-06-13T06:58:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde280bce50 con 0x560b293cc760 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fde280a5ce0 con 0x560b293cc760 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 10 client.25645 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 20 client.25645 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b29448310 0x560b2944a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 4 client.25645 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:58:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.278+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b293bddb0 con 0x560b29448310 2024-06-13T06:58:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.279+0000 7fde35c00640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b29448310 0x560b2944a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.279+0000 7fde35c00640 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b29448310 0x560b2944a730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c8a10b8c68bcfa server_cookie=b7c9cf216556cde5 in_seq=0 out_seq=0 2024-06-13T06:58:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.279+0000 7fde17400640 10 client.25645 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde17400640 1 -- 172.21.15.7:0/3290820793 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560b293bddb0 con 0x560b29448310 2024-06-13T06:58:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde17400640 10 client.25645 handle_command_reply: tid=0 2024-06-13T06:58:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 client.25645 shutdown 2024-06-13T06:58:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 msgr2=0x560b293d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde28043000 tx=0x7fde280bac60 comp rx=0 tx=0).stop 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 shutdown_connections 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560b29448310 0x560b2944a730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde17400640 0 client.25645 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b293cc760 0x560b293d7240 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560b293c75f0 0x560b293d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 --2- 172.21.15.7:0/3290820793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560b293c6c00 0x560b293d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 >> 172.21.15.7:0/3290820793 conn(0x560b2938efb0 msgr2=0x560b293c5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 shutdown_connections 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.296+0000 7fde3b37c740 1 -- 172.21.15.7:0/3290820793 wait complete. 2024-06-13T06:58:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.297+0000 7fde3b37c740 20 client.25645 trim_cache size 0 max 16384 2024-06-13T06:58:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:58:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:58:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:58:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.297+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 msgr2=0x7fde180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.297+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 0x7fde180a50f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x560b293c79f0 tx=0x7fde24009110 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.297+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 msgr2=0x7fde30073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.297+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde28009dc0 tx=0x7fde2802f3c0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 shutdown_connections 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde180a2c30 0x7fde180a50f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde30109890 0x7fde30073030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde30075980 0x7fde3006dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 --2- 172.21.15.7:0/1698738214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde300743a0 0x7fde3006da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 >> 172.21.15.7:0/1698738214 conn(0x7fde300fcf90 msgr2=0x7fde30102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 shutdown_connections 2024-06-13T06:58:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:15.298+0000 7fde38600640 1 -- 172.21.15.7:0/1698738214 wait complete. 2024-06-13T06:58:15.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:15 smithi007 ceph-mon[32077]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-06-13T06:58:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:15 smithi189 ceph-mon[36616]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-06-13T06:58:15.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:15 smithi175 ceph-mon[36886]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-06-13T06:58:16.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.498+0000 7f9a6cc00700 1 -- 172.21.15.7:0/704389217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 msgr2=0x7f9a6810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.498+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/704389217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810aa50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009990 tx=0x7f9a58009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 -- 172.21.15.7:0/704389217 shutdown_connections 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/704389217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a68104f80 0x7f9a68111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/704389217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810aa50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/704389217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a681045c0 0x7f9a68104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 -- 172.21.15.7:0/704389217 >> 172.21.15.7:0/704389217 conn(0x7f9a681000d0 msgr2=0x7f9a681024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 -- 172.21.15.7:0/704389217 shutdown_connections 2024-06-13T06:58:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.499+0000 7f9a6cc00700 1 -- 172.21.15.7:0/704389217 wait complete. 2024-06-13T06:58:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.500+0000 7f9a6cc00700 1 Processor -- start 2024-06-13T06:58:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.500+0000 7f9a6cc00700 1 -- start start 2024-06-13T06:58:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 0x7f9a681111e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a68104f80 0x7f9a6810e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a67400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 0x7f9a681111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a67400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 0x7f9a681111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34866/0 (socket says 172.21.15.7:34866) 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a67400700 1 -- 172.21.15.7:0/1039308151 learned_addr learned my addr 172.21.15.7:0/1039308151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68114970 con 0x7f9a68104f80 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a681147f0 con 0x7f9a681045c0 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.501+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68114af0 con 0x7f9a6810a670 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a68104f80 0x7f9a6810e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 -- 172.21.15.7:0/1039308151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 msgr2=0x7f9a681111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 0x7f9a681111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 -- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a68104f80 msgr2=0x7f9a6810e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a68104f80 0x7f9a6810e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6810f120 con 0x7f9a6810a670 2024-06-13T06:58:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.502+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810e820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000efc0 tx=0x7f9a6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.503+0000 7f9a5fe00700 1 -- 172.21.15.7:0/1039308151 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600092e0 con 0x7f9a6810a670 2024-06-13T06:58:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.503+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a681a0ef0 con 0x7f9a6810a670 2024-06-13T06:58:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.503+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a14b0 con 0x7f9a6810a670 2024-06-13T06:58:16.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.503+0000 7f9a5fe00700 1 -- 172.21.15.7:0/1039308151 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a60004bb0 con 0x7f9a6810a670 2024-06-13T06:58:16.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.503+0000 7f9a5fe00700 1 -- 172.21.15.7:0/1039308151 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60004280 con 0x7f9a6810a670 2024-06-13T06:58:16.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.511+0000 7f9a5fe00700 1 -- 172.21.15.7:0/1039308151 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9a6001b030 con 0x7f9a6810a670 2024-06-13T06:58:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.514+0000 7f9a78c58b80 1 Processor -- start 2024-06-13T06:58:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.514+0000 7f9a78c58b80 1 -- start start 2024-06-13T06:58:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be9047420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9047de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.514+0000 7f9a5fe00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 0x7f9a4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564be8f54a10 con 0x564be9047960 2024-06-13T06:58:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564be8f97b00 con 0x564be9050570 2024-06-13T06:58:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564be8f528f0 con 0x564be8fc1e40 2024-06-13T06:58:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a5fe00700 1 -- 172.21.15.7:0/1039308151 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9a600ced70 con 0x7f9a6810a670 2024-06-13T06:58:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34872/0 (socket says 172.21.15.7:34872) 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 -- 172.21.15.7:0/1254685625 learned_addr learned my addr 172.21.15.7:0/1254685625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 -- 172.21.15.7:0/1254685625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 msgr2=0x564be9047420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be9047420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.515+0000 7f9a67e00700 1 -- 172.21.15.7:0/1254685625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 msgr2=0x564be9047de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.516+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9047de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.516+0000 7f9a67e00700 1 -- 172.21.15.7:0/1254685625 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be8f3ff90 con 0x564be9050570 2024-06-13T06:58:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.516+0000 7f9a67e00700 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a60098cd0 tx=0x7f9a60098620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a761ef3182574c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.517+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 0x7f9a4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.517+0000 7f9a5cc00700 1 -- 172.21.15.7:0/1254685625 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6009bc60 con 0x564be9050570 2024-06-13T06:58:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.517+0000 7f9a5cc00700 1 -- 172.21.15.7:0/1254685625 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a60012340 con 0x564be9050570 2024-06-13T06:58:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.517+0000 7f9a5cc00700 1 -- 172.21.15.7:0/1254685625 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60064510 con 0x564be9050570 2024-06-13T06:58:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.518+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 0x7f9a4c0a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x564be9042590 tx=0x7f9a54006df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.518+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1254685625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 msgr2=0x564be9052960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.518+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a60098cd0 tx=0x7f9a60098620 comp rx=0 tx=0).stop 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1254685625 shutdown_connections 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9050570 0x564be9052960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be9047420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1254685625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9047de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1254685625 >> 172.21.15.7:0/1254685625 conn(0x564be8f72170 msgr2=0x564be9041820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.519+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1254685625 shutdown_connections 2024-06-13T06:58:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.520+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1254685625 wait complete. 2024-06-13T06:58:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.520+0000 7f9a78c58b80 1 -- start start 2024-06-13T06:58:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 Processor -- start 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be904fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9050570 0x564be904d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564be8f33f60 con 0x564be9050570 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564be8f93ac0 con 0x564be9047960 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.521+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564be90531e0 con 0x564be8fc1e40 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a78c58b80 10 client.0 _ll_register_callbacks cb 0x564be8f31720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-06-13T06:58:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 -- 172.21.15.7:0/1403829838 learned_addr learned my addr 172.21.15.7:0/1403829838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 -- 172.21.15.7:0/1403829838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 msgr2=0x564be904fc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be904fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 -- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9050570 msgr2=0x564be904d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9050570 0x564be904d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.522+0000 7f9a66a00700 1 -- 172.21.15.7:0/1403829838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be8ee68d0 con 0x564be9047960 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.523+0000 7f9a66a00700 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9a540136a0 tx=0x7f9a54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e828dccb04cf73f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.523+0000 7f9a51600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.523+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54022030 con 0x564be9047960 2024-06-13T06:58:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.523+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a60012340 con 0x564be9047960 2024-06-13T06:58:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.524+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9a54023070 con 0x564be9047960 2024-06-13T06:58:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.524+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a5402c050 con 0x564be9047960 2024-06-13T06:58:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.524+0000 7f9a78c58b80 10 client.25667 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.524+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564be905d9a0 con 0x564be9047960 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a60012340 con 0x564be9047960 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a54027400 con 0x564be9047960 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 10 client.25667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 20 client.25667 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be905ff80 0x564be9062400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 4 client.25667 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be8fa5000 con 0x564be905ff80 2024-06-13T06:58:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.525+0000 7f9a67400700 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be905ff80 0x564be9062400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.527+0000 7f9a51600700 10 client.25667 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:58:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.527+0000 7f9a67400700 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be905ff80 0x564be9062400 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23b5dc076229d6f3 server_cookie=a90c5f4532569389 in_seq=0 out_seq=0 2024-06-13T06:58:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.529+0000 7f9a51600700 1 -- 172.21.15.7:0/1403829838 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564be8fa5000 con 0x564be905ff80 2024-06-13T06:58:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.529+0000 7f9a51600700 10 client.25667 handle_command_reply: tid=0 2024-06-13T06:58:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.529+0000 7f9a78c58b80 1 client.25667 shutdown 2024-06-13T06:58:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 msgr2=0x564be904cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9a540136a0 tx=0x7f9a54004750 comp rx=0 tx=0).stop 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 shutdown_connections 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9050570 0x564be904d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be8fc1e40 0x564be904fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be905ff80 0x564be9062400 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/1403829838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be9047960 0x564be904cf20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 >> 172.21.15.7:0/1403829838 conn(0x564be8f72170 msgr2=0x564be9041410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.530+0000 7f9a51600700 0 client.25667 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:58:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.531+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 shutdown_connections 2024-06-13T06:58:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.531+0000 7f9a78c58b80 1 -- 172.21.15.7:0/1403829838 wait complete. 2024-06-13T06:58:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.531+0000 7f9a78c58b80 20 client.25667 trim_cache size 0 max 16384 2024-06-13T06:58:16.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.555+0000 7f9a78c58b80 1 Processor -- start 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 -- start start 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9041960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be904cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48011220 con 0x564be9047960 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800eab0 con 0x564be8fc1e40 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.556+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800c900 con 0x564be9050570 2024-06-13T06:58:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47626/0 (socket says 172.21.15.7:47626) 2024-06-13T06:58:16.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 -- 172.21.15.7:0/3302358813 learned_addr learned my addr 172.21.15.7:0/3302358813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 -- 172.21.15.7:0/3302358813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 msgr2=0x564be904cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be904cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 -- 172.21.15.7:0/3302358813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 msgr2=0x564be9041960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9041960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.557+0000 7f9a66a00700 1 -- 172.21.15.7:0/3302358813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be9048da0 con 0x564be9047960 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.558+0000 7f9a66a00700 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a54027820 tx=0x7f9a54027ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376bcaf52e05079d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.558+0000 7f9a53e00700 1 -- 172.21.15.7:0/3302358813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54036070 con 0x564be9047960 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.558+0000 7f9a53e00700 1 -- 172.21.15.7:0/3302358813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a54017030 con 0x564be9047960 2024-06-13T06:58:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.558+0000 7f9a53e00700 1 -- 172.21.15.7:0/3302358813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54022030 con 0x564be9047960 2024-06-13T06:58:16.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3302358813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 msgr2=0x564be904c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a54027820 tx=0x7f9a54027ee0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3302358813 shutdown_connections 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be904cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9041960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3302358813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be904c7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.559+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3302358813 >> 172.21.15.7:0/3302358813 conn(0x564be8f72170 msgr2=0x564be9061f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.560+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3302358813 shutdown_connections 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.560+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3302358813 wait complete. 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.560+0000 7f9a78c58b80 1 -- start start 2024-06-13T06:58:16.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.560+0000 7f9a78c58b80 1 Processor -- start 2024-06-13T06:58:16.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9053a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be9053fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800cf70 con 0x564be9047960 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800e3e0 con 0x564be8fc1e40 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800d640 con 0x564be9050570 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a67400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a67400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34894/0 (socket says 172.21.15.7:34894) 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a67400700 1 -- 172.21.15.7:0/3906223605 learned_addr learned my addr 172.21.15.7:0/3906223605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a67e00700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be9053fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.561+0000 7f9a78c58b80 10 client.0 _ll_register_callbacks cb 0x564be8f31720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 -- 172.21.15.7:0/3906223605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 msgr2=0x564be9053fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be9053fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 -- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 msgr2=0x564be9053a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9053a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 -- 172.21.15.7:0/3906223605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be90c8f80 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a67400700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a5800b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37467eb30d6b451 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a51600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580052c0 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.562+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9a58005460 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a54017030 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a58032050 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 10 client.25670 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564be90ca270 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a54017030 con 0x564be8fc1e40 2024-06-13T06:58:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a58025600 con 0x564be8fc1e40 2024-06-13T06:58:16.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:16.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 10 client.25670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:58:16.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 20 client.25670 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:16.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be90caa00 0x564be90cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:16.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 4 client.25670 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:58:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.563+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564be8f9f580 con 0x564be90caa00 2024-06-13T06:58:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.564+0000 7f9a66a00700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be90caa00 0x564be90cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.566+0000 7f9a66a00700 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be90caa00 0x564be90cce20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6d28817d4830bc8 server_cookie=2b6ecdc8415583a9 in_seq=0 out_seq=0 2024-06-13T06:58:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.566+0000 7f9a51600700 10 client.25670 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:58:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.567+0000 7f9a51600700 1 -- 172.21.15.7:0/3906223605 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564be8f9f580 con 0x564be90caa00 2024-06-13T06:58:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.567+0000 7f9a51600700 10 client.25670 handle_command_reply: tid=0 2024-06-13T06:58:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.567+0000 7f9a78c58b80 1 client.25670 shutdown 2024-06-13T06:58:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 msgr2=0x564be9053530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a5800b770 comp rx=0 tx=0).stop 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 shutdown_connections 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564be9050570 0x564be9053fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x564be90caa00 0x564be90cce20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564be8fc1e40 0x564be9053530 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 --2- 172.21.15.7:0/3906223605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564be9047960 0x564be9053a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a51600700 0 client.25670 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.568+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 >> 172.21.15.7:0/3906223605 conn(0x564be8f72170 msgr2=0x564be904cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.569+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 shutdown_connections 2024-06-13T06:58:16.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.569+0000 7f9a78c58b80 1 -- 172.21.15.7:0/3906223605 wait complete. 2024-06-13T06:58:16.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.569+0000 7f9a78c58b80 20 client.25670 trim_cache size 0 max 16384 2024-06-13T06:58:16.569 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 msgr2=0x7f9a4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 0x7f9a4c0a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x564be9042590 tx=0x7f9a54006df0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 msgr2=0x7f9a6810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810e820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000efc0 tx=0x7f9a6000c3a0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 shutdown_connections 2024-06-13T06:58:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a6810a670 0x7f9a6810e820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9a4c0a2ca0 0x7f9a4c0a5160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a681045c0 0x7f9a681111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 --2- 172.21.15.7:0/1039308151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a68104f80 0x7f9a6810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 >> 172.21.15.7:0/1039308151 conn(0x7f9a681000d0 msgr2=0x7f9a681016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 shutdown_connections 2024-06-13T06:58:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:16.570+0000 7f9a6cc00700 1 -- 172.21.15.7:0/1039308151 wait complete. 2024-06-13T06:58:17.570 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:17 smithi007 ceph-mon[32077]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-06-13T06:58:17.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T06:58:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:17 smithi189 ceph-mon[36616]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-06-13T06:58:17.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:17 smithi175 ceph-mon[36886]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-06-13T06:58:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 135.0 s, 1083.0 tps, lat 91.215 ms stddev 108.056 2024-06-13T06:58:18.215 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:19.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:19 smithi007 ceph-mon[32077]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-06-13T06:58:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:19 smithi189 ceph-mon[36616]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-06-13T06:58:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:19 smithi175 ceph-mon[36886]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 -- 172.21.15.7:0/968681179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c10d7b0 msgr2=0x7fca1c119c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 --2- 172.21.15.7:0/968681179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c10d7b0 0x7fca1c119c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca1400b0a0 tx=0x7fca1400b3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 -- 172.21.15.7:0/968681179 shutdown_connections 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 --2- 172.21.15.7:0/968681179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c10d7b0 0x7fca1c119c50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 --2- 172.21.15.7:0/968681179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c112ea0 0x7fca1c113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 --2- 172.21.15.7:0/968681179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 0x7fca1c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.363+0000 7fca1be00700 1 -- 172.21.15.7:0/968681179 >> 172.21.15.7:0/968681179 conn(0x7fca1c06d420 msgr2=0x7fca1c10b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.364+0000 7fca1be00700 1 -- 172.21.15.7:0/968681179 shutdown_connections 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.364+0000 7fca1be00700 1 -- 172.21.15.7:0/968681179 wait complete. 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.365+0000 7fca1be00700 1 Processor -- start 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.365+0000 7fca1be00700 1 -- start start 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 0x7fca1c1a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 0x7fca1c1a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c11d150 con 0x7fca1c10cdf0 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c11cfd0 con 0x7fca1c112ea0 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1be00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c11d2d0 con 0x7fca1c10d7b0 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38568/0 (socket says 172.21.15.7:38568) 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1a000700 1 -- 172.21.15.7:0/2304942311 learned_addr learned my addr 172.21.15.7:0/2304942311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.366+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 0x7fca1c1a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 -- 172.21.15.7:0/2304942311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 msgr2=0x7fca1c1a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1b400700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 0x7fca1c1a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 0x7fca1c1a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 -- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 msgr2=0x7fca1c1a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 0x7fca1c1a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 -- 172.21.15.7:0/2304942311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c1a84d0 con 0x7fca1c10d7b0 2024-06-13T06:58:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1b400700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 0x7fca1c1a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca1a000700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b530 tx=0x7fca0400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.367+0000 7fca13400700 1 -- 172.21.15.7:0/2304942311 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04015020 con 0x7fca1c10d7b0 2024-06-13T06:58:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.368+0000 7fca13400700 1 -- 172.21.15.7:0/2304942311 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04011d10 con 0x7fca1c10d7b0 2024-06-13T06:58:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.368+0000 7fca13400700 1 -- 172.21.15.7:0/2304942311 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0400f2e0 con 0x7fca1c10d7b0 2024-06-13T06:58:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.368+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1a87c0 con 0x7fca1c10d7b0 2024-06-13T06:58:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.368+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1b0040 con 0x7fca1c10d7b0 2024-06-13T06:58:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.379+0000 7fca13400700 1 -- 172.21.15.7:0/2304942311 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fca04018020 con 0x7fca1c10d7b0 2024-06-13T06:58:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.380+0000 7fca2c32ab80 1 Processor -- start 2024-06-13T06:58:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.381+0000 7fca2c32ab80 1 -- start start 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.381+0000 7fca13400700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 0x7fca000a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877b4530 0x560d877b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d876b8a10 con 0x560d877a8d60 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d876fbb00 con 0x560d877b4530 2024-06-13T06:58:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d876b68f0 con 0x560d877a9680 2024-06-13T06:58:21.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:21 smithi007 ceph-mon[32077]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 586 op/s 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca13400700 1 -- 172.21.15.7:0/2304942311 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca040cea70 con 0x7fca1c10d7b0 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca1b400700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 0x7fca000a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.382+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51454/0 (socket says 172.21.15.7:51454) 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 -- 172.21.15.7:0/3055776081 learned_addr learned my addr 172.21.15.7:0/3055776081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1b400700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877b4530 0x560d877b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1a000700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 -- 172.21.15.7:0/3055776081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 msgr2=0x560d877b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 -- 172.21.15.7:0/3055776081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877b4530 msgr2=0x560d877b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877b4530 0x560d877b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.383+0000 7fca1aa00700 1 -- 172.21.15.7:0/3055776081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d876a3f90 con 0x560d877a8d60 2024-06-13T06:58:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.384+0000 7fca1a000700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.385+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009db0 tx=0x7fca0c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df8084f961f0e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.385+0000 7fca1b400700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 0x7fca000a51e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca14009e60 tx=0x7fca1400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.385+0000 7fca0be00700 1 -- 172.21.15.7:0/3055776081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c01e070 con 0x560d877a8d60 2024-06-13T06:58:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.386+0000 7fca0be00700 1 -- 172.21.15.7:0/3055776081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c019b10 con 0x560d877a8d60 2024-06-13T06:58:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.387+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3055776081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 msgr2=0x560d877a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.387+0000 7fca0be00700 1 -- 172.21.15.7:0/3055776081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c01e070 con 0x560d877a8d60 2024-06-13T06:58:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.387+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009db0 tx=0x7fca0c011720 comp rx=0 tx=0).stop 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.388+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3055776081 shutdown_connections 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.388+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877b4530 0x560d877b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.388+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877a9140 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.388+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3055776081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.388+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3055776081 >> 172.21.15.7:0/3055776081 conn(0x560d876d6170 msgr2=0x560d877a5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.389+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3055776081 shutdown_connections 2024-06-13T06:58:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.389+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3055776081 wait complete. 2024-06-13T06:58:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.390+0000 7fca2c32ab80 1 -- start start 2024-06-13T06:58:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.390+0000 7fca2c32ab80 1 Processor -- start 2024-06-13T06:58:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a9680 0x560d877b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877b4530 0x560d877b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51468/0 (socket says 172.21.15.7:51468) 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca1aa00700 1 -- 172.21.15.7:0/2470026669 learned_addr learned my addr 172.21.15.7:0/2470026669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d87697f60 con 0x560d877a8d60 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d876f7ac0 con 0x560d877a9680 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.391+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d877bf830 con 0x560d877b4530 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1b400700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877b4530 0x560d877b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1a000700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a9680 0x560d877b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca2c32ab80 10 client.0 _ll_register_callbacks cb 0x560d87691380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1aa00700 1 -- 172.21.15.7:0/2470026669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877b4530 msgr2=0x560d877b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877b4530 0x560d877b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1aa00700 1 -- 172.21.15.7:0/2470026669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a9680 msgr2=0x560d877b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a9680 0x560d877b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.392+0000 7fca1aa00700 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d8764a8d0 con 0x560d877a8d60 2024-06-13T06:58:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.393+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca0c019910 tx=0x7fca0c019940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7911f4e3324ffa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.393+0000 7fca08c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x560d877a8d60 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca0c00ec20 con 0x560d877a8d60 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca0c019b10 con 0x560d877a8d60 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca0c042050 con 0x560d877a8d60 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca2c32ab80 10 client.15423 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.394+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d877b7f70 con 0x560d877a8d60 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c019b10 con 0x560d877a8d60 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fca0c0053c0 con 0x560d877a8d60 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 10 client.15423 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 10 client.15423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 20 client.15423 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d877ba4a0 0x560d877bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 4 client.15423 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.396+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d87709000 con 0x560d877ba4a0 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.397+0000 7fca1a000700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d877ba4a0 0x560d877bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.398+0000 7fca1a000700 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d877ba4a0 0x560d877bc900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e58128e76f4806f8 server_cookie=83a3f2927287686c in_seq=0 out_seq=0 2024-06-13T06:58:21.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.398+0000 7fca08c00700 10 client.15423 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:58:21.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.401+0000 7fca08c00700 1 -- 172.21.15.7:0/2470026669 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d87709000 con 0x560d877ba4a0 2024-06-13T06:58:21.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.401+0000 7fca08c00700 10 client.15423 handle_command_reply: tid=0 2024-06-13T06:58:21.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.401+0000 7fca2c32ab80 1 client.15423 shutdown 2024-06-13T06:58:21.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 msgr2=0x560d877b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca0c019910 tx=0x7fca0c019940 comp rx=0 tx=0).stop 2024-06-13T06:58:21.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 shutdown_connections 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d877ba4a0 0x560d877bc900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877b4530 0x560d877b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877a8d60 0x560d877b3c80 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca0c019910 tx=0x7fca0c019940 comp rx=0 tx=0).stop 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2470026669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a9680 0x560d877b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.402+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 >> 172.21.15.7:0/2470026669 conn(0x560d876d6170 msgr2=0x560d877a4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.403+0000 7fca08c00700 0 client.15423 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:58:21.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.403+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 shutdown_connections 2024-06-13T06:58:21.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.403+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2470026669 wait complete. 2024-06-13T06:58:21.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.404+0000 7fca2c32ab80 20 client.15423 trim_cache size 0 max 16384 2024-06-13T06:58:21.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.426+0000 7fca2c32ab80 1 Processor -- start 2024-06-13T06:58:21.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.426+0000 7fca2c32ab80 1 -- start start 2024-06-13T06:58:21.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.426+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.426+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877ac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.426+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b6f70 con 0x560d877b4530 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b4800 con 0x560d877a54c0 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b26a0 con 0x560d877a9680 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38592/0 (socket says 172.21.15.7:38592) 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 -- 172.21.15.7:0/3850983058 learned_addr learned my addr 172.21.15.7:0/3850983058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1b400700 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 -- 172.21.15.7:0/3850983058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 msgr2=0x560d877b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 -- 172.21.15.7:0/3850983058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 msgr2=0x560d877ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.427+0000 7fca1aa00700 1 -- 172.21.15.7:0/3850983058 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d877b1870 con 0x560d877a9680 2024-06-13T06:58:21.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.428+0000 7fca1aa00700 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fca0c011760 tx=0x7fca0c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ede856d4ab6f5c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.428+0000 7fca0b400700 1 -- 172.21.15.7:0/3850983058 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x560d877a9680 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.428+0000 7fca0b400700 1 -- 172.21.15.7:0/3850983058 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c02f040 con 0x560d877a9680 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.428+0000 7fca0b400700 1 -- 172.21.15.7:0/3850983058 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00ec80 con 0x560d877a9680 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3850983058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 msgr2=0x560d877abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fca0c011760 tx=0x7fca0c009880 comp rx=0 tx=0).stop 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3850983058 shutdown_connections 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877abf30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/3850983058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3850983058 >> 172.21.15.7:0/3850983058 conn(0x560d876d6170 msgr2=0x560d877a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3850983058 shutdown_connections 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.429+0000 7fca2c32ab80 1 -- 172.21.15.7:0/3850983058 wait complete. 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.430+0000 7fca2c32ab80 1 -- start start 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.430+0000 7fca2c32ab80 1 Processor -- start 2024-06-13T06:58:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b2cc0 con 0x560d877b4530 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b4130 con 0x560d877a54c0 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b3390 con 0x560d877a9680 2024-06-13T06:58:21.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.431+0000 7fca2c32ab80 10 client.0 _ll_register_callbacks cb 0x560d87691380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:21.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.432+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.432+0000 7fca1aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35398/0 (socket says 172.21.15.7:35398) 2024-06-13T06:58:21.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.432+0000 7fca1aa00700 1 -- 172.21.15.7:0/2138992043 learned_addr learned my addr 172.21.15.7:0/2138992043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:21.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.432+0000 7fca1b400700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.432+0000 7fca1a000700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1aa00700 1 -- 172.21.15.7:0/2138992043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 msgr2=0x560d877bfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877bfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1aa00700 1 -- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 msgr2=0x560d877bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1aa00700 1 -- 172.21.15.7:0/2138992043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d877c06c0 con 0x560d877a54c0 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.434+0000 7fca1b400700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:21.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.435+0000 7fca1aa00700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e6c0 tx=0x7fca0c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa51871813e8aca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.435+0000 7fca08c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x560d877a54c0 2024-06-13T06:58:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca0c02f040 con 0x560d877a54c0 2024-06-13T06:58:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca0c00ee30 con 0x560d877a54c0 2024-06-13T06:58:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca0c00e3b0 con 0x560d877a54c0 2024-06-13T06:58:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca2c32ab80 10 client.25685 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.436+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d8782da90 con 0x560d877a54c0 2024-06-13T06:58:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c02f040 con 0x560d877a54c0 2024-06-13T06:58:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fca0c005060 con 0x560d877a54c0 2024-06-13T06:58:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 10 client.25685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 20 client.25685 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d8782e060 0x560d87830480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 4 client.25685 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.437+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d87703580 con 0x560d8782e060 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.438+0000 7fca1a000700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d8782e060 0x560d87830480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.439+0000 7fca1a000700 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d8782e060 0x560d87830480 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85d52c8545694244 server_cookie=49a7453ce2c98cc8 in_seq=0 out_seq=0 2024-06-13T06:58:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.439+0000 7fca08c00700 10 client.25685 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:58:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.440+0000 7fca08c00700 1 -- 172.21.15.7:0/2138992043 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fca0409f2e0 con 0x560d8782e060 2024-06-13T06:58:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.440+0000 7fca08c00700 10 client.25685 handle_command_reply: tid=0 2024-06-13T06:58:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.440+0000 7fca2c32ab80 1 client.25685 shutdown 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.440+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 msgr2=0x560d877c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.440+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e6c0 tx=0x7fca0c004af0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 shutdown_connections 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d877b4530 0x560d877bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d877a54c0 0x560d877c50d0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e6c0 tx=0x7fca0c004af0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560d8782e060 0x560d87830480 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 --2- 172.21.15.7:0/2138992043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d877a9680 0x560d877bfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 >> 172.21.15.7:0/2138992043 conn(0x560d876d6170 msgr2=0x560d877abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.441+0000 7fca08c00700 0 client.25685 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:58:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.442+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 shutdown_connections 2024-06-13T06:58:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.442+0000 7fca2c32ab80 1 -- 172.21.15.7:0/2138992043 wait complete. 2024-06-13T06:58:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.442+0000 7fca2c32ab80 20 client.25685 trim_cache size 0 max 16384 2024-06-13T06:58:21.442 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:58:21.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 msgr2=0x7fca000a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 0x7fca000a51e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca14009e60 tx=0x7fca1400b700 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 msgr2=0x7fca1c1a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b530 tx=0x7fca0400b840 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 shutdown_connections 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca000a2d20 0x7fca000a51e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca1c112ea0 0x7fca1c1a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1c10cdf0 0x7fca1c1a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 --2- 172.21.15.7:0/2304942311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca1c10d7b0 0x7fca1c1a3a00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 >> 172.21.15.7:0/2304942311 conn(0x7fca1c06d420 msgr2=0x7fca1c10b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.443+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 shutdown_connections 2024-06-13T06:58:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:21.444+0000 7fca1be00700 1 -- 172.21.15.7:0/2304942311 wait complete. 2024-06-13T06:58:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:21 smithi189 ceph-mon[36616]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 586 op/s 2024-06-13T06:58:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:21 smithi175 ceph-mon[36886]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 586 op/s 2024-06-13T06:58:22.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T06:58:22.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 140.0 s, 1050.2 tps, lat 95.323 ms stddev 121.577 2024-06-13T06:58:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:23 smithi007 ceph-mon[32077]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-06-13T06:58:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:23 smithi189 ceph-mon[36616]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-06-13T06:58:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:23 smithi175 ceph-mon[36886]: pgmap v579: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-06-13T06:58:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:25 smithi189 ceph-mon[36616]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-06-13T06:58:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:25 smithi175 ceph-mon[36886]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-06-13T06:58:25.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:25 smithi007 ceph-mon[32077]: pgmap v580: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-06-13T06:58:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.938+0000 7f6ecb000700 1 -- 172.21.15.7:0/4062741318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 msgr2=0x7f6ec41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.938+0000 7f6ecb000700 1 --2- 172.21.15.7:0/4062741318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec41088e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009990 tx=0x7f6eb4009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.939+0000 7f6ecb000700 1 -- 172.21.15.7:0/4062741318 shutdown_connections 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.939+0000 7f6ecb000700 1 --2- 172.21.15.7:0/4062741318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.939+0000 7f6ecb000700 1 --2- 172.21.15.7:0/4062741318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec41088e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.939+0000 7f6ecb000700 1 --2- 172.21.15.7:0/4062741318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.939+0000 7f6ecb000700 1 -- 172.21.15.7:0/4062741318 >> 172.21.15.7:0/4062741318 conn(0x7f6ec4078340 msgr2=0x7f6ec4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.940+0000 7f6ecb000700 1 -- 172.21.15.7:0/4062741318 shutdown_connections 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.940+0000 7f6ecb000700 1 -- 172.21.15.7:0/4062741318 wait complete. 2024-06-13T06:58:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.941+0000 7f6ecb000700 1 Processor -- start 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.941+0000 7f6ecb000700 1 -- start start 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec410d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41105c0 con 0x7f6ec406a170 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4110440 con 0x7f6ec406ab30 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ecb000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4110740 con 0x7f6ec4108500 2024-06-13T06:58:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35404/0 (socket says 172.21.15.7:35404) 2024-06-13T06:58:26.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ec3400700 1 -- 172.21.15.7:0/2680884471 learned_addr learned my addr 172.21.15.7:0/2680884471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:26.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ec9000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.942+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec410d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec9000700 1 -- 172.21.15.7:0/2680884471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 msgr2=0x7f6ec410d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec9000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec9000700 1 -- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 msgr2=0x7f6ec410d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec9000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec410d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec9000700 1 -- 172.21.15.7:0/2680884471 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec4113ff0 con 0x7f6ec4108500 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.943+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.944+0000 7f6ec9000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec410ded0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc007f10 tx=0x7f6ebc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:26.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.945+0000 7f6ec0c00700 1 -- 172.21.15.7:0/2680884471 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00ddd0 con 0x7f6ec4108500 2024-06-13T06:58:26.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.945+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec41142e0 con 0x7f6ec4108500 2024-06-13T06:58:26.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.945+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec4114820 con 0x7f6ec4108500 2024-06-13T06:58:26.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.947+0000 7f6ec0c00700 1 -- 172.21.15.7:0/2680884471 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc004bb0 con 0x7f6ec4108500 2024-06-13T06:58:26.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.947+0000 7f6ec0c00700 1 -- 172.21.15.7:0/2680884471 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00ddd0 con 0x7f6ec4108500 2024-06-13T06:58:26.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.951+0000 7f6ec0c00700 1 -- 172.21.15.7:0/2680884471 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6ebc015400 con 0x7f6ec4108500 2024-06-13T06:58:26.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.954+0000 7f6ed59f3b80 1 Processor -- start 2024-06-13T06:58:26.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.955+0000 7f6ed59f3b80 1 -- start start 2024-06-13T06:58:26.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.955+0000 7f6ec0c00700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 0x7f6ea80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ec3400700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 0x7f6ea80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b145b90 0x55a99b145f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ec0c00700 1 -- 172.21.15.7:0/2680884471 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6ebc0ce8b0 con 0x7f6ec4108500 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b1464b0 0x55a99b13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a99b04ea10 con 0x55a99b145b90 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a99b091b00 con 0x55a99b1464b0 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a99b04c8f0 con 0x55a99b140240 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.956+0000 7f6ec9000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38616/0 (socket says 172.21.15.7:38616) 2024-06-13T06:58:26.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 -- 172.21.15.7:0/3479684764 learned_addr learned my addr 172.21.15.7:0/3479684764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 -- 172.21.15.7:0/3479684764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b1464b0 msgr2=0x55a99b13fd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b1464b0 0x55a99b13fd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 -- 172.21.15.7:0/3479684764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b145b90 msgr2=0x55a99b145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b145b90 0x55a99b145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 -- 172.21.15.7:0/3479684764 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99b039f90 con 0x55a99b140240 2024-06-13T06:58:26.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6ec9000700 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc098310 tx=0x7f6ebc09bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55760034166a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6eb9600700 1 -- 172.21.15.7:0/3479684764 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc09beb0 con 0x55a99b140240 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6eb9600700 1 -- 172.21.15.7:0/3479684764 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc0a0260 con 0x55a99b140240 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.957+0000 7f6eb9600700 1 -- 172.21.15.7:0/3479684764 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0a4690 con 0x55a99b140240 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.958+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/3479684764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 msgr2=0x55a99b140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.958+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc098310 tx=0x7f6ebc09bac0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.958+0000 7f6ec3400700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 0x7f6ea80a5360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4078760 tx=0x7f6eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/3479684764 shutdown_connections 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b140680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b145b90 0x55a99b145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/3479684764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b1464b0 0x55a99b13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/3479684764 >> 172.21.15.7:0/3479684764 conn(0x55a99b06c170 msgr2=0x55a99b13aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/3479684764 shutdown_connections 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.959+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/3479684764 wait complete. 2024-06-13T06:58:26.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.960+0000 7f6ed59f3b80 1 -- start start 2024-06-13T06:58:26.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.960+0000 7f6ed59f3b80 1 Processor -- start 2024-06-13T06:58:26.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b149540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b149a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a99b02df60 con 0x55a99b1464b0 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a99b08dac0 con 0x55a99b145b90 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a99b14ce90 con 0x55a99b140240 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ed59f3b80 10 client.0 _ll_register_callbacks cb 0x55a99b027380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38632/0 (socket says 172.21.15.7:38632) 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ec3e00700 1 -- 172.21.15.7:0/4091575523 learned_addr learned my addr 172.21.15.7:0/4091575523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.962+0000 7f6ec3400700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.963+0000 7f6ec3e00700 1 -- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 msgr2=0x55a99b149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.963+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.963+0000 7f6ec3e00700 1 -- 172.21.15.7:0/4091575523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 msgr2=0x55a99b149a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.963+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.963+0000 7f6ec3e00700 1 -- 172.21.15.7:0/4091575523 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99afe08d0 con 0x55a99b140240 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.964+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005f40 tx=0x7f6eb40056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27abd71666e9b93 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.964+0000 7f6eae000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.964+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4005070 con 0x55a99b140240 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.964+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6eb4005210 con 0x55a99b140240 2024-06-13T06:58:26.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.965+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc0a0260 con 0x55a99b140240 2024-06-13T06:58:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.965+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6eb402d050 con 0x55a99b140240 2024-06-13T06:58:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.965+0000 7f6ed59f3b80 10 client.25681 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.965+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a99b1576a0 con 0x55a99b140240 2024-06-13T06:58:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.965+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc0a0260 con 0x55a99b140240 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb40039b0 con 0x55a99b140240 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 10 client.25681 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 20 client.25681 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b159d80 0x55a99b15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 4 client.25681 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.966+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99b09f000 con 0x55a99b159d80 2024-06-13T06:58:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.967+0000 7f6ec3400700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b159d80 0x55a99b15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.967+0000 7f6ec3400700 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b159d80 0x55a99b15c1d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9c7c7162ecdcfe server_cookie=4feb5ed90b93ce4d in_seq=0 out_seq=0 2024-06-13T06:58:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.968+0000 7f6eae000700 10 client.25681 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.968+0000 7f6eae000700 1 -- 172.21.15.7:0/4091575523 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a99b09f000 con 0x55a99b159d80 2024-06-13T06:58:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.968+0000 7f6eae000700 10 client.25681 handle_command_reply: tid=0 2024-06-13T06:58:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 client.25681 shutdown 2024-06-13T06:58:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 msgr2=0x55a99b14c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005f40 tx=0x7f6eb40056a0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 shutdown_connections 2024-06-13T06:58:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b159d80 0x55a99b15c1d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b140240 0x55a99b14c240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/4091575523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 >> 172.21.15.7:0/4091575523 conn(0x55a99b06c170 msgr2=0x55a99b134790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.969+0000 7f6eae000700 0 client.25681 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.970+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 shutdown_connections 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.970+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/4091575523 wait complete. 2024-06-13T06:58:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:26.970+0000 7f6ed59f3b80 20 client.25681 trim_cache size 0 max 16384 2024-06-13T06:58:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.002+0000 7f6ed59f3b80 1 Processor -- start 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.002+0000 7f6ed59f3b80 1 -- start start 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 0x55a99b149c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b135130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4011200 con 0x55a99b1464b0 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea400ea90 con 0x55a99b140240 2024-06-13T06:58:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea400c8e0 con 0x55a99b145b90 2024-06-13T06:58:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35438/0 (socket says 172.21.15.7:35438) 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.003+0000 7f6ec3e00700 1 -- 172.21.15.7:0/2833186555 learned_addr learned my addr 172.21.15.7:0/2833186555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.004+0000 7f6ec3e00700 1 -- 172.21.15.7:0/2833186555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 msgr2=0x55a99b149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3400700 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 0x55a99b149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 0x55a99b149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3e00700 1 -- 172.21.15.7:0/2833186555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 msgr2=0x55a99b135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3e00700 1 -- 172.21.15.7:0/2833186555 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99b1462b0 con 0x55a99b140240 2024-06-13T06:58:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.005+0000 7f6ec3400700 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 0x55a99b149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.006+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40098e0 tx=0x7f6eb4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b670dfc482a961 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.007+0000 7f6eb8c00700 1 -- 172.21.15.7:0/2833186555 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4035020 con 0x55a99b140240 2024-06-13T06:58:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.007+0000 7f6eb8c00700 1 -- 172.21.15.7:0/2833186555 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb4003680 con 0x55a99b140240 2024-06-13T06:58:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.007+0000 7f6eb8c00700 1 -- 172.21.15.7:0/2833186555 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400fc80 con 0x55a99b140240 2024-06-13T06:58:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/2833186555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 msgr2=0x55a99b13fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40098e0 tx=0x7f6eb4005e80 comp rx=0 tx=0).stop 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/2833186555 shutdown_connections 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b1464b0 0x55a99b135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b140240 0x55a99b13fd00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.009+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/2833186555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b145b90 0x55a99b149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.010+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/2833186555 >> 172.21.15.7:0/2833186555 conn(0x55a99b06c170 msgr2=0x55a99b149500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.010+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/2833186555 shutdown_connections 2024-06-13T06:58:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.010+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/2833186555 wait complete. 2024-06-13T06:58:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.011+0000 7f6ed59f3b80 1 -- start start 2024-06-13T06:58:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.011+0000 7f6ed59f3b80 1 Processor -- start 2024-06-13T06:58:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.012+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b140240 0x55a99b152180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 0x55a99b160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea400cf50 con 0x55a99b140240 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea400e3c0 con 0x55a99b145b90 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea400d620 con 0x55a99b1464b0 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ed59f3b80 10 client.0 _ll_register_callbacks cb 0x55a99b027380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b1526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35454/0 (socket says 172.21.15.7:35454) 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b140240 0x55a99b152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ec3400700 1 -- 172.21.15.7:0/945266415 learned_addr learned my addr 172.21.15.7:0/945266415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.013+0000 7f6ec9000700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 0x55a99b160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.014+0000 7f6ec9000700 1 -- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 msgr2=0x55a99b1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.014+0000 7f6ec9000700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.014+0000 7f6ec9000700 1 -- 172.21.15.7:0/945266415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b140240 msgr2=0x55a99b152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.014+0000 7f6ec9000700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b140240 0x55a99b152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.014+0000 7f6ec9000700 1 -- 172.21.15.7:0/945266415 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99b1624e0 con 0x55a99b1464b0 2024-06-13T06:58:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.015+0000 7f6ec9000700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 0x55a99b160a90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00dfd0 tx=0x7f6ebc0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c31622c2b84ee00 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.015+0000 7f6eae000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.016+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0a0260 con 0x55a99b1464b0 2024-06-13T06:58:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.016+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6ebc0a4690 con 0x55a99b1464b0 2024-06-13T06:58:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.017+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb4003680 con 0x55a99b1464b0 2024-06-13T06:58:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.017+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc009490 con 0x55a99b1464b0 2024-06-13T06:58:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.017+0000 7f6ed59f3b80 10 client.25684 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.017+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a99b1c3db0 con 0x55a99b1464b0 2024-06-13T06:58:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.018+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb4003680 con 0x55a99b1464b0 2024-06-13T06:58:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.018+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6ebc097c00 con 0x55a99b1464b0 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 20 client.25684 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b1c4090 0x55a99b1c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 4 client.25684 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a99b099580 con 0x55a99b1c4090 2024-06-13T06:58:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.020+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b1c4090 0x55a99b1c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.022+0000 7f6ec3e00700 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b1c4090 0x55a99b1c6480 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b26abfc8378cd server_cookie=4d2ec132bc6205dd in_seq=0 out_seq=0 2024-06-13T06:58:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.022+0000 7f6eae000700 10 client.25684 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.022+0000 7f6eae000700 1 -- 172.21.15.7:0/945266415 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a99b099580 con 0x55a99b1c4090 2024-06-13T06:58:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.022+0000 7f6eae000700 10 client.25684 handle_command_reply: tid=0 2024-06-13T06:58:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.022+0000 7f6ed59f3b80 1 client.25684 shutdown 2024-06-13T06:58:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 msgr2=0x55a99b160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 0x55a99b160a90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00dfd0 tx=0x7f6ebc0096d0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 shutdown_connections 2024-06-13T06:58:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a99b1c4090 0x55a99b1c6480 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a99b1464b0 0x55a99b160a90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a99b140240 0x55a99b152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 --2- 172.21.15.7:0/945266415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a99b145b90 0x55a99b1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 >> 172.21.15.7:0/945266415 conn(0x55a99b06c170 msgr2=0x55a99b1494a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6eae000700 0 client.25684 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 shutdown_connections 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 1 -- 172.21.15.7:0/945266415 wait complete. 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.023+0000 7f6ed59f3b80 20 client.25684 trim_cache size 0 max 16384 2024-06-13T06:58:27.023 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.024+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 msgr2=0x7f6ea80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.024+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 0x7f6ea80a5360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4078760 tx=0x7f6eb0009260 comp rx=0 tx=0).stop 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.024+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 msgr2=0x7f6ec410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.024+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec410ded0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc007f10 tx=0x7f6ebc00d3b0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 shutdown_connections 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ec4108500 0x7f6ec410ded0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec406a170 0x7f6ec410d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6ea80a2ea0 0x7f6ea80a5360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 --2- 172.21.15.7:0/2680884471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec406ab30 0x7f6ec410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 >> 172.21.15.7:0/2680884471 conn(0x7f6ec4078340 msgr2=0x7f6ec40ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 shutdown_connections 2024-06-13T06:58:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:27.025+0000 7f6ecb000700 1 -- 172.21.15.7:0/2680884471 wait complete. 2024-06-13T06:58:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:27 smithi189 ceph-mon[36616]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-06-13T06:58:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:27 smithi175 ceph-mon[36886]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-06-13T06:58:27.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:27 smithi007 ceph-mon[32077]: pgmap v581: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-06-13T06:58:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 145.0 s, 1377.0 tps, lat 72.144 ms stddev 94.261 2024-06-13T06:58:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 -- 172.21.15.7:0/1320907955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524101a00 msgr2=0x7f4524101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 --2- 172.21.15.7:0/1320907955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524101a00 0x7f4524101e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4514009900 tx=0x7f451402a400 comp rx=0 tx=0).stop 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 -- 172.21.15.7:0/1320907955 shutdown_connections 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 --2- 172.21.15.7:0/1320907955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45241035b0 0x7f4524109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 --2- 172.21.15.7:0/1320907955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f4524103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 --2- 172.21.15.7:0/1320907955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4524101a00 0x7f4524101e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.506+0000 7f4528a00640 1 -- 172.21.15.7:0/1320907955 >> 172.21.15.7:0/1320907955 conn(0x7f45240fcf90 msgr2=0x7f45240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.507+0000 7f4528a00640 1 -- 172.21.15.7:0/1320907955 shutdown_connections 2024-06-13T06:58:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.507+0000 7f4528a00640 1 -- 172.21.15.7:0/1320907955 wait complete. 2024-06-13T06:58:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.508+0000 7f4528a00640 1 Processor -- start 2024-06-13T06:58:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.509+0000 7f4528a00640 1 -- start start 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4524101a00 0x7f4524199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f452419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f452410d6a0 con 0x7f4524102bf0 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f452410d520 con 0x7f45241035b0 2024-06-13T06:58:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4528a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f452410d820 con 0x7f4524101a00 2024-06-13T06:58:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.510+0000 7f4522800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4524101a00 0x7f4524199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4522800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35476/0 (socket says 172.21.15.7:35476) 2024-06-13T06:58:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4522800640 1 -- 172.21.15.7:0/2897563065 learned_addr learned my addr 172.21.15.7:0/2897563065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f452419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 -- 172.21.15.7:0/2897563065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4524101a00 msgr2=0x7f4524199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4524101a00 0x7f4524199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 -- 172.21.15.7:0/2897563065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 msgr2=0x7f45241a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.511+0000 7f4521400640 1 -- 172.21.15.7:0/2897563065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45241a1a20 con 0x7f4524102bf0 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.512+0000 7f4522800640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.512+0000 7f4521400640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f452419a340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f451000d6d0 tx=0x7f451000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.513+0000 7f451aa00640 1 -- 172.21.15.7:0/2897563065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510017070 con 0x7f4524102bf0 2024-06-13T06:58:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.513+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45241a1d10 con 0x7f4524102bf0 2024-06-13T06:58:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.513+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241a2250 con 0x7f4524102bf0 2024-06-13T06:58:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.514+0000 7f451aa00640 1 -- 172.21.15.7:0/2897563065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4510009820 con 0x7f4524102bf0 2024-06-13T06:58:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.514+0000 7f451aa00640 1 -- 172.21.15.7:0/2897563065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510020060 con 0x7f4524102bf0 2024-06-13T06:58:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.519+0000 7f451aa00640 1 -- 172.21.15.7:0/2897563065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4510004c00 con 0x7f4524102bf0 2024-06-13T06:58:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.522+0000 7f451aa00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 0x7f45080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.522+0000 7f451aa00640 1 -- 172.21.15.7:0/2897563065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f45100cfdc0 con 0x7f4524102bf0 2024-06-13T06:58:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.523+0000 7f4521e00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 0x7f45080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.523+0000 7f452b82a740 1 Processor -- start 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.524+0000 7f452b82a740 1 -- start start 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.524+0000 7f452b82a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.524+0000 7f452b82a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.525+0000 7f4521e00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 0x7f45080a51f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f451402a9f0 tx=0x7f45140095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.525+0000 7f4521e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.525+0000 7f4521400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.525+0000 7f4521e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35484/0 (socket says 172.21.15.7:35484) 2024-06-13T06:58:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.525+0000 7f4521400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38696/0 (socket says 172.21.15.7:38696) 2024-06-13T06:58:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.524+0000 7f452b82a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92faef70 0x55be92fb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.526+0000 7f452b82a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55be92f04420 con 0x55be92faef70 2024-06-13T06:58:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.526+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be92eabc50 con 0x55be92fa9410 2024-06-13T06:58:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.526+0000 7f4521e00640 1 -- 172.21.15.7:0/3804124828 learned_addr learned my addr 172.21.15.7:0/3804124828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.526+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55be92f003e0 con 0x55be92fa9e00 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 -- 172.21.15.7:0/3804124828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 msgr2=0x55be92fae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 -- 172.21.15.7:0/3804124828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92faef70 msgr2=0x55be92fb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92faef70 0x55be92fb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 -- 172.21.15.7:0/3804124828 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be92c18210 con 0x55be92fa9410 2024-06-13T06:58:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521400640 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.527+0000 7f4521e00640 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4524102a50 tx=0x7f451403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa03f6f0ac852e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.528+0000 7f450f000640 1 -- 172.21.15.7:0/3804124828 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451404c070 con 0x55be92fa9410 2024-06-13T06:58:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.528+0000 7f450f000640 1 -- 172.21.15.7:0/3804124828 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4514003ad0 con 0x55be92fa9410 2024-06-13T06:58:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.528+0000 7f450f000640 1 -- 172.21.15.7:0/3804124828 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514035050 con 0x55be92fa9410 2024-06-13T06:58:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 msgr2=0x55be92fa9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4524102a50 tx=0x7f451403c000 comp rx=0 tx=0).stop 2024-06-13T06:58:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 shutdown_connections 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92faef70 0x55be92fb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9e00 0x55be92fae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.532+0000 7f452b82a740 1 --2- 172.21.15.7:0/3804124828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9410 0x55be92fa9830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.533+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 >> 172.21.15.7:0/3804124828 conn(0x55be92f68fb0 msgr2=0x55be92f9c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.533+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 shutdown_connections 2024-06-13T06:58:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.533+0000 7f452b82a740 1 -- 172.21.15.7:0/3804124828 wait complete. 2024-06-13T06:58:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.534+0000 7f452b82a740 1 -- start start 2024-06-13T06:58:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.535+0000 7f452b82a740 1 Processor -- start 2024-06-13T06:58:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.536+0000 7f452b82a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.536+0000 7f452b82a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.536+0000 7f452b82a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be92fbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.536+0000 7f452b82a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55be92fb1ec0 con 0x55be92fa9e00 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.536+0000 7f452b82a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55be92ebf0b0 con 0x55be92faef70 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f452b82a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55be92fb2040 con 0x55be92fa9410 2024-06-13T06:58:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f452b82a740 10 client.0 _ll_register_callbacks cb 0x55be92eaaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f4521e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f4521e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38704/0 (socket says 172.21.15.7:38704) 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f4521e00640 1 -- 172.21.15.7:0/2237308819 learned_addr learned my addr 172.21.15.7:0/2237308819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f4521400640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.537+0000 7f4522800640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be92fbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 -- 172.21.15.7:0/2237308819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 msgr2=0x55be92fb3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 -- 172.21.15.7:0/2237308819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 msgr2=0x55be92fb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 -- 172.21.15.7:0/2237308819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be92ce4620 con 0x55be92faef70 2024-06-13T06:58:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.538+0000 7f4522800640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be92fbac90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9c0 tx=0x7f451c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462ec80ab9e4811f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.539+0000 7f4503400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.539+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c00ccc0 con 0x55be92faef70 2024-06-13T06:58:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.539+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f451c003970 con 0x55be92faef70 2024-06-13T06:58:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.540+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4514003ad0 con 0x55be92faef70 2024-06-13T06:58:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.542+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f451c00ce60 con 0x55be92faef70 2024-06-13T06:58:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.542+0000 7f452b82a740 10 client.25712 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.542+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be92fbe070 con 0x55be92faef70 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4514003ad0 con 0x55be92faef70 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f451c024700 con 0x55be92faef70 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 10 client.25712 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 20 client.25712 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be92fc1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 4 client.25712 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be92f11b50 con 0x55be92fbf870 2024-06-13T06:58:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.544+0000 7f4521e00640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be92fc1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.546+0000 7f4503400640 10 client.25712 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.546+0000 7f4521e00640 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be92fc1cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870bb9420950ab server_cookie=5d4a5d7785549155 in_seq=0 out_seq=0 2024-06-13T06:58:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f4503400640 1 -- 172.21.15.7:0/2237308819 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be92f11b50 con 0x55be92fbf870 2024-06-13T06:58:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f4503400640 10 client.25712 handle_command_reply: tid=0 2024-06-13T06:58:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f452b82a740 1 client.25712 shutdown 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 msgr2=0x55be92fbac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be92fbac90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f451c00e9c0 tx=0x7f451c00ee80 comp rx=0 tx=0).stop 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.548+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 shutdown_connections 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be92fc1cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be92fbac90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f4503400640 0 client.25712 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 --2- 172.21.15.7:0/2237308819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 >> 172.21.15.7:0/2237308819 conn(0x55be92f68fb0 msgr2=0x55be92f9b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 shutdown_connections 2024-06-13T06:58:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 1 -- 172.21.15.7:0/2237308819 wait complete. 2024-06-13T06:58:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.549+0000 7f452b82a740 20 client.25712 trim_cache size 0 max 16384 2024-06-13T06:58:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 Processor -- start 2024-06-13T06:58:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 -- start start 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9410 0x55be92f9c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 0x55be92fbabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 0x55be92faa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4504012400 con 0x55be92fa9410 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.585+0000 7f452b82a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f450400fcd0 con 0x55be92fa9e00 2024-06-13T06:58:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.586+0000 7f452b82a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f450400db20 con 0x55be92faef70 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.586+0000 7f4522800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 0x55be92faa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.586+0000 7f4521400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 0x55be92fbabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.586+0000 7f4522800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 0x55be92faa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38706/0 (socket says 172.21.15.7:38706) 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.586+0000 7f4522800640 1 -- 172.21.15.7:0/1066552793 learned_addr learned my addr 172.21.15.7:0/1066552793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521e00640 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9410 0x55be92f9c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521400640 1 -- 172.21.15.7:0/1066552793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 msgr2=0x55be92faa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521400640 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 0x55be92faa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521400640 1 -- 172.21.15.7:0/1066552793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9410 msgr2=0x55be92f9c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521400640 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9410 0x55be92f9c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.587+0000 7f4521400640 1 -- 172.21.15.7:0/1066552793 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be92f9d330 con 0x55be92fa9e00 2024-06-13T06:58:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.588+0000 7f4521400640 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 0x55be92fbabb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45100986e0 tx=0x7f45100bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd09eb33f86dc58 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.589+0000 7f450e600640 1 -- 172.21.15.7:0/1066552793 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100a4ef0 con 0x55be92fa9e00 2024-06-13T06:58:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.589+0000 7f450e600640 1 -- 172.21.15.7:0/1066552793 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45100cd630 con 0x55be92fa9e00 2024-06-13T06:58:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.589+0000 7f450e600640 1 -- 172.21.15.7:0/1066552793 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100bc310 con 0x55be92fa9e00 2024-06-13T06:58:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 -- 172.21.15.7:0/1066552793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 msgr2=0x55be92fbabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 0x55be92fbabb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45100986e0 tx=0x7f45100bc040 comp rx=0 tx=0).stop 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 -- 172.21.15.7:0/1066552793 shutdown_connections 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92faef70 0x55be92faa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92fa9e00 0x55be92fbabb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 --2- 172.21.15.7:0/1066552793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9410 0x55be92f9c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.590+0000 7f452b82a740 1 -- 172.21.15.7:0/1066552793 >> 172.21.15.7:0/1066552793 conn(0x55be92f68fb0 msgr2=0x55be92f9c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.591+0000 7f452b82a740 1 -- 172.21.15.7:0/1066552793 shutdown_connections 2024-06-13T06:58:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.591+0000 7f452b82a740 1 -- 172.21.15.7:0/1066552793 wait complete. 2024-06-13T06:58:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.591+0000 7f452b82a740 1 -- start start 2024-06-13T06:58:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.591+0000 7f452b82a740 1 Processor -- start 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be930395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f450400e190 con 0x55be92fa9e00 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f450400f600 con 0x55be92faef70 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f450400e860 con 0x55be92fa9410 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f452b82a740 10 client.0 _ll_register_callbacks cb 0x7f451c0075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f4521e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f4521e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38714/0 (socket says 172.21.15.7:38714) 2024-06-13T06:58:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.593+0000 7f4521e00640 1 -- 172.21.15.7:0/556511047 learned_addr learned my addr 172.21.15.7:0/556511047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4522800640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be930395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521400640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521e00640 1 -- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 msgr2=0x55be930395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521e00640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be930395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521e00640 1 -- 172.21.15.7:0/556511047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 msgr2=0x55be92fb6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521e00640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521e00640 1 -- 172.21.15.7:0/556511047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be93039af0 con 0x55be92fa9410 2024-06-13T06:58:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.594+0000 7f4521400640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.595+0000 7f4521e00640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f451402a920 tx=0x7f4514031d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f488f41ac6235b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.595+0000 7f4503400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.595+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451403f040 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.595+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4514048940 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.596+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45100cd630 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.596+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f451406a050 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.596+0000 7f452b82a740 10 client.25699 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.596+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be9303ae30 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.596+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45100cd630 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4514065030 con 0x55be92fa9410 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 10 client.25699 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 20 client.25699 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be9303d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 4 client.25699 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:58:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be92f97db0 con 0x55be92fbf870 2024-06-13T06:58:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.597+0000 7f4521400640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be9303d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.599+0000 7f4521400640 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be9303d410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7185f72fde38c091 server_cookie=648a900f77c7425b in_seq=0 out_seq=0 2024-06-13T06:58:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.599+0000 7f4503400640 10 client.25699 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f4503400640 1 -- 172.21.15.7:0/556511047 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be92f97db0 con 0x55be92fbf870 2024-06-13T06:58:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f4503400640 10 client.25699 handle_command_reply: tid=0 2024-06-13T06:58:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 client.25699 shutdown 2024-06-13T06:58:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 msgr2=0x55be92fb5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f451402a920 tx=0x7f4514031d90 comp rx=0 tx=0).stop 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 shutdown_connections 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55be92fbf870 0x55be9303d410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55be92faef70 0x55be930395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be92fa9e00 0x55be92fb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f4503400640 0 client.25699 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 --2- 172.21.15.7:0/556511047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55be92fa9410 0x55be92fb5b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 >> 172.21.15.7:0/556511047 conn(0x55be92f68fb0 msgr2=0x55be92f9afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.610+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 shutdown_connections 2024-06-13T06:58:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.611+0000 7f452b82a740 1 -- 172.21.15.7:0/556511047 wait complete. 2024-06-13T06:58:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.611+0000 7f452b82a740 20 client.25699 trim_cache size 0 max 16384 2024-06-13T06:58:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:58:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:58:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 msgr2=0x7f45080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 0x7f45080a51f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f451402a9f0 tx=0x7f45140095f0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 msgr2=0x7f452419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f452419a340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f451000d6d0 tx=0x7f451000db90 comp rx=0 tx=0).stop 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 shutdown_connections 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45080a2d30 0x7f45080a51f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45241035b0 0x7f45241a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4524102bf0 0x7f452419a340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 --2- 172.21.15.7:0/2897563065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4524101a00 0x7f4524199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 >> 172.21.15.7:0/2897563065 conn(0x7f45240fcf90 msgr2=0x7f4524073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.612+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 shutdown_connections 2024-06-13T06:58:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:28.613+0000 7f4528a00640 1 -- 172.21.15.7:0/2897563065 wait complete. 2024-06-13T06:58:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:29 smithi175 ceph-mon[36886]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2024-06-13T06:58:29.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:29 smithi007 ceph-mon[32077]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2024-06-13T06:58:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:29 smithi189 ceph-mon[36616]: pgmap v582: 129 pgs: 129 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2024-06-13T06:58:29.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T06:58:29.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T06:58:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:31 smithi175 ceph-mon[36886]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-06-13T06:58:31.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:31 smithi007 ceph-mon[32077]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-06-13T06:58:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:31 smithi189 ceph-mon[36616]: pgmap v583: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 575 op/s 2024-06-13T06:58:32.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:32 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:32.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:32 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:32.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:32 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:58:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 150.0 s, 1016.4 tps, lat 99.738 ms stddev 135.483 2024-06-13T06:58:33.609 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:33.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:58:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-13T06:58:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:58:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: pgmap v584: 129 pgs: 129 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 525 op/s 2024-06-13T06:58:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:58:33.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:58:33.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:58:33.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:58:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:34 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T06:58:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:34 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 -- 172.21.15.7:0/1045852278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990101ac0 msgr2=0x7f099010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 --2- 172.21.15.7:0/1045852278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990101ac0 0x7f099010e780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0978009990 tx=0x7f0978009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 -- 172.21.15.7:0/1045852278 shutdown_connections 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 --2- 172.21.15.7:0/1045852278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f099010ecc0 0x7f09901110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 --2- 172.21.15.7:0/1045852278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f09901014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 --2- 172.21.15.7:0/1045852278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0990101ac0 0x7f099010e780 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 -- 172.21.15.7:0/1045852278 >> 172.21.15.7:0/1045852278 conn(0x7f09900fcdb0 msgr2=0x7f09900ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 -- 172.21.15.7:0/1045852278 shutdown_connections 2024-06-13T06:58:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.828+0000 7f098fe00700 1 -- 172.21.15.7:0/1045852278 wait complete. 2024-06-13T06:58:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.829+0000 7f098fe00700 1 Processor -- start 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 -- start start 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f099010ecc0 0x7f09901a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0990104f10 con 0x7f099010ecc0 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0990104d90 con 0x7f0990101110 2024-06-13T06:58:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.830+0000 7f098fe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0990105090 con 0x7f0990101ac0 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.831+0000 7f0987e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.831+0000 7f098ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.831+0000 7f098ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35686/0 (socket says 172.21.15.7:35686) 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.831+0000 7f0987e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38722/0 (socket says 172.21.15.7:38722) 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.831+0000 7f0987e00700 1 -- 172.21.15.7:0/1521648611 learned_addr learned my addr 172.21.15.7:0/1521648611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f098f400700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f099010ecc0 0x7f09901a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 -- 172.21.15.7:0/1521648611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 msgr2=0x7f099010dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 -- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f099010ecc0 msgr2=0x7f09901a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f099010ecc0 0x7f09901a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 -- 172.21.15.7:0/1521648611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09901a41d0 con 0x7f0990101ac0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f098ea00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.832+0000 7f0987e00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f097800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.833+0000 7f0987400700 1 -- 172.21.15.7:0/1521648611 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978027070 con 0x7f0990101ac0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.833+0000 7f0987400700 1 -- 172.21.15.7:0/1521648611 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0978003d70 con 0x7f0990101ac0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.833+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09901a4460 con 0x7f0990101ac0 2024-06-13T06:58:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.833+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0990112d60 con 0x7f0990101ac0 2024-06-13T06:58:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.834+0000 7f0987400700 1 -- 172.21.15.7:0/1521648611 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097800f460 con 0x7f0990101ac0 2024-06-13T06:58:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:34 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T06:58:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:34 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T06:58:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.838+0000 7f0987400700 1 -- 172.21.15.7:0/1521648611 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0978009d10 con 0x7f0990101ac0 2024-06-13T06:58:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.839+0000 7f09a024bb80 1 Processor -- start 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 -- start start 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 0x5569576504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957650e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 0x55695765b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55695755da90 con 0x5569576509e0 2024-06-13T06:58:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5569575a0b80 con 0x5569576595f0 2024-06-13T06:58:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f09a024bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55695755b970 con 0x5569575caec0 2024-06-13T06:58:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f0987400700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 0x7f09740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f098f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 0x5569576504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f0987400700 1 -- 172.21.15.7:0/1521648611 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f09780d0100 con 0x7f0990101ac0 2024-06-13T06:58:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f098f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 0x5569576504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38734/0 (socket says 172.21.15.7:38734) 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.840+0000 7f098f400700 1 -- 172.21.15.7:0/2493812588 learned_addr learned my addr 172.21.15.7:0/2493812588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.841+0000 7f098f400700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 0x7f09740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.841+0000 7f098ea00700 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957650e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.841+0000 7f0987e00700 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 0x55695765b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 -- 172.21.15.7:0/2493812588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 msgr2=0x5569576504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 0x5569576504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 -- 172.21.15.7:0/2493812588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 msgr2=0x556957650e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957650e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 -- 172.21.15.7:0/2493812588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556957549010 con 0x5569576595f0 2024-06-13T06:58:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f098f400700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 0x7f09740a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0988009710 tx=0x7f0988009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.842+0000 7f0987e00700 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 0x55695765b9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09780cd860 tx=0x7f0978099b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991b2e4d2a8ef04 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.843+0000 7f097fe00700 1 -- 172.21.15.7:0/2493812588 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097809deb0 con 0x5569576595f0 2024-06-13T06:58:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.843+0000 7f097fe00700 1 -- 172.21.15.7:0/2493812588 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097809d7b0 con 0x5569576595f0 2024-06-13T06:58:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.843+0000 7f097fe00700 1 -- 172.21.15.7:0/2493812588 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097809da10 con 0x5569576595f0 2024-06-13T06:58:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.844+0000 7f09a024bb80 1 -- 172.21.15.7:0/2493812588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 msgr2=0x55695765b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.844+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 0x55695765b9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09780cd860 tx=0x7f0978099b70 comp rx=0 tx=0).stop 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 -- 172.21.15.7:0/2493812588 shutdown_connections 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576595f0 0x55695765b9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569575caec0 0x5569576504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2493812588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957650e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 -- 172.21.15.7:0/2493812588 >> 172.21.15.7:0/2493812588 conn(0x55695757b1f0 msgr2=0x55695764a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 -- 172.21.15.7:0/2493812588 shutdown_connections 2024-06-13T06:58:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.845+0000 7f09a024bb80 1 -- 172.21.15.7:0/2493812588 wait complete. 2024-06-13T06:58:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.846+0000 7f09a024bb80 1 -- start start 2024-06-13T06:58:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.846+0000 7f09a024bb80 1 Processor -- start 2024-06-13T06:58:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957658d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 0x556957656050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 0x556957656590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55695753cfe0 con 0x5569576595f0 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55695759cb40 con 0x5569575caec0 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55695765c010 con 0x5569576509e0 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.847+0000 7f09a024bb80 10 client.0 _ll_register_callbacks cb 0x55695753a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.848+0000 7f098f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 0x556957656590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.848+0000 7f098f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 0x556957656590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44876/0 (socket says 172.21.15.7:44876) 2024-06-13T06:58:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.848+0000 7f098f400700 1 -- 172.21.15.7:0/3732565199 learned_addr learned my addr 172.21.15.7:0/3732565199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.848+0000 7f0987e00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 0x556957656050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.848+0000 7f098ea00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957658d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 -- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 msgr2=0x556957658d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957658d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 -- 172.21.15.7:0/3732565199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 msgr2=0x556957656590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 0x556957656590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 -- 172.21.15.7:0/3732565199 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569574ef950 con 0x5569576509e0 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f098ea00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957658d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f0987e00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 0x556957656050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0978099400 tx=0x7f0978009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba12734d3a7cf17 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.849+0000 7f097cc00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.850+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09780ba060 con 0x5569576509e0 2024-06-13T06:58:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.850+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f097809dc10 con 0x5569576509e0 2024-06-13T06:58:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.851+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f097809d7b0 con 0x5569576509e0 2024-06-13T06:58:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.851+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f097809deb0 con 0x5569576509e0 2024-06-13T06:58:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.852+0000 7f09a024bb80 10 client.25711 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.852+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569576666a0 con 0x5569576509e0 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.852+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097809d7b0 con 0x5569576509e0 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.852+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f097809cd10 con 0x5569576509e0 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 10 client.25711 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 20 client.25711 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576689a0 0x55695766ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 4 client.25711 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.853+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569575ae080 con 0x5569576689a0 2024-06-13T06:58:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.854+0000 7f098ea00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576689a0 0x55695766ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.855+0000 7f098ea00700 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576689a0 0x55695766ad90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9471d8c52e74d6 server_cookie=db166927cc4d4ec8 in_seq=0 out_seq=0 2024-06-13T06:58:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.855+0000 7f097cc00700 10 client.25711 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.855+0000 7f097cc00700 1 -- 172.21.15.7:0/3732565199 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569575ae080 con 0x5569576689a0 2024-06-13T06:58:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.855+0000 7f097cc00700 10 client.25711 handle_command_reply: tid=0 2024-06-13T06:58:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 client.25711 shutdown 2024-06-13T06:58:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 msgr2=0x556957656050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 0x556957656050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0978099400 tx=0x7f0978009910 comp rx=0 tx=0).stop 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 shutdown_connections 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576689a0 0x55695766ad90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576595f0 0x556957656590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957658d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3732565199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576509e0 0x556957656050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 >> 172.21.15.7:0/3732565199 conn(0x55695757b1f0 msgr2=0x55695764a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f097cc00700 0 client.25711 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.856+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 shutdown_connections 2024-06-13T06:58:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.857+0000 7f09a024bb80 1 -- 172.21.15.7:0/3732565199 wait complete. 2024-06-13T06:58:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.857+0000 7f09a024bb80 20 client.25711 trim_cache size 0 max 16384 2024-06-13T06:58:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:34 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T06:58:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:34 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T06:58:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.901+0000 7f09a024bb80 1 Processor -- start 2024-06-13T06:58:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 -- start start 2024-06-13T06:58:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55695757b1f0 0x556957658890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957656550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f096c011330 con 0x5569576509e0 2024-06-13T06:58:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f096c00ebc0 con 0x5569575caec0 2024-06-13T06:58:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.902+0000 7f09a024bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f096c00ca10 con 0x55695757b1f0 2024-06-13T06:58:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35724/0 (socket says 172.21.15.7:35724) 2024-06-13T06:58:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 -- 172.21.15.7:0/3682243317 learned_addr learned my addr 172.21.15.7:0/3682243317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:34.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f098f400700 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957656550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 -- 172.21.15.7:0/3682243317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55695757b1f0 msgr2=0x556957658890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55695757b1f0 0x556957658890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.903+0000 7f0987e00700 1 -- 172.21.15.7:0/3682243317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 msgr2=0x556957656550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.905+0000 7f0987e00700 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957656550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.905+0000 7f0987e00700 1 -- 172.21.15.7:0/3682243317 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55695764a9e0 con 0x5569575caec0 2024-06-13T06:58:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.905+0000 7f098f400700 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957656550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:58:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.905+0000 7f0987e00700 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0978099c10 tx=0x7f0978009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9327be87e032993 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.907+0000 7f097f400700 1 -- 172.21.15.7:0/3682243317 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09780ba070 con 0x5569575caec0 2024-06-13T06:58:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.907+0000 7f097f400700 1 -- 172.21.15.7:0/3682243317 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09780a6390 con 0x5569575caec0 2024-06-13T06:58:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.907+0000 7f097f400700 1 -- 172.21.15.7:0/3682243317 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09780a5f00 con 0x5569575caec0 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 -- 172.21.15.7:0/3682243317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 msgr2=0x556957656010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0978099c10 tx=0x7f0978009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 -- 172.21.15.7:0/3682243317 shutdown_connections 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5569576509e0 0x556957656550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55695757b1f0 0x556957658890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 --2- 172.21.15.7:0/3682243317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569575caec0 0x556957656010 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0978099c10 tx=0x7f0978009ca0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.908+0000 7f09a024bb80 1 -- 172.21.15.7:0/3682243317 >> 172.21.15.7:0/3682243317 conn(0x556957651120 msgr2=0x556957655900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.909+0000 7f09a024bb80 1 -- 172.21.15.7:0/3682243317 shutdown_connections 2024-06-13T06:58:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.910+0000 7f09a024bb80 1 -- 172.21.15.7:0/3682243317 wait complete. 2024-06-13T06:58:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.910+0000 7f09a024bb80 1 -- start start 2024-06-13T06:58:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.911+0000 7f09a024bb80 1 Processor -- start 2024-06-13T06:58:34.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 0x55695765e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576595f0 0x55695765ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f096c00d080 con 0x55695757b1f0 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f096c00e4f0 con 0x5569576509e0 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f096c00d750 con 0x5569576595f0 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.912+0000 7f09a024bb80 10 client.0 _ll_register_callbacks cb 0x55695753a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.913+0000 7f098ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.913+0000 7f098ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44896/0 (socket says 172.21.15.7:44896) 2024-06-13T06:58:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.913+0000 7f098ea00700 1 -- 172.21.15.7:0/2976926985 learned_addr learned my addr 172.21.15.7:0/2976926985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.914+0000 7f0987e00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 0x55695765e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.914+0000 7f098ea00700 1 -- 172.21.15.7:0/2976926985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576595f0 msgr2=0x55695765ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.914+0000 7f098ea00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576595f0 0x55695765ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.914+0000 7f098ea00700 1 -- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 msgr2=0x55695765e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.915+0000 7f098ea00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 0x55695765e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.915+0000 7f098ea00700 1 -- 172.21.15.7:0/2976926985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569576e46b0 con 0x55695757b1f0 2024-06-13T06:58:34.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.915+0000 7f0987e00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 0x55695765e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:34.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.915+0000 7f098ea00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0980000c00 tx=0x7f09800046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe84fb099f21254 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:34.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.915+0000 7f097cc00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:58:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.916+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098001d800 con 0x55695757b1f0 2024-06-13T06:58:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.916+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f098001d9a0 con 0x55695757b1f0 2024-06-13T06:58:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.917+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09780a6390 con 0x55695757b1f0 2024-06-13T06:58:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.917+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f098001dba0 con 0x55695757b1f0 2024-06-13T06:58:34.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.917+0000 7f09a024bb80 10 client.15465 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569576e5a10 con 0x55695757b1f0 2024-06-13T06:58:34.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09780a6390 con 0x55695757b1f0 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0980026400 con 0x55695757b1f0 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 10 client.15465 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 20 client.15465 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576e6140 0x5569576e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 4 client.15465 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.918+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5569575a8600 con 0x5569576e6140 2024-06-13T06:58:34.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.920+0000 7f0987e00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576e6140 0x5569576e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:34.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.921+0000 7f0987e00700 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576e6140 0x5569576e8560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5dbb11d7d2612c2 server_cookie=f95f93e47e527f7b in_seq=0 out_seq=0 2024-06-13T06:58:34.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.921+0000 7f097cc00700 10 client.15465 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:34.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.922+0000 7f097cc00700 1 -- 172.21.15.7:0/2976926985 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5569575a8600 con 0x5569576e6140 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.922+0000 7f097cc00700 10 client.15465 handle_command_reply: tid=0 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.922+0000 7f09a024bb80 1 client.15465 shutdown 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 msgr2=0x55695765e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0980000c00 tx=0x7f09800046a0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 shutdown_connections 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5569576e6140 0x5569576e8560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569576595f0 0x55695765ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55695757b1f0 0x55695765e230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 --2- 172.21.15.7:0/2976926985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5569576509e0 0x55695765e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 >> 172.21.15.7:0/2976926985 conn(0x556957651120 msgr2=0x556957656190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.923+0000 7f097cc00700 0 client.15465 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.924+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 shutdown_connections 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.924+0000 7f09a024bb80 1 -- 172.21.15.7:0/2976926985 wait complete. 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.924+0000 7f09a024bb80 20 client.15465 trim_cache size 0 max 16384 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T06:58:34.925 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f5608747-880f-47c6-86ae-5aa797222439", 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.925+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 msgr2=0x7f09740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.925+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 0x7f09740a5160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0988009710 tx=0x7f0988009490 comp rx=0 tx=0).stop 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.925+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 msgr2=0x7f099010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.925+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f097800ba90 comp rx=0 tx=0).stop 2024-06-13T06:58:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 shutdown_connections 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f099010ecc0 0x7f09901a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f09740a2ca0 0x7f09740a5160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0990101110 0x7f099010dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 --2- 172.21.15.7:0/1521648611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0990101ac0 0x7f099010e120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 >> 172.21.15.7:0/1521648611 conn(0x7f09900fcdb0 msgr2=0x7f099010ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 shutdown_connections 2024-06-13T06:58:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:34.926+0000 7f098fe00700 1 -- 172.21.15.7:0/1521648611 wait complete. 2024-06-13T06:58:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:35 smithi007 ceph-mon[32077]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-06-13T06:58:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:35 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:58:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:35 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:35 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:58:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:35 smithi189 ceph-mon[36616]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-06-13T06:58:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:35 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:58:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:35 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:35 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:58:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:35 smithi175 ceph-mon[36886]: pgmap v585: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 610 op/s 2024-06-13T06:58:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:35 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:58:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:35 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:35 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:58:36.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:58:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T06:58:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:58:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:58:36.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:36 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T06:58:36.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:58:36.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:58:36.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:36 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:36.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:36 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:58:37.109 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:58:37.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:37 smithi007 ceph-mon[32077]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-06-13T06:58:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:37 smithi189 ceph-mon[36616]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-06-13T06:58:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:37 smithi175 ceph-mon[36886]: pgmap v586: 129 pgs: 129 active+clean; 1.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 602 op/s 2024-06-13T06:58:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 155.0 s, 794.2 tps, lat 124.931 ms stddev 169.928 2024-06-13T06:58:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 -- 172.21.15.7:0/3643464849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 msgr2=0x7f7dd010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 --2- 172.21.15.7:0/3643464849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd010f320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009f90 tx=0x7f7dc800b330 comp rx=0 tx=0).stop 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 -- 172.21.15.7:0/3643464849 shutdown_connections 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 --2- 172.21.15.7:0/3643464849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd010f320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 --2- 172.21.15.7:0/3643464849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd0108500 0x7f7dd01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 --2- 172.21.15.7:0/3643464849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd006a170 0x7f7dd006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.515+0000 7f7dd7000700 1 -- 172.21.15.7:0/3643464849 >> 172.21.15.7:0/3643464849 conn(0x7f7dd0078340 msgr2=0x7f7dd0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.516+0000 7f7dd7000700 1 -- 172.21.15.7:0/3643464849 shutdown_connections 2024-06-13T06:58:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.516+0000 7f7dd7000700 1 -- 172.21.15.7:0/3643464849 wait complete. 2024-06-13T06:58:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 Processor -- start 2024-06-13T06:58:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 -- start start 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd0108500 0x7f7dd019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.518+0000 7f7dd7000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01127b0 con 0x7f7dd0108500 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.519+0000 7f7dd7000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0112630 con 0x7f7dd006ab30 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.519+0000 7f7dd7000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0112930 con 0x7f7dd006a170 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.519+0000 7f7dcfe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.519+0000 7f7dcfe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38790/0 (socket says 172.21.15.7:38790) 2024-06-13T06:58:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2340930075 learned_addr learned my addr 172.21.15.7:0/2340930075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2340930075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 msgr2=0x7f7dd019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd0108500 msgr2=0x7f7dd019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd0108500 0x7f7dd019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.520+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2340930075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd019fd90 con 0x7f7dd006a170 2024-06-13T06:58:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.522+0000 7f7dcf400700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.522+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000c9b0 tx=0x7f7dc000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.522+0000 7f7dccc00700 1 -- 172.21.15.7:0/2340930075 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0007b00 con 0x7f7dd006a170 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.523+0000 7f7dccc00700 1 -- 172.21.15.7:0/2340930075 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc0007ca0 con 0x7f7dd006a170 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.523+0000 7f7dccc00700 1 -- 172.21.15.7:0/2340930075 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0018660 con 0x7f7dd006a170 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.523+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd01a0080 con 0x7f7dd006a170 2024-06-13T06:58:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.523+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd006beb0 con 0x7f7dd006a170 2024-06-13T06:58:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.527+0000 7f7dccc00700 1 -- 172.21.15.7:0/2340930075 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7dc000f450 con 0x7f7dd006a170 2024-06-13T06:58:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.531+0000 7f7de1a64b80 1 Processor -- start 2024-06-13T06:58:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 -- start start 2024-06-13T06:58:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644eab2aa10 con 0x5644eac224b0 2024-06-13T06:58:39.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5644eab6db00 con 0x5644eac1c240 2024-06-13T06:58:39.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.532+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644eab288f0 con 0x5644eac21b90 2024-06-13T06:58:39.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:39 smithi007 ceph-mon[32077]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 663 op/s 2024-06-13T06:58:39.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38804/0 (socket says 172.21.15.7:38804) 2024-06-13T06:58:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 -- 172.21.15.7:0/4051414437 learned_addr learned my addr 172.21.15.7:0/4051414437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 -- 172.21.15.7:0/4051414437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 msgr2=0x5644eac1c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dccc00700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 0x7f7db40ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 -- 172.21.15.7:0/4051414437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 msgr2=0x5644eac1bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.533+0000 7f7dd5000700 1 -- 172.21.15.7:0/4051414437 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644eab15f90 con 0x5644eac21b90 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.534+0000 7f7dccc00700 1 -- 172.21.15.7:0/2340930075 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7dc00cf290 con 0x7f7dd006a170 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.534+0000 7f7dd5000700 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80094a0 tx=0x7f7dc80094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd5c0dff57cb2a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.535+0000 7f7dd5000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 0x7f7db40ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.535+0000 7f7dc5600700 1 -- 172.21.15.7:0/4051414437 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc801f070 con 0x5644eac21b90 2024-06-13T06:58:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.535+0000 7f7dc5600700 1 -- 172.21.15.7:0/4051414437 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc80050c0 con 0x5644eac21b90 2024-06-13T06:58:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.535+0000 7f7dc5600700 1 -- 172.21.15.7:0/4051414437 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80246b0 con 0x5644eac21b90 2024-06-13T06:58:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 -- 172.21.15.7:0/4051414437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 msgr2=0x5644eac21f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80094a0 tx=0x7f7dc80094d0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7dd5000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 0x7f7db40ad9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800bfd0 tx=0x7f7dc8003f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 -- 172.21.15.7:0/4051414437 shutdown_connections 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac21f70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/4051414437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac1bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 -- 172.21.15.7:0/4051414437 >> 172.21.15.7:0/4051414437 conn(0x5644eab48170 msgr2=0x5644eac16ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 -- 172.21.15.7:0/4051414437 shutdown_connections 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.536+0000 7f7de1a64b80 1 -- 172.21.15.7:0/4051414437 wait complete. 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.537+0000 7f7de1a64b80 1 -- start start 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.537+0000 7f7de1a64b80 1 Processor -- start 2024-06-13T06:58:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 0x5644eac282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac224b0 0x5644eac25b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644eab09f60 con 0x5644eac1c240 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5644eab69ac0 con 0x5644eac224b0 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644eac28e30 con 0x5644eac21b90 2024-06-13T06:58:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7de1a64b80 10 client.0 _ll_register_callbacks cb 0x5644eab03380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38806/0 (socket says 172.21.15.7:38806) 2024-06-13T06:58:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.538+0000 7f7dcf400700 1 -- 172.21.15.7:0/1558551065 learned_addr learned my addr 172.21.15.7:0/1558551065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 0x5644eac282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 -- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac224b0 msgr2=0x5644eac25b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac224b0 0x5644eac25b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 -- 172.21.15.7:0/1558551065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 msgr2=0x5644eac282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 0x5644eac282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 -- 172.21.15.7:0/1558551065 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644eaabc8d0 con 0x5644eac21b90 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 0x5644eac282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dcf400700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc012d20 tx=0x7f7dbc012e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67672a2cae68a8ac server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dba000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.539+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc00f870 con 0x5644eac21b90 2024-06-13T06:58:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc80050c0 con 0x5644eac21b90 2024-06-13T06:58:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7dbc00fa10 con 0x5644eac21b90 2024-06-13T06:58:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7dbc00fc10 con 0x5644eac21b90 2024-06-13T06:58:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7de1a64b80 10 client.25720 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644eac33740 con 0x5644eac21b90 2024-06-13T06:58:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.540+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc80050c0 con 0x5644eac21b90 2024-06-13T06:58:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7dbc01c420 con 0x5644eac21b90 2024-06-13T06:58:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 10 client.25720 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 20 client.25720 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eac35d70 0x5644eac381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 4 client.25720 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.541+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644eab7b000 con 0x5644eac35d70 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.542+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eac35d70 0x5644eac381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.542+0000 7f7dba000700 10 client.25720 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.542+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eac35d70 0x5644eac381c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95227633be756d4 server_cookie=52191fad167a09d1 in_seq=0 out_seq=0 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.544+0000 7f7dba000700 1 -- 172.21.15.7:0/1558551065 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644eab7b000 con 0x5644eac35d70 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.544+0000 7f7dba000700 10 client.25720 handle_command_reply: tid=0 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.544+0000 7f7de1a64b80 1 client.25720 shutdown 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 msgr2=0x5644eac255d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc012d20 tx=0x7f7dbc012e00 comp rx=0 tx=0).stop 2024-06-13T06:58:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 shutdown_connections 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eac35d70 0x5644eac381c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac224b0 0x5644eac25b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac1c240 0x5644eac282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/1558551065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac255d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 >> 172.21.15.7:0/1558551065 conn(0x5644eab48170 msgr2=0x5644eac10790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.547+0000 7f7dba000700 0 client.25720 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.548+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 shutdown_connections 2024-06-13T06:58:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.548+0000 7f7de1a64b80 1 -- 172.21.15.7:0/1558551065 wait complete. 2024-06-13T06:58:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.548+0000 7f7de1a64b80 20 client.25720 trim_cache size 0 max 16384 2024-06-13T06:58:39.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 Processor -- start 2024-06-13T06:58:39.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 -- start start 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0072d10 con 0x5644eac224b0 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00705a0 con 0x5644eac1c240 2024-06-13T06:58:39.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.594+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd010d4c0 con 0x5644eac21b90 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcf400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac1fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38820/0 (socket says 172.21.15.7:38820) 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcf400700 1 -- 172.21.15.7:0/2014708631 learned_addr learned my addr 172.21.15.7:0/2014708631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2014708631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 msgr2=0x5644eac1fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac1fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2014708631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 msgr2=0x5644eac11130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.595+0000 7f7dcfe00700 1 -- 172.21.15.7:0/2014708631 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644eac23b60 con 0x5644eac1c240 2024-06-13T06:58:39.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.597+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1bd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000afd0 tx=0x7f7dc000a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce18f91b5ab47858 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.597+0000 7f7dc4c00700 1 -- 172.21.15.7:0/2014708631 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00b3070 con 0x5644eac1c240 2024-06-13T06:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.597+0000 7f7dc4c00700 1 -- 172.21.15.7:0/2014708631 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc009fb00 con 0x5644eac1c240 2024-06-13T06:58:39.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.597+0000 7f7dc4c00700 1 -- 172.21.15.7:0/2014708631 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0086210 con 0x5644eac1c240 2024-06-13T06:58:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 -- 172.21.15.7:0/2014708631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 msgr2=0x5644eac1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1bd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000afd0 tx=0x7f7dc000a6e0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 -- 172.21.15.7:0/2014708631 shutdown_connections 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac11130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac1bd00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/2014708631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac1fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.598+0000 7f7de1a64b80 1 -- 172.21.15.7:0/2014708631 >> 172.21.15.7:0/2014708631 conn(0x5644eab48170 msgr2=0x5644eac228f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.599+0000 7f7de1a64b80 1 -- 172.21.15.7:0/2014708631 shutdown_connections 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.599+0000 7f7de1a64b80 1 -- 172.21.15.7:0/2014708631 wait complete. 2024-06-13T06:58:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.599+0000 7f7de1a64b80 1 -- start start 2024-06-13T06:58:39.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.600+0000 7f7de1a64b80 1 Processor -- start 2024-06-13T06:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac2e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006ea60 con 0x5644eac224b0 2024-06-13T06:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006fed0 con 0x5644eac1c240 2024-06-13T06:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd006f130 con 0x5644eac21b90 2024-06-13T06:58:39.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.601+0000 7f7de1a64b80 10 client.0 _ll_register_callbacks cb 0x5644eab03380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.602+0000 7f7dcfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.603+0000 7f7dcfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35770/0 (socket says 172.21.15.7:35770) 2024-06-13T06:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.603+0000 7f7dcfe00700 1 -- 172.21.15.7:0/3602177912 learned_addr learned my addr 172.21.15.7:0/3602177912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.603+0000 7f7dd5000700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.603+0000 7f7dcf400700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac2e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.604+0000 7f7dd5000700 1 -- 172.21.15.7:0/3602177912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 msgr2=0x5644eac2e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.605+0000 7f7dd5000700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac2e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.605+0000 7f7dd5000700 1 -- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 msgr2=0x5644eac2e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:39.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.605+0000 7f7dd5000700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.605+0000 7f7dd5000700 1 -- 172.21.15.7:0/3602177912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644eac3d950 con 0x5644eac224b0 2024-06-13T06:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.605+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.606+0000 7f7dd5000700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac3d0e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5644eac28d10 tx=0x7f7dc8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf877a37b0f145b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:39.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.606+0000 7f7dba000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.607+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8009e60 con 0x5644eac224b0 2024-06-13T06:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.607+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc009fb00 con 0x5644eac224b0 2024-06-13T06:58:39.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.607+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7dc801b830 con 0x5644eac224b0 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.608+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7dc80246b0 con 0x5644eac224b0 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.608+0000 7f7de1a64b80 10 client.15471 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.608+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644eaca05d0 con 0x5644eac224b0 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.608+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc009fb00 con 0x5644eac224b0 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7dc802e070 con 0x5644eac224b0 2024-06-13T06:58:39.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 20 client.15471 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eaca0960 0x5644eaca2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 4 client.15471 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:58:39.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.609+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644eab75580 con 0x5644eaca0960 2024-06-13T06:58:39.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.611+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eaca0960 0x5644eaca2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:39.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.612+0000 7f7dcfe00700 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eaca0960 0x5644eaca2d50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198bce52ecfe78d9 server_cookie=fcb715db3d1f794f in_seq=0 out_seq=0 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.612+0000 7f7dba000700 10 client.15471 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.612+0000 7f7dba000700 1 -- 172.21.15.7:0/3602177912 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5644eab75580 con 0x5644eaca0960 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.612+0000 7f7dba000700 10 client.15471 handle_command_reply: tid=0 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.612+0000 7f7de1a64b80 1 client.15471 shutdown 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.614+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 msgr2=0x5644eac3d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.614+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac3d0e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5644eac28d10 tx=0x7f7dc8009b60 comp rx=0 tx=0).stop 2024-06-13T06:58:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 shutdown_connections 2024-06-13T06:58:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644eaca0960 0x5644eaca2d50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7dba000700 0 client.15471 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644eac224b0 0x5644eac3d0e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644eac1c240 0x5644eac2e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 --2- 172.21.15.7:0/3602177912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644eac21b90 0x5644eac2e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.615+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 >> 172.21.15.7:0/3602177912 conn(0x5644eab48170 msgr2=0x5644eac23120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.616+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 shutdown_connections 2024-06-13T06:58:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.617+0000 7f7de1a64b80 1 -- 172.21.15.7:0/3602177912 wait complete. 2024-06-13T06:58:39.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.617+0000 7f7de1a64b80 20 client.15471 trim_cache size 0 max 16384 2024-06-13T06:58:39.618 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "f5608747-880f-47c6-86ae-5aa797222439": { 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "f5608747-880f-47c6-86ae-5aa797222439", 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:58:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 msgr2=0x7f7db40ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 0x7f7db40ad9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800bfd0 tx=0x7f7dc8003f40 comp rx=0 tx=0).stop 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 msgr2=0x7f7dd019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000c9b0 tx=0x7f7dc000ccc0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 shutdown_connections 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dd0108500 0x7f7dd019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dd006a170 0x7f7dd019ad80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7db40ab530 0x7f7db40ad9f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 --2- 172.21.15.7:0/2340930075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dd006ab30 0x7f7dd019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.619+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 >> 172.21.15.7:0/2340930075 conn(0x7f7dd0078340 msgr2=0x7f7dd00ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.620+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 shutdown_connections 2024-06-13T06:58:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:58:39.620+0000 7f7dd7000700 1 -- 172.21.15.7:0/2340930075 wait complete. 2024-06-13T06:58:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:39 smithi189 ceph-mon[36616]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 663 op/s 2024-06-13T06:58:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:39 smithi175 ceph-mon[36886]: pgmap v587: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 663 op/s 2024-06-13T06:58:40.479 INFO:tasks.cephfs.filesystem:scrub status for tag:f5608747-880f-47c6-86ae-5aa797222439 - {'path': '/', 'tag': 'f5608747-880f-47c6-86ae-5aa797222439', 'options': 'recursive,force'} 2024-06-13T06:58:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 -- 172.21.15.7:0/1552933036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10dcc0 msgr2=0x7f0e1c10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 --2- 172.21.15.7:0/1552933036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c10e140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009ca0 tx=0x7f0e1402a8d0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 -- 172.21.15.7:0/1552933036 shutdown_connections 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 --2- 172.21.15.7:0/1552933036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c10e140 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 --2- 172.21.15.7:0/1552933036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c10b310 0x7f0e1c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 --2- 172.21.15.7:0/1552933036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.816+0000 7f0e23200640 1 -- 172.21.15.7:0/1552933036 >> 172.21.15.7:0/1552933036 conn(0x7f0e1c06ea10 msgr2=0x7f0e1c070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.817+0000 7f0e23200640 1 -- 172.21.15.7:0/1552933036 shutdown_connections 2024-06-13T06:58:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.817+0000 7f0e23200640 1 -- 172.21.15.7:0/1552933036 wait complete. 2024-06-13T06:58:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.818+0000 7f0e23200640 1 Processor -- start 2024-06-13T06:58:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.819+0000 7f0e23200640 1 -- start start 2024-06-13T06:58:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c1a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10b310 0x7f0e1c1a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c1adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c11c4e0 con 0x7f0e1c1089a0 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c11c360 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e23200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c11c660 con 0x7f0e1c10b310 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e21200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c1adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10b310 0x7f0e1c1a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c1a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c1a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44026/0 (socket says 172.21.15.7:44026) 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.820+0000 7f0e1be00640 1 -- 172.21.15.7:0/599597434 learned_addr learned my addr 172.21.15.7:0/599597434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 -- 172.21.15.7:0/599597434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10b310 msgr2=0x7f0e1c1a6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10b310 0x7f0e1c1a6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 -- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 msgr2=0x7f0e1c1a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c1a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 -- 172.21.15.7:0/599597434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c1ae5e0 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.821+0000 7f0e21200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c1adf70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402adf0 tx=0x7f0e14003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.822+0000 7f0e19600640 1 -- 172.21.15.7:0/599597434 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003d90 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.822+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c1ae870 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.822+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1aee10 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.822+0000 7f0e19600640 1 -- 172.21.15.7:0/599597434 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e140098a0 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.822+0000 7f0e19600640 1 -- 172.21.15.7:0/599597434 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402fab0 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.829+0000 7f0e19600640 1 -- 172.21.15.7:0/599597434 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0e1402fc50 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.832+0000 7f0e19600640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 0x7f0e000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.832+0000 7f0e19600640 1 -- 172.21.15.7:0/599597434 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0e140f07c0 con 0x7f0e1c10dcc0 2024-06-13T06:58:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.832+0000 7f0e1be00640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 0x7f0e000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.832+0000 7f0e25fc2740 1 Processor -- start 2024-06-13T06:58:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.833+0000 7f0e25fc2740 1 -- start start 2024-06-13T06:58:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.833+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.833+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.833+0000 7f0e1be00640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 0x7f0e000a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009c90 tx=0x7f0e0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.833+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e1003760 0x55f3e1005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 -- 172.21.15.7:0/4208449053 learned_addr learned my addr 172.21.15.7:0/4208449053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3e0f61420 con 0x55f3e0ffdc00 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f3e0f08c50 con 0x55f3e1003760 2024-06-13T06:58:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f3e0f5d3e0 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e21200640 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e1003760 0x55f3e1005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 -- 172.21.15.7:0/4208449053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 msgr2=0x55f3e10030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 -- 172.21.15.7:0/4208449053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e1003760 msgr2=0x55f3e1005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e1003760 0x55f3e1005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 -- 172.21.15.7:0/4208449053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e0c75210 con 0x55f3e0ffdc00 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e13e00640 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.834+0000 7f0e1be00640 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009fc0 tx=0x7f0e0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c36596117743048 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.835+0000 7f0e11600640 1 -- 172.21.15.7:0/4208449053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c02b070 con 0x55f3e0ffdc00 2024-06-13T06:58:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.835+0000 7f0e11600640 1 -- 172.21.15.7:0/4208449053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c026b00 con 0x55f3e0ffdc00 2024-06-13T06:58:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.835+0000 7f0e11600640 1 -- 172.21.15.7:0/4208449053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00eb50 con 0x55f3e0ffdc00 2024-06-13T06:58:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:41 smithi189 ceph-mon[36616]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 710 op/s 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 msgr2=0x55f3e0ffe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009fc0 tx=0x7f0e0c014000 comp rx=0 tx=0).stop 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 shutdown_connections 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e1003760 0x55f3e1005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/4208449053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffdc00 0x55f3e0ffe020 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 >> 172.21.15.7:0/4208449053 conn(0x55f3e0fc5fb0 msgr2=0x55f3e0ff9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 shutdown_connections 2024-06-13T06:58:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.836+0000 7f0e25fc2740 1 -- 172.21.15.7:0/4208449053 wait complete. 2024-06-13T06:58:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.837+0000 7f0e25fc2740 1 -- start start 2024-06-13T06:58:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.838+0000 7f0e25fc2740 1 Processor -- start 2024-06-13T06:58:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e1007fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e100f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f3e1006710 con 0x55f3e1003760 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f3e0f1c0b0 con 0x55f3e0ffdc00 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f3e1006890 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e25fc2740 10 client.0 _ll_register_callbacks cb 0x55f3e0f07f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45950/0 (socket says 172.21.15.7:45950) 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.840+0000 7f0e13e00640 1 -- 172.21.15.7:0/3363353672 learned_addr learned my addr 172.21.15.7:0/3363353672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e1007fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e21200640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e100f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 -- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 msgr2=0x55f3e1007fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e1007fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 -- 172.21.15.7:0/3363353672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 msgr2=0x55f3e100f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e100f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 -- 172.21.15.7:0/3363353672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e0d41620 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e1007fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e13e00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400bad0 tx=0x7f0e0400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9031953f0df0f6e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.841+0000 7f0e0a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003dc0 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0e04005020 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c026b00 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e04024050 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e25fc2740 10 client.25738 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.842+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3e1012900 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c026b00 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0e0401b470 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 10 client.25738 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 20 client.25738 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e1014100 0x55f3e1016560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 4 client.25738 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.843+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e0f6eb50 con 0x55f3e1014100 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.844+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e1014100 0x55f3e1016560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.845+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e1014100 0x55f3e1016560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6052a30e2185f20 server_cookie=bcafab216f5c6aeb in_seq=0 out_seq=0 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.845+0000 7f0e0a000640 10 client.25738 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.846+0000 7f0e0a000640 1 -- 172.21.15.7:0/3363353672 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f3e0f6eb50 con 0x55f3e1014100 2024-06-13T06:58:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.846+0000 7f0e0a000640 10 client.25738 handle_command_reply: tid=0 2024-06-13T06:58:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.846+0000 7f0e25fc2740 1 client.25738 shutdown 2024-06-13T06:58:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.847+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 msgr2=0x55f3e10084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.847+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400bad0 tx=0x7f0e0400bf90 comp rx=0 tx=0).stop 2024-06-13T06:58:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.847+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 shutdown_connections 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e1014100 0x55f3e1016560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e100f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffe5f0 0x55f3e10084e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e0a000640 0 client.25738 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3363353672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e1007fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 >> 172.21.15.7:0/3363353672 conn(0x55f3e0fc5fb0 msgr2=0x55f3e0ff8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 shutdown_connections 2024-06-13T06:58:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3363353672 wait complete. 2024-06-13T06:58:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.848+0000 7f0e25fc2740 20 client.25738 trim_cache size 0 max 16384 2024-06-13T06:58:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:41 smithi175 ceph-mon[36886]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 710 op/s 2024-06-13T06:58:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.875+0000 7f0e25fc2740 1 Processor -- start 2024-06-13T06:58:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 -- start start 2024-06-13T06:58:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 0x55f3e10079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e0ff9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8012310 con 0x55f3e1003760 2024-06-13T06:58:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0df800fbe0 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.876+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0df800da30 con 0x55f3e0ffdc00 2024-06-13T06:58:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34948/0 (socket says 172.21.15.7:34948) 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e21200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e0ff9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e13e00640 1 -- 172.21.15.7:0/1547459840 learned_addr learned my addr 172.21.15.7:0/1547459840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 0x55f3e10079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 -- 172.21.15.7:0/1547459840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 msgr2=0x55f3e100f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 -- 172.21.15.7:0/1547459840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 msgr2=0x55f3e0ff9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e0ff9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.877+0000 7f0e1be00640 1 -- 172.21.15.7:0/1547459840 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e1008f90 con 0x55f3e0ffdc00 2024-06-13T06:58:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.878+0000 7f0e1be00640 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 0x55f3e10079a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c011ca0 tx=0x7f0e0c01cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e50fa18937dd06 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.879+0000 7f0e10c00640 1 -- 172.21.15.7:0/1547459840 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c02b040 con 0x55f3e0ffdc00 2024-06-13T06:58:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.879+0000 7f0e10c00640 1 -- 172.21.15.7:0/1547459840 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c027e00 con 0x55f3e0ffdc00 2024-06-13T06:58:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.879+0000 7f0e10c00640 1 -- 172.21.15.7:0/1547459840 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c004e20 con 0x55f3e0ffdc00 2024-06-13T06:58:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.881+0000 7f0e25fc2740 1 -- 172.21.15.7:0/1547459840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 msgr2=0x55f3e10079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.881+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 0x55f3e10079a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c011ca0 tx=0x7f0e0c01cda0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.882+0000 7f0e25fc2740 1 -- 172.21.15.7:0/1547459840 shutdown_connections 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.882+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e1003760 0x55f3e0ff9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.882+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.882+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/1547459840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e0ffdc00 0x55f3e10079a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.882+0000 7f0e25fc2740 1 -- 172.21.15.7:0/1547459840 >> 172.21.15.7:0/1547459840 conn(0x55f3e0fc5fb0 msgr2=0x55f3e1007290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.883+0000 7f0e25fc2740 1 -- 172.21.15.7:0/1547459840 shutdown_connections 2024-06-13T06:58:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.883+0000 7f0e25fc2740 1 -- 172.21.15.7:0/1547459840 wait complete. 2024-06-13T06:58:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.884+0000 7f0e25fc2740 1 -- start start 2024-06-13T06:58:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.884+0000 7f0e25fc2740 1 Processor -- start 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.884+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e1003760 0x55f3e100da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df800e0a0 con 0x55f3e0ffe5f0 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0df800f510 con 0x55f3e0ffdc00 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0df800e770 con 0x55f3e1003760 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e25fc2740 10 client.0 _ll_register_callbacks cb 0x7f0e04004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34960/0 (socket says 172.21.15.7:34960) 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 -- 172.21.15.7:0/3646077865 learned_addr learned my addr 172.21.15.7:0/3646077865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e21200640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e1003760 0x55f3e100da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 -- 172.21.15.7:0/3646077865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e1003760 msgr2=0x55f3e100da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e1003760 0x55f3e100da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 -- 172.21.15.7:0/3646077865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffe5f0 msgr2=0x55f3e100cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.885+0000 7f0e1be00640 1 -- 172.21.15.7:0/3646077865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e100e020 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e1be00640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c027fd0 tx=0x7f0e0c004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27159991d9d483f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e0a000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c02b070 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0e0c00e5e0 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c027e00 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c046050 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e25fc2740 10 client.25763 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.886+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3e100f330 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c00ecc0 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0e0c03bc10 con 0x55f3e0ffdc00 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 10 client.25763 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 20 client.25763 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e107ee40 0x55f3e1081230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 4 client.25763 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:58:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0e0c027e00 con 0x55f3e107ee40 2024-06-13T06:58:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.887+0000 7f0e13e00640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e107ee40 0x55f3e1081230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.888+0000 7f0e13e00640 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e107ee40 0x55f3e1081230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f57b4a630d1929 server_cookie=ee464711d6b287c0 in_seq=0 out_seq=0 2024-06-13T06:58:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.888+0000 7f0e0a000640 10 client.25763 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:41.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:41 smithi007 ceph-mon[32077]: pgmap v588: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 710 op/s 2024-06-13T06:58:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.984+0000 7f0e0a000640 1 -- 172.21.15.7:0/3646077865 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f0e0c027e00 con 0x55f3e107ee40 2024-06-13T06:58:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.984+0000 7f0e0a000640 10 client.25763 handle_command_reply: tid=0 2024-06-13T06:58:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.984+0000 7f0e25fc2740 1 client.25763 shutdown 2024-06-13T06:58:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.984+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 msgr2=0x55f3e100c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.984+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c027fd0 tx=0x7f0e0c004870 comp rx=0 tx=0).stop 2024-06-13T06:58:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 shutdown_connections 2024-06-13T06:58:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f3e107ee40 0x55f3e1081230 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f3e1003760 0x55f3e100da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3e0ffe5f0 0x55f3e100cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 --2- 172.21.15.7:0/3646077865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f3e0ffdc00 0x55f3e100c5a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e0a000640 0 client.25763 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.986+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 >> 172.21.15.7:0/3646077865 conn(0x55f3e0fc5fb0 msgr2=0x55f3e0ff9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.987+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 shutdown_connections 2024-06-13T06:58:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.987+0000 7f0e25fc2740 1 -- 172.21.15.7:0/3646077865 wait complete. 2024-06-13T06:58:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.987+0000 7f0e25fc2740 20 client.25763 trim_cache size 0 max 16384 2024-06-13T06:58:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:58:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:58:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.988+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 msgr2=0x7f0e000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.988+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 0x7f0e000a5170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009c90 tx=0x7f0e0c005f90 comp rx=0 tx=0).stop 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.988+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 msgr2=0x7f0e1c1adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.988+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c1adf70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402adf0 tx=0x7f0e14003ba0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 shutdown_connections 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0e000a2cb0 0x7f0e000a5170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e1c10dcc0 0x7f0e1c1adf70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e1c10b310 0x7f0e1c1a6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 --2- 172.21.15.7:0/599597434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e1c1089a0 0x7f0e1c1a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 >> 172.21.15.7:0/599597434 conn(0x7f0e1c06ea10 msgr2=0x7f0e1c10c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 shutdown_connections 2024-06-13T06:58:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:41.989+0000 7f0e23200640 1 -- 172.21.15.7:0/599597434 wait complete. 2024-06-13T06:58:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 160.0 s, 948.6 tps, lat 104.611 ms stddev 119.982 2024-06-13T06:58:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:43 smithi189 ceph-mon[36616]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-06-13T06:58:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:58:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:43 smithi175 ceph-mon[36886]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-06-13T06:58:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:58:43.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:43 smithi007 ceph-mon[32077]: pgmap v589: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-06-13T06:58:43.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:43.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:43.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:58:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:45 smithi189 ceph-mon[36616]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 738 op/s 2024-06-13T06:58:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:45 smithi175 ceph-mon[36886]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 738 op/s 2024-06-13T06:58:45.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:45 smithi007 ceph-mon[32077]: pgmap v590: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 738 op/s 2024-06-13T06:58:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:47 smithi189 ceph-mon[36616]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-06-13T06:58:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:47 smithi175 ceph-mon[36886]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-06-13T06:58:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 165.0 s, 1109.8 tps, lat 92.236 ms stddev 99.986 2024-06-13T06:58:47.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:47 smithi007 ceph-mon[32077]: pgmap v591: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 673 op/s 2024-06-13T06:58:49.716 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:49 smithi007 ceph-mon[32077]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 639 op/s 2024-06-13T06:58:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:49 smithi189 ceph-mon[36616]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 639 op/s 2024-06-13T06:58:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:49 smithi175 ceph-mon[36886]: pgmap v592: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 639 op/s 2024-06-13T06:58:51.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:51 smithi007 ceph-mon[32077]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-06-13T06:58:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:51 smithi189 ceph-mon[36616]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-06-13T06:58:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:51 smithi175 ceph-mon[36886]: pgmap v593: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-06-13T06:58:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 170.0 s, 1029.3 tps, lat 90.637 ms stddev 110.764 2024-06-13T06:58:53.463 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:53 smithi007 ceph-mon[32077]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-13T06:58:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:53 smithi189 ceph-mon[36616]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-13T06:58:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:53 smithi175 ceph-mon[36886]: pgmap v594: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.7 MiB/s wr, 529 op/s 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.193+0000 7f49fce00640 1 -- 172.21.15.7:0/2689753601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f81035b0 msgr2=0x7f49f8109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.193+0000 7f49fce00640 1 --2- 172.21.15.7:0/2689753601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f81035b0 0x7f49f8109e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009ca0 tx=0x7f49f002a8c0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 -- 172.21.15.7:0/2689753601 shutdown_connections 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 --2- 172.21.15.7:0/2689753601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f81035b0 0x7f49f8109e40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 --2- 172.21.15.7:0/2689753601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f8102bf0 0x7f49f8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 --2- 172.21.15.7:0/2689753601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101a00 0x7f49f8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 -- 172.21.15.7:0/2689753601 >> 172.21.15.7:0/2689753601 conn(0x7f49f80fcf70 msgr2=0x7f49f80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 -- 172.21.15.7:0/2689753601 shutdown_connections 2024-06-13T06:58:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.194+0000 7f49fce00640 1 -- 172.21.15.7:0/2689753601 wait complete. 2024-06-13T06:58:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.196+0000 7f49fce00640 1 Processor -- start 2024-06-13T06:58:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.196+0000 7f49fce00640 1 -- start start 2024-06-13T06:58:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.196+0000 7f49fce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101a00 0x7f49f819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.196+0000 7f49fce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.196+0000 7f49fce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f81035b0 0x7f49f81a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49fce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80751c0 con 0x7f49f8101a00 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49fce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8075040 con 0x7f49f8102bf0 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49fce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8075340 con 0x7f49f81035b0 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56370/0 (socket says 172.21.15.7:56370) 2024-06-13T06:58:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 -- 172.21.15.7:0/1616631334 learned_addr learned my addr 172.21.15.7:0/1616631334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f6800640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f81035b0 0x7f49f81a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 -- 172.21.15.7:0/1616631334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f81035b0 msgr2=0x7f49f81a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f81035b0 0x7f49f81a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 -- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101a00 msgr2=0x7f49f819e1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101a00 0x7f49f819e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.197+0000 7f49f5400640 1 -- 172.21.15.7:0/1616631334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a5e00 con 0x7f49f8102bf0 2024-06-13T06:58:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49f5400640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49eea00640 1 -- 172.21.15.7:0/1616631334 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cd80 con 0x7f49f8102bf0 2024-06-13T06:58:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49eea00640 1 -- 172.21.15.7:0/1616631334 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4010910 con 0x7f49f8102bf0 2024-06-13T06:58:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a60f0 con 0x7f49f8102bf0 2024-06-13T06:58:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49eea00640 1 -- 172.21.15.7:0/1616631334 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4018980 con 0x7f49f8102bf0 2024-06-13T06:58:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.198+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a6640 con 0x7f49f8102bf0 2024-06-13T06:58:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.201+0000 7f49eea00640 1 -- 172.21.15.7:0/1616631334 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f49e4010ac0 con 0x7f49f8102bf0 2024-06-13T06:58:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.204+0000 7f49eea00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 0x7f49dc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.204+0000 7f49eea00640 1 -- 172.21.15.7:0/1616631334 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49e40cf7d0 con 0x7f49f8102bf0 2024-06-13T06:58:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.204+0000 7f49f5e00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 0x7f49dc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49f5e00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 0x7f49dc0a5140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49f8102a50 tx=0x7f49e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 Processor -- start 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 -- start start 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 0x56528509b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565284ffe420 con 0x56528509b5f0 2024-06-13T06:58:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565284fa5c50 con 0x5652850a0760 2024-06-13T06:58:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.206+0000 7f49ffc82740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565284ffa3e0 con 0x56528509ac00 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41772/0 (socket says 172.21.15.7:41772) 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 -- 172.21.15.7:0/2920541298 learned_addr learned my addr 172.21.15.7:0/2920541298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5e00640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 0x56528509b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f6800640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 -- 172.21.15.7:0/2920541298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 msgr2=0x56528509b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 0x56528509b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 -- 172.21.15.7:0/2920541298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 msgr2=0x5652850a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.207+0000 7f49f5400640 1 -- 172.21.15.7:0/2920541298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565284d12210 con 0x56528509b5f0 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.208+0000 7f49f5e00640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 0x56528509b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.208+0000 7f49f5400640 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a4430 tx=0x7f49e409c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c8f10b3da87d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.208+0000 7f49e3400640 1 -- 172.21.15.7:0/2920541298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40a4960 con 0x56528509b5f0 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.209+0000 7f49e3400640 1 -- 172.21.15.7:0/2920541298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4099ff0 con 0x56528509b5f0 2024-06-13T06:58:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.209+0000 7f49e3400640 1 -- 172.21.15.7:0/2920541298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4010690 con 0x56528509b5f0 2024-06-13T06:58:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 -- 172.21.15.7:0/2920541298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 msgr2=0x5652850a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a4430 tx=0x7f49e409c4b0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 -- 172.21.15.7:0/2920541298 shutdown_connections 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x5652850a00f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2920541298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509ac00 0x56528509b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.210+0000 7f49ffc82740 1 -- 172.21.15.7:0/2920541298 >> 172.21.15.7:0/2920541298 conn(0x565285062fb0 msgr2=0x565285096680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.211+0000 7f49ffc82740 1 -- 172.21.15.7:0/2920541298 shutdown_connections 2024-06-13T06:58:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.211+0000 7f49ffc82740 1 -- 172.21.15.7:0/2920541298 wait complete. 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.211+0000 7f49ffc82740 1 -- start start 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 Processor -- start 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5652850a3710 con 0x56528509ac00 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565284fb90b0 con 0x5652850a0760 2024-06-13T06:58:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5652850a3890 con 0x56528509b5f0 2024-06-13T06:58:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.212+0000 7f49ffc82740 10 client.0 _ll_register_callbacks cb 0x565284fa4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56392/0 (socket says 172.21.15.7:56392) 2024-06-13T06:58:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 -- 172.21.15.7:0/413121274 learned_addr learned my addr 172.21.15.7:0/413121274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f5400640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f5e00640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 -- 172.21.15.7:0/413121274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 msgr2=0x5652850a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 -- 172.21.15.7:0/413121274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 msgr2=0x5652850a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.213+0000 7f49f6800640 1 -- 172.21.15.7:0/413121274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565284dde620 con 0x5652850a0760 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.214+0000 7f49f6800640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009ca0 tx=0x7f49f00319a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73492e2745c74c6b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.214+0000 7f49d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:58:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.214+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0003d80 con 0x5652850a0760 2024-06-13T06:58:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.214+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49f0004040 con 0x5652850a0760 2024-06-13T06:58:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.216+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e4099ff0 con 0x5652850a0760 2024-06-13T06:58:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.216+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49f0004240 con 0x5652850a0760 2024-06-13T06:58:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.216+0000 7f49ffc82740 10 client.25775 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.217+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652850af8d0 con 0x5652850a0760 2024-06-13T06:58:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.217+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4099ff0 con 0x5652850a0760 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.217+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f49f0044070 con 0x5652850a0760 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.217+0000 7f49ffc82740 10 client.25775 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.217+0000 7f49ffc82740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.218+0000 7f49ffc82740 20 client.25775 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.218+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5652850b1010 0x5652850b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.218+0000 7f49ffc82740 4 client.25775 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.218+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56528500bb50 con 0x5652850b1010 2024-06-13T06:58:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.218+0000 7f49f5e00640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5652850b1010 0x5652850b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.219+0000 7f49d7e00640 10 client.25775 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.219+0000 7f49f5e00640 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5652850b1010 0x5652850b3490 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5eed0df0c00602 server_cookie=72d2c7a30d19247 in_seq=0 out_seq=0 2024-06-13T06:58:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.220+0000 7f49d7e00640 1 -- 172.21.15.7:0/413121274 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56528500bb50 con 0x5652850b1010 2024-06-13T06:58:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.220+0000 7f49d7e00640 10 client.25775 handle_command_reply: tid=0 2024-06-13T06:58:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.220+0000 7f49ffc82740 1 client.25775 shutdown 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 msgr2=0x5652850ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009ca0 tx=0x7f49f00319a0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 shutdown_connections 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5652850b1010 0x5652850b3490 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x5652850ac560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49d7e00640 0 client.25775 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 --2- 172.21.15.7:0/413121274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.221+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 >> 172.21.15.7:0/413121274 conn(0x565285062fb0 msgr2=0x565285095840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.222+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 shutdown_connections 2024-06-13T06:58:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.222+0000 7f49ffc82740 1 -- 172.21.15.7:0/413121274 wait complete. 2024-06-13T06:58:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.222+0000 7f49ffc82740 20 client.25775 trim_cache size 0 max 16384 2024-06-13T06:58:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.254+0000 7f49ffc82740 1 Processor -- start 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 -- start start 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 0x56528509c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x56528509ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 0x5652850ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b32f0 con 0x56528509b5f0 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b0bc0 con 0x56528509ac00 2024-06-13T06:58:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49ffc82740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81aea10 con 0x5652850a0760 2024-06-13T06:58:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 0x5652850ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 0x5652850ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53376/0 (socket says 172.21.15.7:53376) 2024-06-13T06:58:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.255+0000 7f49f6800640 1 -- 172.21.15.7:0/713841992 learned_addr learned my addr 172.21.15.7:0/713841992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.256+0000 7f49f5e00640 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 0x56528509c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.256+0000 7f49f5400640 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x56528509ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.257+0000 7f49f5e00640 1 -- 172.21.15.7:0/713841992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 msgr2=0x5652850ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.257+0000 7f49f5e00640 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 0x5652850ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.257+0000 7f49f5e00640 1 -- 172.21.15.7:0/713841992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 msgr2=0x56528509ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.257+0000 7f49f5e00640 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x56528509ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.257+0000 7f49f5e00640 1 -- 172.21.15.7:0/713841992 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56528509fc30 con 0x56528509ac00 2024-06-13T06:58:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.258+0000 7f49f5e00640 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 0x56528509c970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5652850b3ca0 tx=0x7f49e8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e759bd064bae752d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.261+0000 7f49e2a00640 1 -- 172.21.15.7:0/713841992 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80285b0 con 0x56528509ac00 2024-06-13T06:58:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.261+0000 7f49e2a00640 1 -- 172.21.15.7:0/713841992 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8028c30 con 0x56528509ac00 2024-06-13T06:58:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.261+0000 7f49e2a00640 1 -- 172.21.15.7:0/713841992 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8027de0 con 0x56528509ac00 2024-06-13T06:58:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.265+0000 7f49ffc82740 1 -- 172.21.15.7:0/713841992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 msgr2=0x56528509c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.265+0000 7f49ffc82740 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 0x56528509c970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5652850b3ca0 tx=0x7f49e8011000 comp rx=0 tx=0).stop 2024-06-13T06:58:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.266+0000 7f49ffc82740 1 -- 172.21.15.7:0/713841992 shutdown_connections 2024-06-13T06:58:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.266+0000 7f49ffc82740 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652850a0760 0x5652850ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.266+0000 7f49ffc82740 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509b5f0 0x56528509ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.266+0000 7f49ffc82740 1 --2- 172.21.15.7:0/713841992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56528509ac00 0x56528509c970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.266+0000 7f49ffc82740 1 -- 172.21.15.7:0/713841992 >> 172.21.15.7:0/713841992 conn(0x565285062fb0 msgr2=0x565285096100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.269+0000 7f49ffc82740 1 -- 172.21.15.7:0/713841992 shutdown_connections 2024-06-13T06:58:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.269+0000 7f49ffc82740 1 -- 172.21.15.7:0/713841992 wait complete. 2024-06-13T06:58:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.269+0000 7f49ffc82740 1 -- start start 2024-06-13T06:58:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.270+0000 7f49ffc82740 1 Processor -- start 2024-06-13T06:58:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.273+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.273+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.274+0000 7f49ffc82740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x56528512ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.275+0000 7f49f6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x56528512ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.275+0000 7f49f6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x56528512ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56406/0 (socket says 172.21.15.7:56406) 2024-06-13T06:58:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.276+0000 7f49f6800640 1 -- 172.21.15.7:0/2206377315 learned_addr learned my addr 172.21.15.7:0/2206377315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:58:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.276+0000 7f49f5400640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.276+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81af080 con 0x56528509ac00 2024-06-13T06:58:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.276+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81b04f0 con 0x5652850a0760 2024-06-13T06:58:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.276+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81af750 con 0x56528509b5f0 2024-06-13T06:58:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.277+0000 7f49ffc82740 10 client.0 _ll_register_callbacks cb 0x7f49e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:58:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.279+0000 7f49f5e00640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.279+0000 7f49f5400640 1 -- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 msgr2=0x56528512ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.279+0000 7f49f5400640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x56528512ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.279+0000 7f49f5400640 1 -- 172.21.15.7:0/2206377315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 msgr2=0x5652850a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:58:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.280+0000 7f49f5400640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.280+0000 7f49f5400640 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56528512b270 con 0x56528509b5f0 2024-06-13T06:58:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.280+0000 7f49f5400640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a7af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a4430 tx=0x7f49e409b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a991ff3fb9d401fb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:58:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e409b340 con 0x56528509b5f0 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49e4018b20 con 0x56528509b5f0 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e8028c30 con 0x56528509b5f0 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49e40a4960 con 0x56528509b5f0 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49ffc82740 10 client.25762 fetch_fsmap learned FSMap version 30 2024-06-13T06:58:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.283+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56528512c5c0 con 0x56528509b5f0 2024-06-13T06:58:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.284+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8028c30 con 0x56528509b5f0 2024-06-13T06:58:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.284+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f49e4097a40 con 0x56528509b5f0 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 10 client.25762 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 20 client.25762 populate_metadata read hostname 'smithi007' 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56528512ca90 0x56528512ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 4 client.25762 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:58:55.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.285+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565285005ea0 con 0x56528512ca90 2024-06-13T06:58:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.286+0000 7f49f5e00640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56528512ca90 0x56528512ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:58:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.287+0000 7f49f5e00640 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56528512ca90 0x56528512ee80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2a40caaf5ffec4 server_cookie=702aaf75442193c4 in_seq=0 out_seq=0 2024-06-13T06:58:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.287+0000 7f49d7e00640 10 client.25762 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49d7e00640 1 -- 172.21.15.7:0/2206377315 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565285005ea0 con 0x56528512ca90 2024-06-13T06:58:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49d7e00640 10 client.25762 handle_command_reply: tid=0 2024-06-13T06:58:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 client.25762 shutdown 2024-06-13T06:58:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 msgr2=0x5652850a7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a7af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a4430 tx=0x7f49e409b780 comp rx=0 tx=0).stop 2024-06-13T06:58:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 shutdown_connections 2024-06-13T06:58:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56528512ca90 0x56528512ee80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5652850a0760 0x56528512ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56528509b5f0 0x5652850a7af0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 --2- 172.21.15.7:0/2206377315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56528509ac00 0x5652850a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 >> 172.21.15.7:0/2206377315 conn(0x565285062fb0 msgr2=0x56528509c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.419+0000 7f49d7e00640 0 client.25762 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.420+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 shutdown_connections 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.420+0000 7f49ffc82740 1 -- 172.21.15.7:0/2206377315 wait complete. 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.420+0000 7f49ffc82740 20 client.25762 trim_cache size 0 max 16384 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:58:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:58:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.421+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 msgr2=0x7f49dc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.421+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 0x7f49dc0a5140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49f8102a50 tx=0x7f49e8005f90 comp rx=0 tx=0).stop 2024-06-13T06:58:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.421+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 msgr2=0x7f49f819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:58:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.421+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400eeb0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 shutdown_connections 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49dc0a2c80 0x7f49dc0a5140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49f81035b0 0x7f49f81a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f8102bf0 0x7f49f819e720 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 --2- 172.21.15.7:0/1616631334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101a00 0x7f49f819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 >> 172.21.15.7:0/1616631334 conn(0x7f49f80fcf70 msgr2=0x7f49f80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 shutdown_connections 2024-06-13T06:58:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:58:55.422+0000 7f49fce00640 1 -- 172.21.15.7:0/1616631334 wait complete. 2024-06-13T06:58:55.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:55 smithi189 ceph-mon[36616]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-06-13T06:58:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:55 smithi175 ceph-mon[36886]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-06-13T06:58:55.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:55 smithi007 ceph-mon[32077]: pgmap v595: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-06-13T06:58:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:57 smithi189 ceph-mon[36616]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-06-13T06:58:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:57 smithi175 ceph-mon[36886]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-06-13T06:58:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 175.0 s, 1271.5 tps, lat 83.279 ms stddev 124.827 2024-06-13T06:58:57.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:57 smithi007 ceph-mon[32077]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-06-13T06:58:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:58:59 smithi189 ceph-mon[36616]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-06-13T06:58:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:58:59 smithi175 ceph-mon[36886]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-06-13T06:58:59.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:58:59 smithi007 ceph-mon[32077]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 736 op/s 2024-06-13T06:59:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:01 smithi189 ceph-mon[36616]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 762 op/s 2024-06-13T06:59:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:01 smithi175 ceph-mon[36886]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 762 op/s 2024-06-13T06:59:01.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:01 smithi007 ceph-mon[32077]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 762 op/s 2024-06-13T06:59:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 180.0 s, 1382.0 tps, lat 72.505 ms stddev 82.237 2024-06-13T06:59:03.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:03 smithi007 ceph-mon[32077]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-06-13T06:59:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:03 smithi189 ceph-mon[36616]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-06-13T06:59:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:03 smithi175 ceph-mon[36886]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-06-13T06:59:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:05 smithi189 ceph-mon[36616]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.8 MiB/s wr, 796 op/s 2024-06-13T06:59:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:05 smithi175 ceph-mon[36886]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.8 MiB/s wr, 796 op/s 2024-06-13T06:59:05.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:05 smithi007 ceph-mon[32077]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.8 MiB/s wr, 796 op/s 2024-06-13T06:59:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:07 smithi189 ceph-mon[36616]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 764 op/s 2024-06-13T06:59:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:07 smithi175 ceph-mon[36886]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 764 op/s 2024-06-13T06:59:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 185.0 s, 1156.6 tps, lat 85.794 ms stddev 91.494 2024-06-13T06:59:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:07 smithi007 ceph-mon[32077]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 764 op/s 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.605+0000 7f5765e00640 1 -- 172.21.15.7:0/2860293300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760103c30 msgr2=0x7f5760104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.605+0000 7f5765e00640 1 --2- 172.21.15.7:0/2860293300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760103c30 0x7f5760104050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5750009900 tx=0x7f575002a410 comp rx=0 tx=0).stop 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 -- 172.21.15.7:0/2860293300 shutdown_connections 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 --2- 172.21.15.7:0/2860293300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57601057e0 0x7f576010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 --2- 172.21.15.7:0/2860293300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5760104e20 0x7f57601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 --2- 172.21.15.7:0/2860293300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760103c30 0x7f5760104050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 -- 172.21.15.7:0/2860293300 >> 172.21.15.7:0/2860293300 conn(0x7f57600ff1c0 msgr2=0x7f57601015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 -- 172.21.15.7:0/2860293300 shutdown_connections 2024-06-13T06:59:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.606+0000 7f5765e00640 1 -- 172.21.15.7:0/2860293300 wait complete. 2024-06-13T06:59:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.607+0000 7f5765e00640 1 Processor -- start 2024-06-13T06:59:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 -- start start 2024-06-13T06:59:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5760103c30 0x7f576019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57601057e0 0x7f57601a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f576010f8d0 con 0x7f5760103c30 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f576010f750 con 0x7f57601057e0 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.608+0000 7f5765e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f576010fa50 con 0x7f5760104e20 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57601057e0 0x7f57601a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50134/0 (socket says 172.21.15.7:50134) 2024-06-13T06:59:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 -- 172.21.15.7:0/2163770574 learned_addr learned my addr 172.21.15.7:0/2163770574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575ea00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5760103c30 0x7f576019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 -- 172.21.15.7:0/2163770574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57601057e0 msgr2=0x7f57601a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57601057e0 0x7f57601a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 -- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5760103c30 msgr2=0x7f576019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.609+0000 7f575e000640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5760103c30 0x7f576019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f575e000640 1 -- 172.21.15.7:0/2163770574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57601a5f40 con 0x7f5760104e20 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f575e000640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e9f0 tx=0x7f574c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f5764a00640 1 -- 172.21.15.7:0/2163770574 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c00cd80 con 0x7f5760104e20 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f5764a00640 1 -- 172.21.15.7:0/2163770574 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574c010910 con 0x7f5760104e20 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57601a6230 con 0x7f5760104e20 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f5764a00640 1 -- 172.21.15.7:0/2163770574 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c018990 con 0x7f5760104e20 2024-06-13T06:59:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.610+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57601a67d0 con 0x7f5760104e20 2024-06-13T06:59:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.616+0000 7f5764a00640 1 -- 172.21.15.7:0/2163770574 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f574c018b30 con 0x7f5760104e20 2024-06-13T06:59:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.616+0000 7f5768bd3740 1 Processor -- start 2024-06-13T06:59:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.616+0000 7f5768bd3740 1 -- start start 2024-06-13T06:59:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.617+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.617+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 0x55fd1a3cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.617+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f5768bd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a32a420 con 0x55fd1a3c6c00 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f5768bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a2d1c50 con 0x55fd1a3c75f0 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f5768bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a3263e0 con 0x55fd1a3cc760 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f5764a00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 0x7f57440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f5764a00640 1 -- 172.21.15.7:0/2163770574 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f574c0cf630 con 0x7f5760104e20 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49346/0 (socket says 172.21.15.7:49346) 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575f400640 1 -- 172.21.15.7:0/2748488151 learned_addr learned my addr 172.21.15.7:0/2748488151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 0x55fd1a3cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575e000640 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 -- 172.21.15.7:0/2748488151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 msgr2=0x55fd1a3cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 -- 172.21.15.7:0/2748488151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 msgr2=0x55fd1a3c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575ea00640 1 -- 172.21.15.7:0/2748488151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1a03e210 con 0x55fd1a3c75f0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.618+0000 7f575f400640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 0x7f57440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.619+0000 7f575ea00640 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 0x55fd1a3cc0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5750009230 tx=0x7f57500040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5b0587842754da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.619+0000 7f574be00640 1 -- 172.21.15.7:0/2748488151 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750035040 con 0x55fd1a3c75f0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.619+0000 7f574be00640 1 -- 172.21.15.7:0/2748488151 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5750005520 con 0x55fd1a3c75f0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.619+0000 7f574be00640 1 -- 172.21.15.7:0/2748488151 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750031640 con 0x55fd1a3c75f0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.619+0000 7f575f400640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 0x7f57440a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55fd1a3c7e50 tx=0x7f5758004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 -- 172.21.15.7:0/2748488151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 msgr2=0x55fd1a3cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 0x55fd1a3cc0f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5750009230 tx=0x7f57500040c0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 -- 172.21.15.7:0/2748488151 shutdown_connections 2024-06-13T06:59:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c75f0 0x55fd1a3cc0f0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5750009230 tx=0x7f57500040c0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2748488151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 -- 172.21.15.7:0/2748488151 >> 172.21.15.7:0/2748488151 conn(0x55fd1a38efb0 msgr2=0x55fd1a3c2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.621+0000 7f5768bd3740 1 -- 172.21.15.7:0/2748488151 shutdown_connections 2024-06-13T06:59:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.622+0000 7f5768bd3740 1 -- 172.21.15.7:0/2748488151 wait complete. 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.622+0000 7f5768bd3740 1 -- start start 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.622+0000 7f5768bd3740 1 Processor -- start 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a3cf6b0 con 0x55fd1a3d66a0 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a2e50b0 con 0x55fd1a3c6c00 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd1a3cf830 con 0x55fd1a3cc760 2024-06-13T06:59:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f5768bd3740 10 client.0 _ll_register_callbacks cb 0x55fd1a2d0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40342/0 (socket says 172.21.15.7:40342) 2024-06-13T06:59:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 -- 172.21.15.7:0/659517200 learned_addr learned my addr 172.21.15.7:0/659517200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575e000640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575f400640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 -- 172.21.15.7:0/659517200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 msgr2=0x55fd1a3d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 -- 172.21.15.7:0/659517200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 msgr2=0x55fd1a3d8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.623+0000 7f575ea00640 1 -- 172.21.15.7:0/659517200 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1a10a620 con 0x55fd1a3c6c00 2024-06-13T06:59:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.624+0000 7f575ea00640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5750005ab0 tx=0x7f5750005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee87fea30aa0caed server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.624+0000 7f5748c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.624+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750035050 con 0x55fd1a3c6c00 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.624+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5750005520 con 0x55fd1a3c6c00 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.624+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5750004f00 con 0x55fd1a3c6c00 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.625+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5750005100 con 0x55fd1a3c6c00 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.625+0000 7f5768bd3740 10 client.25793 fetch_fsmap learned FSMap version 30 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.625+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd1a3dbe00 con 0x55fd1a3c6c00 2024-06-13T06:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.625+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5750005520 con 0x55fd1a3c6c00 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f57500035f0 con 0x55fd1a3c6c00 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 10 client.25793 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 10 client.25793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 20 client.25793 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a3dd150 0x55fd1a3df540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 4 client.25793 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.626+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd1a337b50 con 0x55fd1a3dd150 2024-06-13T06:59:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.627+0000 7f575e000640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a3dd150 0x55fd1a3df540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.628+0000 7f575e000640 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a3dd150 0x55fd1a3df540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0bfca423bf40629 server_cookie=59c54d09e977d2d in_seq=0 out_seq=0 2024-06-13T06:59:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.628+0000 7f5748c00640 10 client.25793 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5748c00640 1 -- 172.21.15.7:0/659517200 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fd1a337b50 con 0x55fd1a3dd150 2024-06-13T06:59:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5748c00640 10 client.25793 handle_command_reply: tid=0 2024-06-13T06:59:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5768bd3740 1 client.25793 shutdown 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 msgr2=0x55fd1a3d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5750005ab0 tx=0x7f5750005900 comp rx=0 tx=0).stop 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.629+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 shutdown_connections 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a3dd150 0x55fd1a3df540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 --2- 172.21.15.7:0/659517200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3d1130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5748c00640 0 client.25793 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 >> 172.21.15.7:0/659517200 conn(0x55fd1a38efb0 msgr2=0x55fd1a3c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 shutdown_connections 2024-06-13T06:59:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 1 -- 172.21.15.7:0/659517200 wait complete. 2024-06-13T06:59:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.630+0000 7f5768bd3740 20 client.25793 trim_cache size 0 max 16384 2024-06-13T06:59:08.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.657+0000 7f5768bd3740 1 Processor -- start 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.657+0000 7f5768bd3740 1 -- start start 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57601b3290 con 0x55fd1a3d66a0 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57601b0b60 con 0x55fd1a3c6c00 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f5768bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57601ae9b0 con 0x55fd1a3cc760 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f575f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f575e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f575f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49366/0 (socket says 172.21.15.7:49366) 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f575f400640 1 -- 172.21.15.7:0/236070555 learned_addr learned my addr 172.21.15.7:0/236070555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.658+0000 7f575ea00640 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575f400640 1 -- 172.21.15.7:0/236070555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 msgr2=0x55fd1a3d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575f400640 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575f400640 1 -- 172.21.15.7:0/236070555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 msgr2=0x55fd1a3c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575f400640 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575f400640 1 -- 172.21.15.7:0/236070555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd1a3c1f70 con 0x55fd1a3d66a0 2024-06-13T06:59:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.659+0000 7f575ea00640 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.660+0000 7f575f400640 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57580126b0 tx=0x7f5758009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1d7ab48251284c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.661+0000 7f574b400640 1 -- 172.21.15.7:0/236070555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758003b70 con 0x55fd1a3d66a0 2024-06-13T06:59:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.661+0000 7f574b400640 1 -- 172.21.15.7:0/236070555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5758003d10 con 0x55fd1a3d66a0 2024-06-13T06:59:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.661+0000 7f574b400640 1 -- 172.21.15.7:0/236070555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758021cf0 con 0x55fd1a3d66a0 2024-06-13T06:59:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 -- 172.21.15.7:0/236070555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 msgr2=0x55fd1a3d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57580126b0 tx=0x7f5758009a00 comp rx=0 tx=0).stop 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 -- 172.21.15.7:0/236070555 shutdown_connections 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3d1420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 --2- 172.21.15.7:0/236070555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 -- 172.21.15.7:0/236070555 >> 172.21.15.7:0/236070555 conn(0x55fd1a38efb0 msgr2=0x55fd1a3c2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 -- 172.21.15.7:0/236070555 shutdown_connections 2024-06-13T06:59:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.663+0000 7f5768bd3740 1 -- 172.21.15.7:0/236070555 wait complete. 2024-06-13T06:59:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.664+0000 7f5768bd3740 1 -- start start 2024-06-13T06:59:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.665+0000 7f5768bd3740 1 Processor -- start 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57601af020 con 0x55fd1a3d66a0 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57601b0490 con 0x55fd1a3c6c00 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57601af6f0 con 0x55fd1a3cc760 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f5768bd3740 10 client.0 _ll_register_callbacks cb 0x7f574c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f575ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f575ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40350/0 (socket says 172.21.15.7:40350) 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f575ea00640 1 -- 172.21.15.7:0/2037245001 learned_addr learned my addr 172.21.15.7:0/2037245001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.667+0000 7f575e000640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 -- 172.21.15.7:0/2037245001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 msgr2=0x55fd1a3ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3ee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 -- 172.21.15.7:0/2037245001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 msgr2=0x55fd1a3ef9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 -- 172.21.15.7:0/2037245001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd1a3cf970 con 0x55fd1a3c6c00 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.668+0000 7f575ea00640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5750006010 tx=0x7f5750005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4347f66b45e7478a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.669+0000 7f5748c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.669+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750035040 con 0x55fd1a3c6c00 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.669+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f57500054a0 con 0x55fd1a3c6c00 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.669+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5758003d10 con 0x55fd1a3c6c00 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.669+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5750064050 con 0x55fd1a3c6c00 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 10 client.25799 fetch_fsmap learned FSMap version 30 2024-06-13T06:59:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd1a3f10f0 con 0x55fd1a3c6c00 2024-06-13T06:59:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5758003d10 con 0x55fd1a3c6c00 2024-06-13T06:59:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5750031840 con 0x55fd1a3c6c00 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 10 client.25799 fetch_fsmap finished waiting for FSMap version 30 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 20 client.25799 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a447ca0 0x55fd1a44a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 4 client.25799 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.671+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd1a331ea0 con 0x55fd1a447ca0 2024-06-13T06:59:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.672+0000 7f575e000640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a447ca0 0x55fd1a44a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.673+0000 7f5748c00640 10 client.25799 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.673+0000 7f575e000640 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a447ca0 0x55fd1a44a0c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff379d693bd0f3c server_cookie=d0249dd3e75c8146 in_seq=0 out_seq=0 2024-06-13T06:59:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.804+0000 7f5748c00640 1 -- 172.21.15.7:0/2037245001 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fd1a331ea0 con 0x55fd1a447ca0 2024-06-13T06:59:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.804+0000 7f5748c00640 10 client.25799 handle_command_reply: tid=0 2024-06-13T06:59:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.804+0000 7f5768bd3740 1 client.25799 shutdown 2024-06-13T06:59:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.804+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 msgr2=0x55fd1a3ee470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.804+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5750006010 tx=0x7f5750005c00 comp rx=0 tx=0).stop 2024-06-13T06:59:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 shutdown_connections 2024-06-13T06:59:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fd1a447ca0 0x55fd1a44a0c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd1a3d66a0 0x55fd1a3ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd1a3cc760 0x55fd1a3ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 --2- 172.21.15.7:0/2037245001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd1a3c6c00 0x55fd1a3ee470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 >> 172.21.15.7:0/2037245001 conn(0x55fd1a38efb0 msgr2=0x55fd1a3d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5748c00640 0 client.25799 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 shutdown_connections 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 1 -- 172.21.15.7:0/2037245001 wait complete. 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.805+0000 7f5768bd3740 20 client.25799 trim_cache size 0 max 16384 2024-06-13T06:59:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.806+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 msgr2=0x7f57440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.806+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 0x7f57440a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55fd1a3c7e50 tx=0x7f5758004ab0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.806+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 msgr2=0x7f576019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.806+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e9f0 tx=0x7f574c00eeb0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 shutdown_connections 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f57440a2cb0 0x7f57440a5170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57601057e0 0x7f57601a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5760104e20 0x7f576019e860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 --2- 172.21.15.7:0/2163770574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5760103c30 0x7f576019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 >> 172.21.15.7:0/2163770574 conn(0x7f57600ff1c0 msgr2=0x7f5760077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 shutdown_connections 2024-06-13T06:59:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:08.807+0000 7f5765e00640 1 -- 172.21.15.7:0/2163770574 wait complete. 2024-06-13T06:59:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:09 smithi189 ceph-mon[36616]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-06-13T06:59:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:09 smithi175 ceph-mon[36886]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-06-13T06:59:09.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:09 smithi007 ceph-mon[32077]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-06-13T06:59:10.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:59:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:11.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:11 smithi007 ceph-mon[32077]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T06:59:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:11 smithi189 ceph-mon[36616]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T06:59:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:11 smithi175 ceph-mon[36886]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T06:59:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 190.0 s, 1138.8 tps, lat 83.660 ms stddev 122.599 2024-06-13T06:59:13.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:13 smithi007 ceph-mon[32077]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 555 op/s 2024-06-13T06:59:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:13 smithi189 ceph-mon[36616]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 555 op/s 2024-06-13T06:59:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:13 smithi175 ceph-mon[36886]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 555 op/s 2024-06-13T06:59:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:14 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:14 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:14.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:14 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:15.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.560+0000 7fc605600700 1 -- 172.21.15.7:0/2045358087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 msgr2=0x7fc600115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.560+0000 7fc605600700 1 --2- 172.21.15.7:0/2045358087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc600115df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800d490 tx=0x7fc5f800d7a0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.561+0000 7fc605600700 1 -- 172.21.15.7:0/2045358087 shutdown_connections 2024-06-13T06:59:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.562+0000 7fc605600700 1 --2- 172.21.15.7:0/2045358087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc600115df0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.562+0000 7fc605600700 1 --2- 172.21.15.7:0/2045358087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 0x7fc600109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.562+0000 7fc605600700 1 --2- 172.21.15.7:0/2045358087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 0x7fc60010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.562+0000 7fc605600700 1 -- 172.21.15.7:0/2045358087 >> 172.21.15.7:0/2045358087 conn(0x7fc60006d600 msgr2=0x7fc60006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.564+0000 7fc605600700 1 -- 172.21.15.7:0/2045358087 shutdown_connections 2024-06-13T06:59:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.564+0000 7fc605600700 1 -- 172.21.15.7:0/2045358087 wait complete. 2024-06-13T06:59:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.567+0000 7fc605600700 1 Processor -- start 2024-06-13T06:59:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.568+0000 7fc605600700 1 -- start start 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 0x7fc6001a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 0x7fc6001a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc600118e60 con 0x7fc60010a0a0 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc600118ce0 con 0x7fc6001096f0 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc5ff400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc5ff400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57730/0 (socket says 172.21.15.7:57730) 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc5ff400700 1 -- 172.21.15.7:0/4147494345 learned_addr learned my addr 172.21.15.7:0/4147494345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc600118fe0 con 0x7fc60010e220 2024-06-13T06:59:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.569+0000 7fc5fea00700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 0x7fc6001a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5ff400700 1 -- 172.21.15.7:0/4147494345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 msgr2=0x7fc6001a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5ff400700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 0x7fc6001a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5ff400700 1 -- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 msgr2=0x7fc6001a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5fe000700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 0x7fc6001a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5ff400700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 0x7fc6001a3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.571+0000 7fc5ff400700 1 -- 172.21.15.7:0/4147494345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6001a3210 con 0x7fc60010e220 2024-06-13T06:59:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.572+0000 7fc5ff400700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800a040 tx=0x7fc5f8003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.572+0000 7fc5f7400700 1 -- 172.21.15.7:0/4147494345 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8015030 con 0x7fc60010e220 2024-06-13T06:59:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.572+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6001a8570 con 0x7fc60010e220 2024-06-13T06:59:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.572+0000 7fc5f7400700 1 -- 172.21.15.7:0/4147494345 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8011b40 con 0x7fc60010e220 2024-06-13T06:59:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.572+0000 7fc5f7400700 1 -- 172.21.15.7:0/4147494345 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8005940 con 0x7fc60010e220 2024-06-13T06:59:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.574+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a8ab0 con 0x7fc60010e220 2024-06-13T06:59:15.577 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:15 smithi007 ceph-mon[32077]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-06-13T06:59:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.580+0000 7fc5f7400700 1 -- 172.21.15.7:0/4147494345 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc5f8018070 con 0x7fc60010e220 2024-06-13T06:59:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.581+0000 7fc60ff71b80 1 Processor -- start 2024-06-13T06:59:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.583+0000 7fc5f7400700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 0x7fc5e40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.583+0000 7fc5f7400700 1 -- 172.21.15.7:0/4147494345 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5f80d0840 con 0x7fc60010e220 2024-06-13T06:59:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.583+0000 7fc5fe000700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 0x7fc5e40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 -- start start 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607a27dfa10 con 0x5607a28d1240 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607a2822b00 con 0x5607a28d74b0 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607a27dd8f0 con 0x5607a28d6b90 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52560/0 (socket says 172.21.15.7:52560) 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.584+0000 7fc5fe000700 1 -- 172.21.15.7:0/1774699816 learned_addr learned my addr 172.21.15.7:0/1774699816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5ff400700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 -- 172.21.15.7:0/1774699816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 msgr2=0x5607a28d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 -- 172.21.15.7:0/1774699816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 msgr2=0x5607a28d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.585+0000 7fc5fea00700 1 -- 172.21.15.7:0/1774699816 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a27caf90 con 0x5607a28d6b90 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.586+0000 7fc5fe000700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.586+0000 7fc5fe000700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 0x7fc5e40a51e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006fd0 tx=0x7fc5ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.586+0000 7fc5fea00700 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28d6f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000bc80 tx=0x7fc5f000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4259a5c98dc15367 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.587+0000 7fc5ebe00700 1 -- 172.21.15.7:0/1774699816 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0010ca0 con 0x5607a28d6b90 2024-06-13T06:59:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.587+0000 7fc5ebe00700 1 -- 172.21.15.7:0/1774699816 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0004510 con 0x5607a28d6b90 2024-06-13T06:59:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.587+0000 7fc5ebe00700 1 -- 172.21.15.7:0/1774699816 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0005280 con 0x5607a28d6b90 2024-06-13T06:59:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.588+0000 7fc60ff71b80 1 -- 172.21.15.7:0/1774699816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 msgr2=0x5607a28d6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.588+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28d6f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000bc80 tx=0x7fc5f000bf90 comp rx=0 tx=0).stop 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 -- 172.21.15.7:0/1774699816 shutdown_connections 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28d6f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/1774699816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 -- 172.21.15.7:0/1774699816 >> 172.21.15.7:0/1774699816 conn(0x5607a27fd170 msgr2=0x5607a28cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 -- 172.21.15.7:0/1774699816 shutdown_connections 2024-06-13T06:59:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.589+0000 7fc60ff71b80 1 -- 172.21.15.7:0/1774699816 wait complete. 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.590+0000 7fc60ff71b80 1 -- start start 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.590+0000 7fc60ff71b80 1 Processor -- start 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607a27bef60 con 0x5607a28d1240 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607a281eac0 con 0x5607a28d74b0 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607a28dde90 con 0x5607a28d6b90 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.591+0000 7fc60ff71b80 10 client.0 _ll_register_callbacks cb 0x5607a27b8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5fea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28da590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57756/0 (socket says 172.21.15.7:57756) 2024-06-13T06:59:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5fe000700 1 -- 172.21.15.7:0/2150340275 learned_addr learned my addr 172.21.15.7:0/2150340275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 -- 172.21.15.7:0/2150340275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 msgr2=0x5607a28da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 -- 172.21.15.7:0/2150340275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 msgr2=0x5607a28dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.592+0000 7fc5ff400700 1 -- 172.21.15.7:0/2150340275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a27718d0 con 0x5607a28d74b0 2024-06-13T06:59:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc5fea00700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc5ff400700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28daad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5f809ca90 tx=0x7fc5f800d7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea90e9a275757337 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc5e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b9030 con 0x5607a28d74b0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5f80ce0b0 con 0x5607a28d74b0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.593+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f0004510 con 0x5607a28d74b0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.594+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5f80074c0 con 0x5607a28d74b0 2024-06-13T06:59:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.594+0000 7fc60ff71b80 10 client.25811 fetch_fsmap learned FSMap version 31 2024-06-13T06:59:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.594+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607a28e84e0 con 0x5607a28d74b0 2024-06-13T06:59:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.595+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0004510 con 0x5607a28d74b0 2024-06-13T06:59:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.595+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc5f80a55e0 con 0x5607a28d74b0 2024-06-13T06:59:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.595+0000 7fc60ff71b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T06:59:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.595+0000 7fc60ff71b80 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.595+0000 7fc60ff71b80 20 client.25811 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.596+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a28eaaf0 0x5607a28ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.596+0000 7fc60ff71b80 4 client.25811 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.596+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607a2830000 con 0x5607a28eaaf0 2024-06-13T06:59:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.597+0000 7fc5fea00700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a28eaaf0 0x5607a28ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.598+0000 7fc5fea00700 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a28eaaf0 0x5607a28ecf50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffd359691a094c6 server_cookie=9c7cbcbb90a52e68 in_seq=0 out_seq=0 2024-06-13T06:59:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.598+0000 7fc5e8c00700 10 client.25811 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.599+0000 7fc5e8c00700 1 -- 172.21.15.7:0/2150340275 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607a2830000 con 0x5607a28eaaf0 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.599+0000 7fc5e8c00700 10 client.25811 handle_command_reply: tid=0 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.599+0000 7fc60ff71b80 1 client.25811 shutdown 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 msgr2=0x5607a28daad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28daad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5f809ca90 tx=0x7fc5f800d7a0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 shutdown_connections 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a28eaaf0 0x5607a28ecf50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d74b0 0x5607a28daad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d1240 0x5607a28dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2150340275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d6b90 0x5607a28da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc5e8c00700 0 client.25811 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.600+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 >> 172.21.15.7:0/2150340275 conn(0x5607a27fd170 msgr2=0x5607a28c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.601+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 shutdown_connections 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.601+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2150340275 wait complete. 2024-06-13T06:59:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.601+0000 7fc60ff71b80 20 client.25811 trim_cache size 0 max 16384 2024-06-13T06:59:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.648+0000 7fc60ff71b80 1 Processor -- start 2024-06-13T06:59:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 -- start start 2024-06-13T06:59:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bf780 con 0x5607a28d74b0 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bd010 con 0x5607a28d6b90 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.649+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bae60 con 0x5607a28d1240 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57770/0 (socket says 172.21.15.7:57770) 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fea00700 1 -- 172.21.15.7:0/2214257725 learned_addr learned my addr 172.21.15.7:0/2214257725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 -- 172.21.15.7:0/2214257725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 msgr2=0x5607a28d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 -- 172.21.15.7:0/2214257725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 msgr2=0x5607a28d4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fe000700 1 -- 172.21.15.7:0/2214257725 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a28d8b20 con 0x5607a28d6b90 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.650+0000 7fc5fea00700 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.651+0000 7fc5fe000700 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28cc1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00a4d0 tx=0x7fc5ec00f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d09277553d850f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.651+0000 7fc5eb400700 1 -- 172.21.15.7:0/2214257725 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec01e040 con 0x5607a28d6b90 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.651+0000 7fc5eb400700 1 -- 172.21.15.7:0/2214257725 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec027070 con 0x5607a28d6b90 2024-06-13T06:59:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.651+0000 7fc5eb400700 1 -- 172.21.15.7:0/2214257725 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec022dd0 con 0x5607a28d6b90 2024-06-13T06:59:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.653+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2214257725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 msgr2=0x5607a28cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.653+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28cc1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00a4d0 tx=0x7fc5ec00f000 comp rx=0 tx=0).stop 2024-06-13T06:59:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.653+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2214257725 shutdown_connections 2024-06-13T06:59:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28d4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/2214257725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28cc1a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2214257725 >> 172.21.15.7:0/2214257725 conn(0x5607a27fd170 msgr2=0x5607a28cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2214257725 shutdown_connections 2024-06-13T06:59:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.654+0000 7fc60ff71b80 1 -- 172.21.15.7:0/2214257725 wait complete. 2024-06-13T06:59:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.655+0000 7fc60ff71b80 1 -- start start 2024-06-13T06:59:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.655+0000 7fc60ff71b80 1 Processor -- start 2024-06-13T06:59:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bb4d0 con 0x5607a28d74b0 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bc940 con 0x5607a28d6b90 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001bbba0 con 0x5607a28d1240 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc60ff71b80 10 client.0 _ll_register_callbacks cb 0x5607a27b8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc5fe000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52584/0 (socket says 172.21.15.7:52584) 2024-06-13T06:59:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.656+0000 7fc5fe000700 1 -- 172.21.15.7:0/880055417 learned_addr learned my addr 172.21.15.7:0/880055417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.657+0000 7fc5fe000700 1 -- 172.21.15.7:0/880055417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 msgr2=0x5607a28e3940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:59:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.657+0000 7fc5fe000700 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28e3940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.657+0000 7fc5fe000700 1 -- 172.21.15.7:0/880055417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 msgr2=0x5607a28f26f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:15.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.658+0000 7fc5fe000700 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.659+0000 7fc5fe000700 1 -- 172.21.15.7:0/880055417 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a28f2f30 con 0x5607a28d6b90 2024-06-13T06:59:15.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.660+0000 7fc5fe000700 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0048a0 tx=0x7fc5ec00c210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4b243b1b42f121e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.660+0000 7fc5e8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.660+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec031070 con 0x5607a28d6b90 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.660+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5ec014870 con 0x5607a28d6b90 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.660+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ec027070 con 0x5607a28d6b90 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.661+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc5ec022e20 con 0x5607a28d6b90 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.661+0000 7fc60ff71b80 10 client.25817 fetch_fsmap learned FSMap version 31 2024-06-13T06:59:15.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.661+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607a2955870 con 0x5607a28d6b90 2024-06-13T06:59:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.662+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec027070 con 0x5607a28d6b90 2024-06-13T06:59:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.662+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fc5ec021070 con 0x5607a28d6b90 2024-06-13T06:59:15.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 20 client.25817 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a2955eb0 0x5607a29582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 4 client.25817 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.663+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607a282a580 con 0x5607a2955eb0 2024-06-13T06:59:15.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.664+0000 7fc5fea00700 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a2955eb0 0x5607a29582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.665+0000 7fc5fea00700 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a2955eb0 0x5607a29582d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a85e25ccdc7407 server_cookie=e7af4383a8398e4c in_seq=0 out_seq=0 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.665+0000 7fc5e8c00700 10 client.25817 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.665+0000 7fc5e8c00700 1 -- 172.21.15.7:0/880055417 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607a282a580 con 0x5607a2955eb0 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.665+0000 7fc5e8c00700 10 client.25817 handle_command_reply: tid=0 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.666+0000 7fc60ff71b80 1 client.25817 shutdown 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.666+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 msgr2=0x5607a28f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.666+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0048a0 tx=0x7fc5ec00c210 comp rx=0 tx=0).stop 2024-06-13T06:59:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 shutdown_connections 2024-06-13T06:59:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607a2955eb0 0x5607a29582d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607a28d74b0 0x5607a28f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a28d1240 0x5607a28e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 --2- 172.21.15.7:0/880055417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607a28d6b90 0x5607a28f21b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.667+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 >> 172.21.15.7:0/880055417 conn(0x5607a27fd170 msgr2=0x5607a28d7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.669+0000 7fc5e8c00700 0 client.25817 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.671+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 shutdown_connections 2024-06-13T06:59:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.671+0000 7fc60ff71b80 1 -- 172.21.15.7:0/880055417 wait complete. 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.671+0000 7fc60ff71b80 20 client.25817 trim_cache size 0 max 16384 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:59:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.672+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 msgr2=0x7fc5e40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.672+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 0x7fc5e40a51e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006fd0 tx=0x7fc5ec008040 comp rx=0 tx=0).stop 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.672+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 msgr2=0x7fc6001a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.672+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800a040 tx=0x7fc5f8003f30 comp rx=0 tx=0).stop 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 shutdown_connections 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc60010e220 0x7fc6001a7f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2d20 0x7fc5e40a51e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6001096f0 0x7fc6001a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 --2- 172.21.15.7:0/4147494345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc60010a0a0 0x7fc6001a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 >> 172.21.15.7:0/4147494345 conn(0x7fc60006d600 msgr2=0x7fc600072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 shutdown_connections 2024-06-13T06:59:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:15.674+0000 7fc605600700 1 -- 172.21.15.7:0/4147494345 wait complete. 2024-06-13T06:59:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:15 smithi189 ceph-mon[36616]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-06-13T06:59:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:15 smithi175 ceph-mon[36886]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-06-13T06:59:17.446 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T06:59:17.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T06:59:17.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:17 smithi007 ceph-mon[32077]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-06-13T06:59:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:17 smithi189 ceph-mon[36616]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-06-13T06:59:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:17 smithi175 ceph-mon[36886]: pgmap v606: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 MiB/s wr, 683 op/s 2024-06-13T06:59:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 195.0 s, 667.6 tps, lat 155.656 ms stddev 275.463 2024-06-13T06:59:18.119 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:19 smithi189 ceph-mon[36616]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-06-13T06:59:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:19 smithi175 ceph-mon[36886]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-06-13T06:59:19.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:19 smithi007 ceph-mon[32077]: pgmap v607: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-06-13T06:59:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:21 smithi189 ceph-mon[36616]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:21 smithi175 ceph-mon[36886]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:21.894 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:21 smithi007 ceph-mon[32077]: pgmap v608: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.023+0000 7f4605000640 1 -- 172.21.15.7:0/4154275583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600102480 msgr2=0x7f4600104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.023+0000 7f4605000640 1 --2- 172.21.15.7:0/4154275583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600102480 0x7f4600104870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009860 tx=0x7f45f002a340 comp rx=0 tx=0).stop 2024-06-13T06:59:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 -- 172.21.15.7:0/4154275583 shutdown_connections 2024-06-13T06:59:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 --2- 172.21.15.7:0/4154275583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46001076e0 0x7f4600109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 --2- 172.21.15.7:0/4154275583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4600104db0 0x7f46001071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 --2- 172.21.15.7:0/4154275583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600102480 0x7f4600104870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 -- 172.21.15.7:0/4154275583 >> 172.21.15.7:0/4154275583 conn(0x7f46000fbe90 msgr2=0x7f46000fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.025+0000 7f4605000640 1 -- 172.21.15.7:0/4154275583 shutdown_connections 2024-06-13T06:59:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.026+0000 7f4605000640 1 -- 172.21.15.7:0/4154275583 wait complete. 2024-06-13T06:59:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.027+0000 7f4605000640 1 Processor -- start 2024-06-13T06:59:22.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 -- start start 2024-06-13T06:59:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 0x7f4600195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600104db0 0x7f4600195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 0x7f460019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d680 con 0x7f4600102480 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d500 con 0x7f4600104db0 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.031+0000 7f4605000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f460010d800 con 0x7f46001076e0 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.032+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 0x7f460019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.032+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 0x7f460019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54322/0 (socket says 172.21.15.7:54322) 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.032+0000 7f45fe800640 1 -- 172.21.15.7:0/2588656233 learned_addr learned my addr 172.21.15.7:0/2588656233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.033+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 0x7f4600195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.033+0000 7f45fd400640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600104db0 0x7f4600195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 -- 172.21.15.7:0/2588656233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 msgr2=0x7f460019d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 0x7f460019d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 -- 172.21.15.7:0/2588656233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600104db0 msgr2=0x7f4600195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600104db0 0x7f4600195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 -- 172.21.15.7:0/2588656233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460019d6c0 con 0x7f4600102480 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.034+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 0x7f4600195aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f45f0035040 tx=0x7f45f0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.035+0000 7f45f6a00640 1 -- 172.21.15.7:0/2588656233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0037040 con 0x7f4600102480 2024-06-13T06:59:22.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.035+0000 7f45f6a00640 1 -- 172.21.15.7:0/2588656233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0030850 con 0x7f4600102480 2024-06-13T06:59:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.035+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460019d950 con 0x7f4600102480 2024-06-13T06:59:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.035+0000 7f45f6a00640 1 -- 172.21.15.7:0/2588656233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0004920 con 0x7f4600102480 2024-06-13T06:59:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.036+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460019de60 con 0x7f4600102480 2024-06-13T06:59:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.044+0000 7f4607e19740 1 Processor -- start 2024-06-13T06:59:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 -- start start 2024-06-13T06:59:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d38ff5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff55f0 0x555d38ffa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d38ffcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555d38f58420 con 0x555d38ff4c00 2024-06-13T06:59:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555d38effc50 con 0x555d38ffa760 2024-06-13T06:59:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.045+0000 7f4607e19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d38f543e0 con 0x555d38ff55f0 2024-06-13T06:59:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.047+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d38ffcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.048+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d38ffcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34476/0 (socket says 172.21.15.7:34476) 2024-06-13T06:59:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.048+0000 7f45fd400640 1 -- 172.21.15.7:0/818916628 learned_addr learned my addr 172.21.15.7:0/818916628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.048+0000 7f45f6a00640 1 -- 172.21.15.7:0/2588656233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f45f0004050 con 0x7f4600102480 2024-06-13T06:59:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.049+0000 7f45fe800640 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d38ff5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fde00640 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff55f0 0x555d38ffa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fe800640 1 -- 172.21.15.7:0/818916628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff55f0 msgr2=0x555d38ffa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fe800640 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff55f0 0x555d38ffa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fe800640 1 -- 172.21.15.7:0/818916628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 msgr2=0x555d38ffcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fe800640 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d38ffcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.050+0000 7f45fe800640 1 -- 172.21.15.7:0/818916628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d38c6c210 con 0x555d38ff4c00 2024-06-13T06:59:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.051+0000 7f45fe800640 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d38ff5020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005c40 tx=0x7f45f80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f65d9b031d8ad91 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.052+0000 7f45f6a00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 0x7f45e40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.053+0000 7f45eb400640 1 -- 172.21.15.7:0/818916628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f801e070 con 0x555d38ff4c00 2024-06-13T06:59:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.053+0000 7f45f6a00640 1 -- 172.21.15.7:0/2588656233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f45f00ef1d0 con 0x7f4600102480 2024-06-13T06:59:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.053+0000 7f45eb400640 1 -- 172.21.15.7:0/818916628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f800fa80 con 0x555d38ff4c00 2024-06-13T06:59:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.053+0000 7f45eb400640 1 -- 172.21.15.7:0/818916628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8019bd0 con 0x555d38ff4c00 2024-06-13T06:59:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.053+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 0x7f45e40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.054+0000 7f4607e19740 1 -- 172.21.15.7:0/818916628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 msgr2=0x555d38ff5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.054+0000 7f4607e19740 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d38ff5020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005c40 tx=0x7f45f80118d0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f45fde00640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 0x7f45e40a51f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45f00b7af0 tx=0x7f45f0005330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 -- 172.21.15.7:0/818916628 shutdown_connections 2024-06-13T06:59:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d38ffcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff55f0 0x555d38ffa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 --2- 172.21.15.7:0/818916628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d38ff5020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 -- 172.21.15.7:0/818916628 >> 172.21.15.7:0/818916628 conn(0x555d38fbcfb0 msgr2=0x555d38ff0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 -- 172.21.15.7:0/818916628 shutdown_connections 2024-06-13T06:59:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.055+0000 7f4607e19740 1 -- 172.21.15.7:0/818916628 wait complete. 2024-06-13T06:59:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.056+0000 7f4607e19740 1 -- start start 2024-06-13T06:59:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 Processor -- start 2024-06-13T06:59:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d3900ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 0x555d39017420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555d3900e720 con 0x555d38ff4c00 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555d38f130b0 con 0x555d38ff55f0 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d3900e8a0 con 0x555d38ffa760 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.058+0000 7f4607e19740 10 client.0 _ll_register_callbacks cb 0x555d38efef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 0x555d39017420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 0x555d39017420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54336/0 (socket says 172.21.15.7:54336) 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34488/0 (socket says 172.21.15.7:34488) 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fe800640 1 -- 172.21.15.7:0/3710725001 learned_addr learned my addr 172.21.15.7:0/3710725001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.059+0000 7f45fe800640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d3900ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 -- 172.21.15.7:0/3710725001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 msgr2=0x555d39017420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 0x555d39017420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 -- 172.21.15.7:0/3710725001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 msgr2=0x555d3900ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d3900ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 -- 172.21.15.7:0/3710725001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d38d38620 con 0x555d38ff55f0 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45fd400640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x555d38ff1530 tx=0x7f45ec0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fdad953927aebbd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.060+0000 7f45dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec012860 con 0x555d38ff55f0 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f45ec012a00 con 0x555d38ff55f0 2024-06-13T06:59:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f800fa80 con 0x555d38ff55f0 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45ec025050 con 0x555d38ff55f0 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f4607e19740 10 client.25829 fetch_fsmap learned FSMap version 31 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.061+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555d3901a800 con 0x555d38ff55f0 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f800fa80 con 0x555d38ff55f0 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f45ec01a020 con 0x555d38ff55f0 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 10 client.25829 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 20 client.25829 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d3901bee0 0x555d3901e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 4 client.25829 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.062+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d38f65b50 con 0x555d3901bee0 2024-06-13T06:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.063+0000 7f45fe800640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d3901bee0 0x555d3901e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.064+0000 7f45fe800640 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d3901bee0 0x555d3901e2d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2331d502ac68819f server_cookie=fcef53ab9503094e in_seq=0 out_seq=0 2024-06-13T06:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.064+0000 7f45dfe00640 10 client.25829 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.067+0000 7f45dfe00640 1 -- 172.21.15.7:0/3710725001 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d38f65b50 con 0x555d3901bee0 2024-06-13T06:59:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.067+0000 7f45dfe00640 10 client.25829 handle_command_reply: tid=0 2024-06-13T06:59:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.067+0000 7f4607e19740 1 client.25829 shutdown 2024-06-13T06:59:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 msgr2=0x555d39010450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x555d38ff1530 tx=0x7f45ec0039c0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.894+0000 7fd5d3400700 1 -- 172.21.15.7:0/4221351345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0017400 con 0x7fd5cc098b30 2024-06-13T06:59:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.897+0000 7fd5d9e00700 1 -- 172.21.15.7:0/4221351345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc098b30 msgr2=0x7fd5cc09cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.897+0000 7fd5d9e00700 1 --2- 172.21.15.7:0/4221351345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc098b30 0x7fd5cc09cb80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009920 tx=0x7fd5c0009c30 comp rx=0 tx=0).stop 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.898+0000 7fd5d9e00700 1 -- 172.21.15.7:0/4221351345 shutdown_connections 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.898+0000 7fd5d9e00700 1 --2- 172.21.15.7:0/4221351345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5cc09d260 0x7fd5cc0a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.898+0000 7fd5d9e00700 1 --2- 172.21.15.7:0/4221351345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.898+0000 7fd5d9e00700 1 --2- 172.21.15.7:0/4221351345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc098b30 0x7fd5cc09cb80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.898+0000 7fd5d9e00700 1 -- 172.21.15.7:0/4221351345 >> 172.21.15.7:0/4221351345 conn(0x7fd5cc093a50 msgr2=0x7fd5cc095e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.899+0000 7fd5d9e00700 1 -- 172.21.15.7:0/4221351345 shutdown_connections 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.899+0000 7fd5d9e00700 1 -- 172.21.15.7:0/4221351345 wait complete. 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.900+0000 7fd5d9e00700 1 Processor -- start 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.902+0000 7fd5d9e00700 1 -- start start 2024-06-13T06:59:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.904+0000 7fd5d9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.904+0000 7fd5d9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5cc098b30 0x7fd5cc1349a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d8a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d8a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54310/0 (socket says 172.21.15.7:54310) 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d8a00700 1 -- 172.21.15.7:0/246403520 learned_addr learned my addr 172.21.15.7:0/246403520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d9e00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc09d260 0x7fd5cc12e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d9e00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0a8320 con 0x7fd5cc09d260 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d9e00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0a81a0 con 0x7fd5cc098b30 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.905+0000 7fd5d9e00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc0a84a0 con 0x7fd5cc098180 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 -- 172.21.15.7:0/246403520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5cc098b30 msgr2=0x7fd5cc1349a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5cc098b30 0x7fd5cc1349a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 -- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc09d260 msgr2=0x7fd5cc12e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d9400700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc09d260 0x7fd5cc12e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc09d260 0x7fd5cc12e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc12ed80 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.907+0000 7fd5d8a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.908+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cbd0 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.908+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c800cd70 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.908+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8010430 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.908+0000 7fd5d9e00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc12f070 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.908+0000 7fd5d9e00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc14c340 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.910+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc006250 con 0x7fd5cc098180 2024-06-13T06:59:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.914+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd5c8004da0 con 0x7fd5cc098180 2024-06-13T06:59:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.917+0000 7fd5d1600700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 0x7fd5bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.917+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd5c80cf7b0 con 0x7fd5cc098180 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 shutdown_connections 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d3901bee0 0x555d3901e2d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ffa760 0x555d39017420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39010450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 --2- 172.21.15.7:0/3710725001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff4c00 0x555d3900ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f45dfe00640 0 client.25829 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.068+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 >> 172.21.15.7:0/3710725001 conn(0x555d38fbcfb0 msgr2=0x555d38fef840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.918+0000 7fd5d3e00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 0x7fd5bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.918+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5c80cfc30 con 0x7fd5cc098180 2024-06-13T06:59:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:21.919+0000 7fd5d3e00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 0x7fd5bc0a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009230 tx=0x7fd5c00094c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.069+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 shutdown_connections 2024-06-13T06:59:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.069+0000 7f4607e19740 1 -- 172.21.15.7:0/3710725001 wait complete. 2024-06-13T06:59:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.069+0000 7f4607e19740 20 client.25829 trim_cache size 0 max 16384 2024-06-13T06:59:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.105+0000 7f4607e19740 1 Processor -- start 2024-06-13T06:59:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.105+0000 7f4607e19740 1 -- start start 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff55f0 0x555d390173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d3900ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b33e0 con 0x555d38ff55f0 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b0cb0 con 0x555d38ffa760 2024-06-13T06:59:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.106+0000 7f4607e19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4600071d40 con 0x555d38ff4c00 2024-06-13T06:59:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54348/0 (socket says 172.21.15.7:54348) 2024-06-13T06:59:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 -- 172.21.15.7:0/3631296133 learned_addr learned my addr 172.21.15.7:0/3631296133 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d3900ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 -- 172.21.15.7:0/3631296133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 msgr2=0x555d3900ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d3900ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 -- 172.21.15.7:0/3631296133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff55f0 msgr2=0x555d390173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff55f0 0x555d390173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.107+0000 7f45fe800640 1 -- 172.21.15.7:0/3631296133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3901e640 con 0x555d38ff4c00 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.108+0000 7f45fe800640 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45f8010010 tx=0x7f45f8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf46dfa32f0c254 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.108+0000 7f45eaa00640 1 -- 172.21.15.7:0/3631296133 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f801e040 con 0x555d38ff4c00 2024-06-13T06:59:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.108+0000 7f45eaa00640 1 -- 172.21.15.7:0/3631296133 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8003970 con 0x555d38ff4c00 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.108+0000 7f45eaa00640 1 -- 172.21.15.7:0/3631296133 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f801e040 con 0x555d38ff4c00 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 -- 172.21.15.7:0/3631296133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 msgr2=0x555d38ff3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45f8010010 tx=0x7f45f8000f40 comp rx=0 tx=0).stop 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 -- 172.21.15.7:0/3631296133 shutdown_connections 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ffa760 0x555d3900ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ff55f0 0x555d390173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 --2- 172.21.15.7:0/3631296133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d38ff3c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.109+0000 7f4607e19740 1 -- 172.21.15.7:0/3631296133 >> 172.21.15.7:0/3631296133 conn(0x555d38fbcfb0 msgr2=0x555d38ff3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.110+0000 7f4607e19740 1 -- 172.21.15.7:0/3631296133 shutdown_connections 2024-06-13T06:59:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.110+0000 7f4607e19740 1 -- 172.21.15.7:0/3631296133 wait complete. 2024-06-13T06:59:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.110+0000 7f4607e19740 1 -- start start 2024-06-13T06:59:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 Processor -- start 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39014940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ffa760 0x555d39084ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46000723b0 con 0x555d38ffa760 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4600073820 con 0x555d38ff55f0 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.111+0000 7f4607e19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4600072a80 con 0x555d38ff4c00 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f4607e19740 10 client.0 _ll_register_callbacks cb 0x7f45ec004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39014940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39014940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34512/0 (socket says 172.21.15.7:34512) 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fd400640 1 -- 172.21.15.7:0/2683033183 learned_addr learned my addr 172.21.15.7:0/2683033183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fe800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54350/0 (socket says 172.21.15.7:54350) 2024-06-13T06:59:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.112+0000 7f45fe800640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ffa760 0x555d39084ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 -- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 msgr2=0x555d39014940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39014940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 -- 172.21.15.7:0/2683033183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ffa760 msgr2=0x555d39084ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ffa760 0x555d39084ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 -- 172.21.15.7:0/2683033183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d39085190 con 0x555d38ff4c00 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45fe800640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fb0 tx=0x7f45f8019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe988b93a7be84a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.113+0000 7f45dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800fa90 con 0x555d38ff4c00 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f45f8004a00 con 0x555d38ff4c00 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f8003970 con 0x555d38ff4c00 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f45f8004c00 con 0x555d38ff4c00 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f4607e19740 10 client.25819 fetch_fsmap learned FSMap version 31 2024-06-13T06:59:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.114+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555d390864a0 con 0x555d38ff4c00 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8003970 con 0x555d38ff4c00 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f45f8024040 con 0x555d38ff4c00 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 10 client.25819 fetch_fsmap finished waiting for FSMap version 31 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 20 client.25819 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d39086b80 0x555d39088fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 4 client.25819 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:59:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.115+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d38f5fea0 con 0x555d39086b80 2024-06-13T06:59:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.117+0000 7f45fd400640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d39086b80 0x555d39088fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.118+0000 7f45fd400640 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d39086b80 0x555d39088fa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce7a5e344dc3db server_cookie=161a25f344faeb36 in_seq=0 out_seq=0 2024-06-13T06:59:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.118+0000 7f45dfe00640 10 client.25819 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.134+0000 7f45dfe00640 1 -- 172.21.15.7:0/2683033183 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d38f5fea0 con 0x555d39086b80 2024-06-13T06:59:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.134+0000 7f45dfe00640 10 client.25819 handle_command_reply: tid=0 2024-06-13T06:59:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.134+0000 7f4607e19740 1 client.25819 shutdown 2024-06-13T06:59:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 msgr2=0x555d39014400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fb0 tx=0x7f45f8019410 comp rx=0 tx=0).stop 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 shutdown_connections 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555d39086b80 0x555d39088fa0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555d38ffa760 0x555d39084ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555d38ff55f0 0x555d39014940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 --2- 172.21.15.7:0/2683033183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d38ff4c00 0x555d39014400 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 >> 172.21.15.7:0/2683033183 conn(0x555d38fbcfb0 msgr2=0x555d38ff3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f45dfe00640 0 client.25819 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 shutdown_connections 2024-06-13T06:59:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 1 -- 172.21.15.7:0/2683033183 wait complete. 2024-06-13T06:59:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.135+0000 7f4607e19740 20 client.25819 trim_cache size 0 max 16384 2024-06-13T06:59:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:59:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:59:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 msgr2=0x7f45e40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 0x7f45e40a51f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45f00b7af0 tx=0x7f45f0005330 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 msgr2=0x7f4600195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 0x7f4600195aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f45f0035040 tx=0x7f45f0033d90 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 shutdown_connections 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f45e40a2d30 0x7f45e40a51f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46001076e0 0x7f460019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600104db0 0x7f4600195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 --2- 172.21.15.7:0/2588656233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4600102480 0x7f4600195aa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 >> 172.21.15.7:0/2588656233 conn(0x7f46000fbe90 msgr2=0x7f4600101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 shutdown_connections 2024-06-13T06:59:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:22.137+0000 7f4605000640 1 -- 172.21.15.7:0/2588656233 wait complete. 2024-06-13T06:59:22.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.387+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5cc12fb50 con 0x7fd5cc098180 2024-06-13T06:59:22.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.388+0000 7fd5d1600700 1 -- 172.21.15.7:0/246403520 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11869 (secure 0 0 0) 0x7fd5c809c480 con 0x7fd5cc098180 2024-06-13T06:59:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T06:59:22.388 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T06:59:12.638537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T06:59:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 msgr2=0x7fd5bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 0x7fd5bc0a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009230 tx=0x7fd5c00094c0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 msgr2=0x7fd5cc134460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9c0 tx=0x7fd5c800ed80 comp rx=0 tx=0).stop 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 shutdown_connections 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5cc09d260 0x7fd5cc12e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5cc098180 0x7fd5cc134460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd5bc0a2da0 0x7fd5bc0a5260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 --2- 172.21.15.7:0/246403520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5cc098b30 0x7fd5cc1349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 >> 172.21.15.7:0/246403520 conn(0x7fd5cc093a50 msgr2=0x7fd5cc095d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.391+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 shutdown_connections 2024-06-13T06:59:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:22.392+0000 7fd5c6a00700 1 -- 172.21.15.7:0/246403520 wait complete. 2024-06-13T06:59:22.393 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 31 2024-06-13T06:59:22.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:22 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/246403520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:22.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:22 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/246403520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:22.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 200.0 s, 713.2 tps, lat 140.382 ms stddev 167.282 2024-06-13T06:59:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:22 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/246403520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T06:59:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:23 smithi175 ceph-mon[36886]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 822 op/s 2024-06-13T06:59:23.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:23 smithi007 ceph-mon[32077]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 822 op/s 2024-06-13T06:59:24.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T06:59:24.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:23 smithi189 ceph-mon[36616]: pgmap v609: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 822 op/s 2024-06-13T06:59:24.711 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:24 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:24.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:24 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:25.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:24 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T06:59:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:25 smithi007 ceph-mon[32077]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:25 smithi189 ceph-mon[36616]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:25 smithi175 ceph-mon[36886]: pgmap v610: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 889 op/s 2024-06-13T06:59:26.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:26 smithi007 ceph-mon[32077]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-06-13T06:59:27.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:26 smithi175 ceph-mon[36886]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-06-13T06:59:27.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:26 smithi189 ceph-mon[36616]: pgmap v611: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 MiB/s wr, 836 op/s 2024-06-13T06:59:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.177+0000 7f586e800700 1 -- 172.21.15.7:0/1436681576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109490 msgr2=0x7f5868109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.177+0000 7f586e800700 1 --2- 172.21.15.7:0/1436681576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109490 0x7f5868109870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5858009990 tx=0x7f5858009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 -- 172.21.15.7:0/1436681576 shutdown_connections 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 --2- 172.21.15.7:0/1436681576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 --2- 172.21.15.7:0/1436681576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109490 0x7f5868109870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 --2- 172.21.15.7:0/1436681576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109db0 0x7f58681150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 -- 172.21.15.7:0/1436681576 >> 172.21.15.7:0/1436681576 conn(0x7f586806d600 msgr2=0x7f586806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 -- 172.21.15.7:0/1436681576 shutdown_connections 2024-06-13T06:59:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.180+0000 7f586e800700 1 -- 172.21.15.7:0/1436681576 wait complete. 2024-06-13T06:59:27.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.181+0000 7f586e800700 1 Processor -- start 2024-06-13T06:59:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.182+0000 7f586e800700 1 -- start start 2024-06-13T06:59:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.182+0000 7f586e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109490 0x7f58681a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.182+0000 7f586e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109db0 0x7f58681a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.182+0000 7f586e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f586e800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f586811b160 con 0x7f5868109490 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f586e800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f586811afe0 con 0x7f58681155e0 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f586e800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f586811b2e0 con 0x7f5868109db0 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f5867e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f5867e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34524/0 (socket says 172.21.15.7:34524) 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f5867e00700 1 -- 172.21.15.7:0/3894439208 learned_addr learned my addr 172.21.15.7:0/3894439208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.183+0000 7f5867400700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109490 0x7f58681a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5866a00700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109db0 0x7f58681a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 -- 172.21.15.7:0/3894439208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109db0 msgr2=0x7f58681a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109db0 0x7f58681a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 -- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109490 msgr2=0x7f58681a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109490 0x7f58681a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 -- 172.21.15.7:0/3894439208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681a6610 con 0x7f58681155e0 2024-06-13T06:59:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.184+0000 7f5867e00700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f586000b970 tx=0x7f586000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.185+0000 7f585fe00700 1 -- 172.21.15.7:0/3894439208 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860019070 con 0x7f58681155e0 2024-06-13T06:59:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.185+0000 7f585fe00700 1 -- 172.21.15.7:0/3894439208 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5860014440 con 0x7f58681155e0 2024-06-13T06:59:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.185+0000 7f585fe00700 1 -- 172.21.15.7:0/3894439208 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860007590 con 0x7f58681155e0 2024-06-13T06:59:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.186+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681a6900 con 0x7f58681155e0 2024-06-13T06:59:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.186+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586810ae30 con 0x7f58681155e0 2024-06-13T06:59:27.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.194+0000 7f585fe00700 1 -- 172.21.15.7:0/3894439208 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f586001c070 con 0x7f58681155e0 2024-06-13T06:59:27.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.196+0000 7f585fe00700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 0x7f584c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.197+0000 7f585fe00700 1 -- 172.21.15.7:0/3894439208 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f58600ce2e0 con 0x7f58681155e0 2024-06-13T06:59:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.197+0000 7f587927cb80 1 Processor -- start 2024-06-13T06:59:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.197+0000 7f5867400700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 0x7f584c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.198+0000 7f587927cb80 1 -- start start 2024-06-13T06:59:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.198+0000 7f587927cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f587927cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f587927cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f587927cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5639287bba10 con 0x563928828e40 2024-06-13T06:59:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f5867400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f587927cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5639287feb00 con 0x5639288b4dc0 2024-06-13T06:59:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.199+0000 7f587927cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639287b98f0 con 0x5639288bfb10 2024-06-13T06:59:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.200+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.200+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34526/0 (socket says 172.21.15.7:34526) 2024-06-13T06:59:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.200+0000 7f5866a00700 1 -- 172.21.15.7:0/681645815 learned_addr learned my addr 172.21.15.7:0/681645815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.200+0000 7f5867e00700 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5867400700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 0x7f584c0a5230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858003800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 -- 172.21.15.7:0/681645815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 msgr2=0x5639288c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 -- 172.21.15.7:0/681645815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 msgr2=0x5639288b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 -- 172.21.15.7:0/681645815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639287a6f90 con 0x5639288b4dc0 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f5866a00700 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f585400a950 tx=0x7f585400ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e78a2c7ca36ee7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.201+0000 7f585cc00700 1 -- 172.21.15.7:0/681645815 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854015070 con 0x5639288b4dc0 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.202+0000 7f585cc00700 1 -- 172.21.15.7:0/681645815 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585400cd80 con 0x5639288b4dc0 2024-06-13T06:59:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.202+0000 7f585cc00700 1 -- 172.21.15.7:0/681645815 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854010350 con 0x5639288b4dc0 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 -- 172.21.15.7:0/681645815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 msgr2=0x5639288b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f585400a950 tx=0x7f585400ac60 comp rx=0 tx=0).stop 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 -- 172.21.15.7:0/681645815 shutdown_connections 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 --2- 172.21.15.7:0/681645815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288b5240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.203+0000 7f587927cb80 1 -- 172.21.15.7:0/681645815 >> 172.21.15.7:0/681645815 conn(0x5639287d9170 msgr2=0x5639288a84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.204+0000 7f587927cb80 1 -- 172.21.15.7:0/681645815 shutdown_connections 2024-06-13T06:59:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.204+0000 7f587927cb80 1 -- 172.21.15.7:0/681645815 wait complete. 2024-06-13T06:59:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.205+0000 7f587927cb80 1 -- start start 2024-06-13T06:59:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.205+0000 7f587927cb80 1 Processor -- start 2024-06-13T06:59:27.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.206+0000 7f587927cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288bf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.206+0000 7f587927cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f587927cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5867400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5867400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288bf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47176/0 (socket says 172.21.15.7:47176) 2024-06-13T06:59:27.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5867e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5867e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54368/0 (socket says 172.21.15.7:54368) 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f587927cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56392879af60 con 0x563928828e40 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f587927cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5639287faac0 con 0x5639288b4dc0 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f587927cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639288c2820 con 0x5639288bfb10 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f587927cb80 10 client.0 _ll_register_callbacks cb 0x563928798720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34538/0 (socket says 172.21.15.7:34538) 2024-06-13T06:59:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.207+0000 7f5866a00700 1 -- 172.21.15.7:0/2431679671 learned_addr learned my addr 172.21.15.7:0/2431679671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:27.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5866a00700 1 -- 172.21.15.7:0/2431679671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 msgr2=0x5639288bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5866a00700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5866a00700 1 -- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 msgr2=0x5639288bf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5866a00700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288bf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5866a00700 1 -- 172.21.15.7:0/2431679671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392874d8d0 con 0x5639288b4dc0 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.208+0000 7f5867e00700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.209+0000 7f5851600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:27.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.209+0000 7f5866a00700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f585400c8d0 tx=0x7f585400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afd205614b85c50 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.210+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854015040 con 0x5639288b4dc0 2024-06-13T06:59:27.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.210+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5854026650 con 0x5639288b4dc0 2024-06-13T06:59:27.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.210+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585400cd80 con 0x5639288b4dc0 2024-06-13T06:59:27.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.213+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f585402f050 con 0x5639288b4dc0 2024-06-13T06:59:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.213+0000 7f587927cb80 10 client.25844 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.213+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639288cced0 con 0x5639288b4dc0 2024-06-13T06:59:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.213+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585400cd80 con 0x5639288b4dc0 2024-06-13T06:59:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f585401a5e0 con 0x5639288b4dc0 2024-06-13T06:59:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 10 client.25844 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:59:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 20 client.25844 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288cf3a0 0x5639288d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 4 client.25844 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.214+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56392880c000 con 0x5639288cf3a0 2024-06-13T06:59:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.215+0000 7f5867400700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288cf3a0 0x5639288d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.219+0000 7f5851600700 10 client.25844 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:59:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.219+0000 7f5867400700 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288cf3a0 0x5639288d1790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b778fe47731ae08e server_cookie=9eafe8c3e2f802f9 in_seq=0 out_seq=0 2024-06-13T06:59:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.221+0000 7f5851600700 1 -- 172.21.15.7:0/2431679671 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56392880c000 con 0x5639288cf3a0 2024-06-13T06:59:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.221+0000 7f5851600700 10 client.25844 handle_command_reply: tid=0 2024-06-13T06:59:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.221+0000 7f587927cb80 1 client.25844 shutdown 2024-06-13T06:59:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 msgr2=0x5639288bc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f585400c8d0 tx=0x7f585400c900 comp rx=0 tx=0).stop 2024-06-13T06:59:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 shutdown_connections 2024-06-13T06:59:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639288bfb10 0x5639288bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563928828e40 0x5639288bf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288cf3a0 0x5639288d1790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 --2- 172.21.15.7:0/2431679671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288b4dc0 0x5639288bc460 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 >> 172.21.15.7:0/2431679671 conn(0x5639287d9170 msgr2=0x5639288a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f5851600700 0 client.25844 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 shutdown_connections 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 1 -- 172.21.15.7:0/2431679671 wait complete. 2024-06-13T06:59:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.222+0000 7f587927cb80 20 client.25844 trim_cache size 0 max 16384 2024-06-13T06:59:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 Processor -- start 2024-06-13T06:59:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 -- start start 2024-06-13T06:59:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b7160 con 0x5639288c1440 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b49f0 con 0x5639288bfb10 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.249+0000 7f587927cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b2840 con 0x563928828e40 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5867400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34554/0 (socket says 172.21.15.7:34554) 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 -- 172.21.15.7:0/3656290385 learned_addr learned my addr 172.21.15.7:0/3656290385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5867e00700 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 -- 172.21.15.7:0/3656290385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 msgr2=0x5639288c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 -- 172.21.15.7:0/3656290385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 msgr2=0x5639288bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5866a00700 1 -- 172.21.15.7:0/3656290385 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639288b7880 con 0x5639288bfb10 2024-06-13T06:59:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.250+0000 7f5867400700 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.251+0000 7f5866a00700 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f585401aad0 tx=0x7f585400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89a497db03730e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.251+0000 7f5853e00700 1 -- 172.21.15.7:0/3656290385 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585400c1b0 con 0x5639288bfb10 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.251+0000 7f5853e00700 1 -- 172.21.15.7:0/3656290385 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585401ab50 con 0x5639288bfb10 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.251+0000 7f5853e00700 1 -- 172.21.15.7:0/3656290385 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58540185e0 con 0x5639288bfb10 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 -- 172.21.15.7:0/3656290385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 msgr2=0x5639288c0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f585401aad0 tx=0x7f585400ccb0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 -- 172.21.15.7:0/3656290385 shutdown_connections 2024-06-13T06:59:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 --2- 172.21.15.7:0/3656290385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288c0f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 -- 172.21.15.7:0/3656290385 >> 172.21.15.7:0/3656290385 conn(0x5639287d9170 msgr2=0x5639288c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 -- 172.21.15.7:0/3656290385 shutdown_connections 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.252+0000 7f587927cb80 1 -- 172.21.15.7:0/3656290385 wait complete. 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.253+0000 7f587927cb80 1 -- start start 2024-06-13T06:59:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.253+0000 7f587927cb80 1 Processor -- start 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b2eb0 con 0x5639288c1440 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b4320 con 0x5639288bfb10 2024-06-13T06:59:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b3580 con 0x563928828e40 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f587927cb80 10 client.0 _ll_register_callbacks cb 0x563928798720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f5866a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34568/0 (socket says 172.21.15.7:34568) 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.254+0000 7f5866a00700 1 -- 172.21.15.7:0/1292201915 learned_addr learned my addr 172.21.15.7:0/1292201915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 -- 172.21.15.7:0/1292201915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 msgr2=0x5639288b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 -- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 msgr2=0x5639288b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 -- 172.21.15.7:0/1292201915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639288b9170 con 0x5639288bfb10 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5866a00700 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f585401ab20 tx=0x7f5854006e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f7568eb2cba5a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5851600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.255+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854015030 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f585401a450 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585401ab50 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5854043050 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 10 client.25850 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639288c7880 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585401ab50 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5854034c60 con 0x5639288bfb10 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 10 client.25850 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 10 client.25850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 20 client.25850 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.256+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288c7f80 0x5639288c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.257+0000 7f587927cb80 4 client.25850 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:59:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.257+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563928806580 con 0x5639288c7f80 2024-06-13T06:59:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.259+0000 7f5867400700 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288c7f80 0x5639288c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.262+0000 7f5867400700 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288c7f80 0x5639288c83b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f0440dfe46e24b0 server_cookie=a2067f75291c6a35 in_seq=0 out_seq=0 2024-06-13T06:59:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.263+0000 7f5851600700 10 client.25850 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T06:59:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.263+0000 7f5851600700 1 -- 172.21.15.7:0/1292201915 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563928806580 con 0x5639288c7f80 2024-06-13T06:59:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.263+0000 7f5851600700 10 client.25850 handle_command_reply: tid=0 2024-06-13T06:59:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.263+0000 7f587927cb80 1 client.25850 shutdown 2024-06-13T06:59:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 msgr2=0x5639288b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f585401ab20 tx=0x7f5854006e40 comp rx=0 tx=0).stop 2024-06-13T06:59:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 shutdown_connections 2024-06-13T06:59:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639288c1440 0x5639288b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563928828e40 0x5639288b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5639288c7f80 0x5639288c83b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 --2- 172.21.15.7:0/1292201915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639288bfb10 0x5639288b86f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f5851600700 0 client.25850 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 >> 172.21.15.7:0/1292201915 conn(0x5639287d9170 msgr2=0x5639288b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 shutdown_connections 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.264+0000 7f587927cb80 1 -- 172.21.15.7:0/1292201915 wait complete. 2024-06-13T06:59:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.265+0000 7f587927cb80 20 client.25850 trim_cache size 0 max 16384 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 msgr2=0x7f584c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 0x7f584c0a5230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858003800 comp rx=0 tx=0).stop 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 msgr2=0x7f58681a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f586000b970 tx=0x7f586000bd30 comp rx=0 tx=0).stop 2024-06-13T06:59:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 shutdown_connections 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f584c0a2d70 0x7f584c0a5230 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58681155e0 0x7f58681a5ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868109490 0x7f58681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 --2- 172.21.15.7:0/3894439208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5868109db0 0x7f58681a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.266+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 >> 172.21.15.7:0/3894439208 conn(0x7f586806d600 msgr2=0x7f58680711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.267+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 shutdown_connections 2024-06-13T06:59:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:27.267+0000 7f586e800700 1 -- 172.21.15.7:0/3894439208 wait complete. 2024-06-13T06:59:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 205.0 s, 711.8 tps, lat 141.592 ms stddev 156.866 2024-06-13T06:59:28.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T06:59:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:29.228 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:28 smithi007 ceph-mon[32077]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-06-13T06:59:29.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:28 smithi189 ceph-mon[36616]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-06-13T06:59:29.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:28 smithi175 ceph-mon[36886]: pgmap v612: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-06-13T06:59:31.041 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:30 smithi007 ceph-mon[32077]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-06-13T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.281+0000 7f54dfe00700 1 -- 172.21.15.7:0/1751723779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e010a670 msgr2=0x7f54e010aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.281+0000 7f54dfe00700 1 --2- 172.21.15.7:0/1751723779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e010a670 0x7f54e010aa50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b0d0 tx=0x7f54d800b3e0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.282+0000 7f54dfe00700 1 -- 172.21.15.7:0/1751723779 shutdown_connections 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.282+0000 7f54dfe00700 1 --2- 172.21.15.7:0/1751723779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e0104f80 0x7f54e0111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.282+0000 7f54dfe00700 1 --2- 172.21.15.7:0/1751723779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e010a670 0x7f54e010aa50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.282+0000 7f54dfe00700 1 --2- 172.21.15.7:0/1751723779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e01045c0 0x7f54e0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.282+0000 7f54dfe00700 1 -- 172.21.15.7:0/1751723779 >> 172.21.15.7:0/1751723779 conn(0x7f54e0100070 msgr2=0x7f54e0102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.285+0000 7f54dfe00700 1 -- 172.21.15.7:0/1751723779 shutdown_connections 2024-06-13T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.285+0000 7f54dfe00700 1 -- 172.21.15.7:0/1751723779 wait complete. 2024-06-13T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.286+0000 7f54dfe00700 1 Processor -- start 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.287+0000 7f54dfe00700 1 -- start start 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 0x7f54e0076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e010a670 0x7f54e0077420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0114a80 con 0x7f54e010a670 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0114900 con 0x7f54e01045c0 2024-06-13T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dfe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0114c00 con 0x7f54e0104f80 2024-06-13T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.288+0000 7f54dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.289+0000 7f54dea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57578/0 (socket says 172.21.15.7:57578) 2024-06-13T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.290+0000 7f54de000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 0x7f54e0076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.290+0000 7f54de000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 0x7f54e0076ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49114/0 (socket says 172.21.15.7:49114) 2024-06-13T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.290+0000 7f54de000700 1 -- 172.21.15.7:0/2397316819 learned_addr learned my addr 172.21.15.7:0/2397316819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54df400700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e010a670 0x7f54e0077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54dea00700 1 -- 172.21.15.7:0/2397316819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 msgr2=0x7f54e0076ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54dea00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 0x7f54e0076ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54dea00700 1 -- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e010a670 msgr2=0x7f54e0077420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54dea00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e010a670 0x7f54e0077420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.292+0000 7f54dea00700 1 -- 172.21.15.7:0/2397316819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e0072d90 con 0x7f54e01045c0 2024-06-13T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.296+0000 7f54dea00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b820 tx=0x7f54d8007ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.298+0000 7f54d7400700 1 -- 172.21.15.7:0/2397316819 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8021070 con 0x7f54e01045c0 2024-06-13T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.298+0000 7f54d7400700 1 -- 172.21.15.7:0/2397316819 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d801db30 con 0x7f54e01045c0 2024-06-13T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.298+0000 7f54d7400700 1 -- 172.21.15.7:0/2397316819 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d801bb80 con 0x7f54e01045c0 2024-06-13T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.298+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e0073020 con 0x7f54e01045c0 2024-06-13T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.299+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e00735b0 con 0x7f54e01045c0 2024-06-13T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.306+0000 7f54f0195b80 1 Processor -- start 2024-06-13T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 -- start start 2024-06-13T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5631d61c6a10 con 0x5631d62b6d60 2024-06-13T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5631d6209b00 con 0x5631d62b7680 2024-06-13T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.310+0000 7f54f0195b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631d61c48f0 con 0x5631d62c2530 2024-06-13T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.311+0000 7f54de000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.311+0000 7f54de000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49130/0 (socket says 172.21.15.7:49130) 2024-06-13T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.311+0000 7f54de000700 1 -- 172.21.15.7:0/3904935425 learned_addr learned my addr 172.21.15.7:0/3904935425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.311+0000 7f54df400700 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.311+0000 7f54d7400700 1 -- 172.21.15.7:0/2397316819 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f54d8024070 con 0x7f54e01045c0 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 -- 172.21.15.7:0/3904935425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 msgr2=0x5631d62c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 -- 172.21.15.7:0/3904935425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 msgr2=0x5631d62c1ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 -- 172.21.15.7:0/3904935425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d61b1f90 con 0x5631d62b6d60 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.312+0000 7f54df400700 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62b7140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00a820 tx=0x7f54cc00ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e8193a243a675 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.313+0000 7f54d7400700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 0x7f54c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.313+0000 7f54d7400700 1 -- 172.21.15.7:0/2397316819 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54d800f020 con 0x7f54e01045c0 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.314+0000 7f54cbe00700 1 -- 172.21.15.7:0/3904935425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc015070 con 0x5631d62b6d60 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.314+0000 7f54cbe00700 1 -- 172.21.15.7:0/3904935425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc011ce0 con 0x5631d62b6d60 2024-06-13T06:59:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.314+0000 7f54dea00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 0x7f54c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.315+0000 7f54f0195b80 1 -- 172.21.15.7:0/3904935425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 msgr2=0x5631d62b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54cbe00700 1 -- 172.21.15.7:0/3904935425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc018360 con 0x5631d62b6d60 2024-06-13T06:59:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62b7140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00a820 tx=0x7f54cc00ab30 comp rx=0 tx=0).stop 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 -- 172.21.15.7:0/3904935425 shutdown_connections 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62b7140 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3904935425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.316+0000 7f54f0195b80 1 -- 172.21.15.7:0/3904935425 >> 172.21.15.7:0/3904935425 conn(0x5631d61e4170 msgr2=0x5631d62b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.317+0000 7f54f0195b80 1 -- 172.21.15.7:0/3904935425 shutdown_connections 2024-06-13T06:59:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.317+0000 7f54dea00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 0x7f54c40a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54d809c6c0 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.317+0000 7f54f0195b80 1 -- 172.21.15.7:0/3904935425 wait complete. 2024-06-13T06:59:31.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.319+0000 7f54f0195b80 1 -- start start 2024-06-13T06:59:31.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.319+0000 7f54f0195b80 1 Processor -- start 2024-06-13T06:59:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.325+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.325+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.325+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54df400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55392/0 (socket says 172.21.15.7:55392) 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54f0195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5631d61a5f60 con 0x5631d62b6d60 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54f0195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5631d6205ac0 con 0x5631d62b7680 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54f0195b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631d62c5020 con 0x5631d62c2530 2024-06-13T06:59:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54f0195b80 10 client.0 _ll_register_callbacks cb 0x5631d619f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:31.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.326+0000 7f54df400700 1 -- 172.21.15.7:0/1484143232 learned_addr learned my addr 172.21.15.7:0/1484143232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:31.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.327+0000 7f54de000700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.328+0000 7f54df400700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.329+0000 7f54de000700 1 -- 172.21.15.7:0/1484143232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 msgr2=0x5631d62bf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.329+0000 7f54de000700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62bf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.329+0000 7f54de000700 1 -- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 msgr2=0x5631d62c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.329+0000 7f54de000700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.329+0000 7f54de000700 1 -- 172.21.15.7:0/1484143232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d61588d0 con 0x5631d62b7680 2024-06-13T06:59:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.331+0000 7f54de000700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62bef30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54d0011290 tx=0x7f54d00115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b54b8ee013c75f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.332+0000 7f54c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.332+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0019070 con 0x5631d62b7680 2024-06-13T06:59:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.332+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54d0015460 con 0x5631d62b7680 2024-06-13T06:59:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.334+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc011ce0 con 0x5631d62b7680 2024-06-13T06:59:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.335+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54d0023050 con 0x5631d62b7680 2024-06-13T06:59:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.335+0000 7f54f0195b80 10 client.25859 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.335+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631d62cf530 con 0x5631d62b7680 2024-06-13T06:59:31.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:30 smithi189 ceph-mon[36616]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-06-13T06:59:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc011ce0 con 0x5631d62b7680 2024-06-13T06:59:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f54d001e9b0 con 0x5631d62b7680 2024-06-13T06:59:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 10 client.25859 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 10 client.25859 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:59:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 20 client.25859 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d62d1b70 0x5631d62d3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 4 client.25859 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.338+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d6217000 con 0x5631d62d1b70 2024-06-13T06:59:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.339+0000 7f54df400700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d62d1b70 0x5631d62d3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.341+0000 7f54df400700 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d62d1b70 0x5631d62d3f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=886589cb62df4ec7 server_cookie=69002aedf005e6bf in_seq=0 out_seq=0 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.343+0000 7f54c8c00700 10 client.25859 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.344+0000 7f54c8c00700 1 -- 172.21.15.7:0/1484143232 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5631d6217000 con 0x5631d62d1b70 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.344+0000 7f54c8c00700 10 client.25859 handle_command_reply: tid=0 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.344+0000 7f54f0195b80 1 client.25859 shutdown 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.344+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 msgr2=0x5631d62bef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.344+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62bef30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54d0011290 tx=0x7f54d00115a0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 shutdown_connections 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d62d1b70 0x5631d62d3f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 --2- 172.21.15.7:0/1484143232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62bef30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 >> 172.21.15.7:0/1484143232 conn(0x5631d61e4170 msgr2=0x5631d62b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54c8c00700 0 client.25859 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:59:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 shutdown_connections 2024-06-13T06:59:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 1 -- 172.21.15.7:0/1484143232 wait complete. 2024-06-13T06:59:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.345+0000 7f54f0195b80 20 client.25859 trim_cache size 0 max 16384 2024-06-13T06:59:31.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:30 smithi175 ceph-mon[36886]: pgmap v613: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-06-13T06:59:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.389+0000 7f54f0195b80 1 Processor -- start 2024-06-13T06:59:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.389+0000 7f54f0195b80 1 -- start start 2024-06-13T06:59:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b7680 0x5631d62ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62c2530 0x5631d62d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01c7f60 con 0x5631d62b7680 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b27d0 con 0x5631d62c2530 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.390+0000 7f54f0195b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b0620 con 0x5631d62b6d60 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.391+0000 7f54df400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.391+0000 7f54df400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49154/0 (socket says 172.21.15.7:49154) 2024-06-13T06:59:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.391+0000 7f54df400700 1 -- 172.21.15.7:0/3575255580 learned_addr learned my addr 172.21.15.7:0/3575255580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.391+0000 7f54df400700 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62c2530 0x5631d62d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.392+0000 7f54df400700 1 -- 172.21.15.7:0/3575255580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62c2530 msgr2=0x5631d62d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.392+0000 7f54df400700 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62c2530 0x5631d62d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.392+0000 7f54df400700 1 -- 172.21.15.7:0/3575255580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b7680 msgr2=0x5631d62ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.392+0000 7f54df400700 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b7680 0x5631d62ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.392+0000 7f54df400700 1 -- 172.21.15.7:0/3575255580 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d62bd4a0 con 0x5631d62b6d60 2024-06-13T06:59:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.393+0000 7f54df400700 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00cac0 tx=0x7f54cc00ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dcadc0d6b3feb35 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.394+0000 7f54cb400700 1 -- 172.21.15.7:0/3575255580 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc030070 con 0x5631d62b6d60 2024-06-13T06:59:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.395+0000 7f54cb400700 1 -- 172.21.15.7:0/3575255580 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc006ab0 con 0x5631d62b6d60 2024-06-13T06:59:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.395+0000 7f54cb400700 1 -- 172.21.15.7:0/3575255580 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc015040 con 0x5631d62b6d60 2024-06-13T06:59:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.397+0000 7f54f0195b80 1 -- 172.21.15.7:0/3575255580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 msgr2=0x5631d62b9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.397+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00cac0 tx=0x7f54cc00ca50 comp rx=0 tx=0).stop 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 -- 172.21.15.7:0/3575255580 shutdown_connections 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62c2530 0x5631d62d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62b6d60 0x5631d62b9f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 --2- 172.21.15.7:0/3575255580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b7680 0x5631d62ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 -- 172.21.15.7:0/3575255580 >> 172.21.15.7:0/3575255580 conn(0x5631d61e4170 msgr2=0x5631d62bd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.398+0000 7f54f0195b80 1 -- 172.21.15.7:0/3575255580 shutdown_connections 2024-06-13T06:59:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.399+0000 7f54f0195b80 1 -- 172.21.15.7:0/3575255580 wait complete. 2024-06-13T06:59:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.399+0000 7f54f0195b80 1 -- start start 2024-06-13T06:59:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.401+0000 7f54f0195b80 1 Processor -- start 2024-06-13T06:59:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.403+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b0c90 con 0x5631d62b6d60 2024-06-13T06:59:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b2100 con 0x5631d62b7680 2024-06-13T06:59:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b1360 con 0x5631d62c2530 2024-06-13T06:59:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54f0195b80 10 client.0 _ll_register_callbacks cb 0x5631d619f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54de000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54de000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57598/0 (socket says 172.21.15.7:57598) 2024-06-13T06:59:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.404+0000 7f54de000700 1 -- 172.21.15.7:0/405175257 learned_addr learned my addr 172.21.15.7:0/405175257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.405+0000 7f54de000700 1 -- 172.21.15.7:0/405175257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 msgr2=0x5631d62c3a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.405+0000 7f54de000700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.405+0000 7f54de000700 1 -- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 msgr2=0x5631d62ca7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.406+0000 7f54df400700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.407+0000 7f54de000700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.407+0000 7f54de000700 1 -- 172.21.15.7:0/405175257 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d62c3fd0 con 0x5631d62b7680 2024-06-13T06:59:31.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.408+0000 7f54c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:31.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.408+0000 7f54de000700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d0008010 tx=0x7f54d00092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124ff153b399d112 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.409+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d002c070 con 0x5631d62b7680 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.409+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54d001c360 con 0x5631d62b7680 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.410+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc006ab0 con 0x5631d62b7680 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.410+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54d0036050 con 0x5631d62b7680 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.410+0000 7f54f0195b80 10 client.25865 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.410+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631d62c90a0 con 0x5631d62b7680 2024-06-13T06:59:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.410+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc006ab0 con 0x5631d62b7680 2024-06-13T06:59:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f54d00148b0 con 0x5631d62b7680 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 10 client.25865 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 10 client.25865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 20 client.25865 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d633cd70 0x5631d633f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 4 client.25865 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631d6211580 con 0x5631d633cd70 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.411+0000 7f54df400700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d633cd70 0x5631d633f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.413+0000 7f54df400700 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d633cd70 0x5631d633f190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d34287609c84ec95 server_cookie=8f4048144d79b1e9 in_seq=0 out_seq=0 2024-06-13T06:59:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.413+0000 7f54c8c00700 10 client.25865 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T06:59:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.413+0000 7f54c8c00700 1 -- 172.21.15.7:0/405175257 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631d6211580 con 0x5631d633cd70 2024-06-13T06:59:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.413+0000 7f54c8c00700 10 client.25865 handle_command_reply: tid=0 2024-06-13T06:59:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.413+0000 7f54f0195b80 1 client.25865 shutdown 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 msgr2=0x5631d62c34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d0008010 tx=0x7f54d00092c0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 shutdown_connections 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5631d633cd70 0x5631d633f190 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631d62c2530 0x5631d62c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5631d62b6d60 0x5631d62ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 --2- 172.21.15.7:0/405175257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5631d62b7680 0x5631d62c34d0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d0008010 tx=0x7f54d00092c0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54c8c00700 0 client.25865 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T06:59:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.414+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 >> 172.21.15.7:0/405175257 conn(0x5631d61e4170 msgr2=0x5631d62be7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.415+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 shutdown_connections 2024-06-13T06:59:31.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.415+0000 7f54f0195b80 1 -- 172.21.15.7:0/405175257 wait complete. 2024-06-13T06:59:31.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.416+0000 7f54f0195b80 20 client.25865 trim_cache size 0 max 16384 2024-06-13T06:59:31.415 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:59:31.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 msgr2=0x7f54c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 0x7f54c40a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54d809c6c0 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-06-13T06:59:31.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 msgr2=0x7f54e00769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:31.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b820 tx=0x7f54d8007ae0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 shutdown_connections 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e010a670 0x7f54e0077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54c40a2cf0 0x7f54c40a51b0 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54d809c6c0 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54e01045c0 0x7f54e00769a0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b820 tx=0x7f54d8007ae0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 --2- 172.21.15.7:0/2397316819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54e0104f80 0x7f54e0076ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.417+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 >> 172.21.15.7:0/2397316819 conn(0x7f54e0100070 msgr2=0x7f54e010b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.418+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 shutdown_connections 2024-06-13T06:59:31.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:31.418+0000 7f54dfe00700 1 -- 172.21.15.7:0/2397316819 wait complete. 2024-06-13T06:59:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 210.0 s, 1129.0 tps, lat 89.560 ms stddev 158.635 2024-06-13T06:59:32.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T06:59:33.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:32 smithi007 ceph-mon[32077]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 510 op/s 2024-06-13T06:59:33.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:32 smithi189 ceph-mon[36616]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 510 op/s 2024-06-13T06:59:33.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:32 smithi175 ceph-mon[36886]: pgmap v614: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 510 op/s 2024-06-13T06:59:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:33 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:33 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:33 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T06:59:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-06-13T06:59:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T06:59:35.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:34 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T06:59:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.364+0000 7ff66b200640 1 -- 172.21.15.7:0/2042853917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c109970 msgr2=0x7ff66c10e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.364+0000 7ff66b200640 1 --2- 172.21.15.7:0/2042853917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c109970 0x7ff66c10e2e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009be0 tx=0x7ff65c02a6d0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.364+0000 7ff66b200640 1 -- 172.21.15.7:0/2042853917 shutdown_connections 2024-06-13T06:59:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.364+0000 7ff66b200640 1 --2- 172.21.15.7:0/2042853917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c109970 0x7ff66c10e2e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.364+0000 7ff66b200640 1 --2- 172.21.15.7:0/2042853917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c108fb0 0x7ff66c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.365+0000 7ff66b200640 1 --2- 172.21.15.7:0/2042853917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c072cd0 0x7ff66c0730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.365+0000 7ff66b200640 1 -- 172.21.15.7:0/2042853917 >> 172.21.15.7:0/2042853917 conn(0x7ff66c06e840 msgr2=0x7ff66c070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T06:59:35.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T06:59:35.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:34 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T06:59:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.367+0000 7ff66b200640 1 -- 172.21.15.7:0/2042853917 shutdown_connections 2024-06-13T06:59:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.368+0000 7ff66b200640 1 -- 172.21.15.7:0/2042853917 wait complete. 2024-06-13T06:59:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.371+0000 7ff66b200640 1 Processor -- start 2024-06-13T06:59:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.372+0000 7ff66b200640 1 -- start start 2024-06-13T06:59:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff66b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff66b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff66b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c109970 0x7ff66c1adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff669e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff669e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49170/0 (socket says 172.21.15.7:49170) 2024-06-13T06:59:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.373+0000 7ff669400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55428/0 (socket says 172.21.15.7:55428) 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669e00640 1 -- 172.21.15.7:0/3904320345 learned_addr learned my addr 172.21.15.7:0/3904320345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c11c310 con 0x7ff66c108fb0 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c11c190 con 0x7ff66c109970 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c11c490 con 0x7ff66c072cd0 2024-06-13T06:59:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff66a800640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c109970 0x7ff66c1adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669400640 1 -- 172.21.15.7:0/3904320345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 msgr2=0x7ff66c1a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669400640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669400640 1 -- 172.21.15.7:0/3904320345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c109970 msgr2=0x7ff66c1adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.374+0000 7ff669400640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c109970 0x7ff66c1adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.375+0000 7ff669400640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1ae550 con 0x7ff66c108fb0 2024-06-13T06:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.375+0000 7ff669e00640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.376+0000 7ff669400640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d740 tx=0x7ff65400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.377+0000 7ff662a00640 1 -- 172.21.15.7:0/3904320345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654017070 con 0x7ff66c108fb0 2024-06-13T06:59:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.378+0000 7ff662a00640 1 -- 172.21.15.7:0/3904320345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65400de50 con 0x7ff66c108fb0 2024-06-13T06:59:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.378+0000 7ff662a00640 1 -- 172.21.15.7:0/3904320345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400f620 con 0x7ff66c108fb0 2024-06-13T06:59:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.378+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1ae840 con 0x7ff66c108fb0 2024-06-13T06:59:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.378+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1aed40 con 0x7ff66c108fb0 2024-06-13T06:59:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.394+0000 7ff662a00640 1 -- 172.21.15.7:0/3904320345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff65401a070 con 0x7ff66c108fb0 2024-06-13T06:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.395+0000 7ff67384c740 1 Processor -- start 2024-06-13T06:59:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.397+0000 7ff67384c740 1 -- start start 2024-06-13T06:59:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbaf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bb0d420 con 0x55bb7bbaf920 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bab4c50 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff67384c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bb093e0 con 0x55bb7bba9dc0 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff66a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff66a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49178/0 (socket says 172.21.15.7:49178) 2024-06-13T06:59:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.398+0000 7ff66a800640 1 -- 172.21.15.7:0/995075525 learned_addr learned my addr 172.21.15.7:0/995075525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff669e00640 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbaf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff66a800640 1 -- 172.21.15.7:0/995075525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 msgr2=0x55bb7bbaf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff66a800640 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbaf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff66a800640 1 -- 172.21.15.7:0/995075525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 msgr2=0x55bb7bbb1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff66a800640 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.399+0000 7ff66a800640 1 -- 172.21.15.7:0/995075525 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7b821210 con 0x55bb7bba9dc0 2024-06-13T06:59:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.401+0000 7ff66a800640 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009be0 tx=0x7ff65c030be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1b104fd72f1c77 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.402+0000 7ff65b000640 1 -- 172.21.15.7:0/995075525 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c041030 con 0x55bb7bba9dc0 2024-06-13T06:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.402+0000 7ff65b000640 1 -- 172.21.15.7:0/995075525 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c030e10 con 0x55bb7bba9dc0 2024-06-13T06:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.402+0000 7ff65b000640 1 -- 172.21.15.7:0/995075525 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c036040 con 0x55bb7bba9dc0 2024-06-13T06:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.403+0000 7ff67384c740 1 -- 172.21.15.7:0/995075525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 msgr2=0x55bb7bbaa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.403+0000 7ff67384c740 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009be0 tx=0x7ff65c030be0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.404+0000 7ff662a00640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.404+0000 7ff662a00640 1 -- 172.21.15.7:0/3904320345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff6540cec30 con 0x7ff66c108fb0 2024-06-13T06:59:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff67384c740 1 -- 172.21.15.7:0/995075525 shutdown_connections 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff67384c740 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff67384c740 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbaf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff67384c740 1 --2- 172.21.15.7:0/995075525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbaa1e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff67384c740 1 -- 172.21.15.7:0/995075525 >> 172.21.15.7:0/995075525 conn(0x55bb7bb71fb0 msgr2=0x55bb7bba5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.405+0000 7ff66a800640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.406+0000 7ff67384c740 1 -- 172.21.15.7:0/995075525 shutdown_connections 2024-06-13T06:59:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.406+0000 7ff67384c740 1 -- 172.21.15.7:0/995075525 wait complete. 2024-06-13T06:59:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.407+0000 7ff67384c740 1 -- start start 2024-06-13T06:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.407+0000 7ff67384c740 1 Processor -- start 2024-06-13T06:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bbb2870 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bac80b0 con 0x55bb7bba9dc0 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bb7bbb29f0 con 0x55bb7bbaf920 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff66a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff669e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff66a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49180/0 (socket says 172.21.15.7:49180) 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff669e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57638/0 (socket says 172.21.15.7:57638) 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff66a800640 1 -- 172.21.15.7:0/281338995 learned_addr learned my addr 172.21.15.7:0/281338995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.408+0000 7ff67384c740 10 client.0 _ll_register_callbacks cb 0x55bb7bab3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff669400640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 -- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 msgr2=0x55bb7bbb41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 -- 172.21.15.7:0/281338995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaa7b0 msgr2=0x55bb7bbb46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 -- 172.21.15.7:0/281338995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7b8ed620 con 0x55bb7bbaf920 2024-06-13T06:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff64f400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.409+0000 7ff66a800640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff65c030960 tx=0x7ff65c030610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91eb98213cf57f68 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.410+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c041070 con 0x55bb7bbaf920 2024-06-13T06:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.410+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff65c053450 con 0x55bb7bbaf920 2024-06-13T06:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.410+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c030e10 con 0x55bb7bbaf920 2024-06-13T06:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.411+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff65c058060 con 0x55bb7bbaf920 2024-06-13T06:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.411+0000 7ff67384c740 10 client.25855 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.411+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb7bbb2870 con 0x55bb7bbaf920 2024-06-13T06:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c030e10 con 0x55bb7bbaf920 2024-06-13T06:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff65c05b050 con 0x55bb7bbaf920 2024-06-13T06:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff66a800640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 0x7ff6500a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff65c055000 tx=0x7ff65c04b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: pgmap v615: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T06:59:35.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T06:59:35.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:34 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T06:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 10 client.25855 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 20 client.25855 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bbbfc90 0x55bb7bbc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 4 client.25855 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.412+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7bb1ab50 con 0x55bb7bbbfc90 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.413+0000 7ff669e00640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bbbfc90 0x55bb7bbc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.413+0000 7ff64f400640 10 client.25855 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.414+0000 7ff669e00640 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bbbfc90 0x55bb7bbc2080 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2a11aab5366ba8 server_cookie=9e32bc39b0ffcce in_seq=0 out_seq=0 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.415+0000 7ff64f400640 1 -- 172.21.15.7:0/281338995 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb7bb1ab50 con 0x55bb7bbbfc90 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.415+0000 7ff64f400640 10 client.25855 handle_command_reply: tid=0 2024-06-13T06:59:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.416+0000 7ff67384c740 1 client.25855 shutdown 2024-06-13T06:59:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 msgr2=0x55bb7bbbb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff65c030960 tx=0x7ff65c030610 comp rx=0 tx=0).stop 2024-06-13T06:59:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 shutdown_connections 2024-06-13T06:59:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bbbfc90 0x55bb7bbc2080 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaf920 0x55bb7bbbb730 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 --2- 172.21.15.7:0/281338995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff64f400640 0 client.25855 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.417+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 >> 172.21.15.7:0/281338995 conn(0x55bb7bb71fb0 msgr2=0x55bb7bba49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.418+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 shutdown_connections 2024-06-13T06:59:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.419+0000 7ff67384c740 1 -- 172.21.15.7:0/281338995 wait complete. 2024-06-13T06:59:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.419+0000 7ff67384c740 20 client.25855 trim_cache size 0 max 16384 2024-06-13T06:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.477+0000 7ff67384c740 1 Processor -- start 2024-06-13T06:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.477+0000 7ff67384c740 1 -- start start 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.479+0000 7ff67384c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bba9dc0 0x55bb7bba59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.480+0000 7ff67384c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.480+0000 7ff67384c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.480+0000 7ff67384c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1bb970 con 0x55bb7bba9dc0 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.480+0000 7ff67384c740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1b9240 con 0x55bb7bbaf920 2024-06-13T06:59:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.480+0000 7ff67384c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1b7090 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.481+0000 7ff669400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.481+0000 7ff669400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49182/0 (socket says 172.21.15.7:49182) 2024-06-13T06:59:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.481+0000 7ff669400640 1 -- 172.21.15.7:0/2592864292 learned_addr learned my addr 172.21.15.7:0/2592864292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.481+0000 7ff669e00640 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bba9dc0 0x55bb7bba59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.481+0000 7ff66a800640 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 -- 172.21.15.7:0/2592864292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaf920 msgr2=0x55bb7bbb2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 -- 172.21.15.7:0/2592864292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bba9dc0 msgr2=0x55bb7bba59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bba9dc0 0x55bb7bba59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 -- 172.21.15.7:0/2592864292 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7bbc26e0 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff669400640 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff654097660 tx=0x7ff6540cef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22cdc658dbfb4614 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff65a600640 1 -- 172.21.15.7:0/2592864292 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654021070 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff65a600640 1 -- 172.21.15.7:0/2592864292 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff654099500 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.482+0000 7ff65a600640 1 -- 172.21.15.7:0/2592864292 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65409f120 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.483+0000 7ff67384c740 1 -- 172.21.15.7:0/2592864292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 msgr2=0x55bb7bbbb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.483+0000 7ff67384c740 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff654097660 tx=0x7ff6540cef30 comp rx=0 tx=0).stop 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.484+0000 7ff67384c740 1 -- 172.21.15.7:0/2592864292 shutdown_connections 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.484+0000 7ff67384c740 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.484+0000 7ff67384c740 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbbb6f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.484+0000 7ff67384c740 1 --2- 172.21.15.7:0/2592864292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bba9dc0 0x55bb7bba59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.484+0000 7ff67384c740 1 -- 172.21.15.7:0/2592864292 >> 172.21.15.7:0/2592864292 conn(0x55bb7bb71fb0 msgr2=0x55bb7bba52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.488+0000 7ff67384c740 1 -- 172.21.15.7:0/2592864292 shutdown_connections 2024-06-13T06:59:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.488+0000 7ff67384c740 1 -- 172.21.15.7:0/2592864292 wait complete. 2024-06-13T06:59:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.489+0000 7ff67384c740 1 -- start start 2024-06-13T06:59:35.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.489+0000 7ff67384c740 1 Processor -- start 2024-06-13T06:59:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.490+0000 7ff67384c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.491+0000 7ff669e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.491+0000 7ff669e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:49198/0 (socket says 172.21.15.7:49198) 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.491+0000 7ff67384c740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.491+0000 7ff67384c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.492+0000 7ff66a800640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.492+0000 7ff669e00640 1 -- 172.21.15.7:0/447972591 learned_addr learned my addr 172.21.15.7:0/447972591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:35.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.492+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1b7700 con 0x55bb7bbaf920 2024-06-13T06:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.493+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1b8b70 con 0x55bb7bbaa7b0 2024-06-13T06:59:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.493+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1b7dd0 con 0x55bb7bba9dc0 2024-06-13T06:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.494+0000 7ff67384c740 10 client.0 _ll_register_callbacks cb 0x7ff65c030b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.495+0000 7ff669400640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.496+0000 7ff66a800640 1 -- 172.21.15.7:0/447972591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 msgr2=0x55bb7bbb8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff66a800640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff66a800640 1 -- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 msgr2=0x55bb7bbb9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff66a800640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff66a800640 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7bbba5e0 con 0x55bb7bbaf920 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff669e00640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.497+0000 7ff669400640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.498+0000 7ff64f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.498+0000 7ff66a800640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb9ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f80 tx=0x7ff65c030700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265c40a8cbc194da server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.499+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c041040 con 0x55bb7bbaf920 2024-06-13T06:59:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.500+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff65c003f20 con 0x55bb7bbaf920 2024-06-13T06:59:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.500+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff654099500 con 0x55bb7bbaf920 2024-06-13T06:59:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.502+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff65c036040 con 0x55bb7bbaf920 2024-06-13T06:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.505+0000 7ff67384c740 10 client.15564 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.505+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bb7bbbb8b0 con 0x55bb7bbaf920 2024-06-13T06:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.506+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff654099500 con 0x55bb7bbaf920 2024-06-13T06:59:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.506+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff65c05b020 con 0x55bb7bbaf920 2024-06-13T06:59:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 10 client.15564 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 20 client.15564 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bc2aea0 0x55bb7bc2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 4 client.15564 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.507+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7bb14ea0 con 0x55bb7bc2aea0 2024-06-13T06:59:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.508+0000 7ff669e00640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bc2aea0 0x55bb7bc2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.510+0000 7ff64f400640 10 client.15564 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:35.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.510+0000 7ff669e00640 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bc2aea0 0x55bb7bc2d2c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93722bde0b75aaf5 server_cookie=938c5a14f8ad2bf6 in_seq=0 out_seq=0 2024-06-13T06:59:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.830+0000 7ff64f400640 1 -- 172.21.15.7:0/447972591 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb7bb14ea0 con 0x55bb7bc2aea0 2024-06-13T06:59:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.830+0000 7ff64f400640 10 client.15564 handle_command_reply: tid=0 2024-06-13T06:59:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.831+0000 7ff67384c740 1 client.15564 shutdown 2024-06-13T06:59:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.833+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 msgr2=0x55bb7bbb9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.833+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb9ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f80 tx=0x7ff65c030700 comp rx=0 tx=0).stop 2024-06-13T06:59:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.834+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 shutdown_connections 2024-06-13T06:59:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.834+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55bb7bc2aea0 0x55bb7bc2d2c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.834+0000 7ff64f400640 0 client.15564 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.836+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7bbaf920 0x55bb7bbb9ff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.836+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bb7bbaa7b0 0x55bb7bbb9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.836+0000 7ff67384c740 1 --2- 172.21.15.7:0/447972591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb7bba9dc0 0x55bb7bbb8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.836+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 >> 172.21.15.7:0/447972591 conn(0x55bb7bb71fb0 msgr2=0x55bb7bba4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.836+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 shutdown_connections 2024-06-13T06:59:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.837+0000 7ff67384c740 1 -- 172.21.15.7:0/447972591 wait complete. 2024-06-13T06:59:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.837+0000 7ff67384c740 20 client.15564 trim_cache size 0 max 16384 2024-06-13T06:59:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:59:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:59:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 msgr2=0x7ff6500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 0x7ff6500a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff65c055000 tx=0x7ff65c04b000 comp rx=0 tx=0).stop 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 msgr2=0x7ff66c1a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff65400d740 tx=0x7ff65400dc00 comp rx=0 tx=0).stop 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 shutdown_connections 2024-06-13T06:59:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff6500a2c00 0x7ff6500a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff66c109970 0x7ff66c1adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c108fb0 0x7ff66c1a6e70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 --2- 172.21.15.7:0/3904320345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff66c072cd0 0x7ff66c1a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.840+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 >> 172.21.15.7:0/3904320345 conn(0x7ff66c06e840 msgr2=0x7ff66c070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.841+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 shutdown_connections 2024-06-13T06:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:35.841+0000 7ff66b200640 1 -- 172.21.15.7:0/3904320345 wait complete. 2024-06-13T06:59:36.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.636+0000 7f8137800700 1 -- 172.21.15.7:0/1458158483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 msgr2=0x7f8130109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.636+0000 7f8137800700 1 --2- 172.21.15.7:0/1458158483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f8130109e50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81180098f0 tx=0x7f8118009c00 comp rx=0 tx=0).stop 2024-06-13T06:59:36.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.638+0000 7f8137800700 1 -- 172.21.15.7:0/1458158483 shutdown_connections 2024-06-13T06:59:36.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.638+0000 7f8137800700 1 --2- 172.21.15.7:0/1458158483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f813011bc70 0x7f813011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.638+0000 7f8137800700 1 --2- 172.21.15.7:0/1458158483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8130110c70 0x7f8130109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.638+0000 7f8137800700 1 --2- 172.21.15.7:0/1458158483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f8130109e50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.638+0000 7f8137800700 1 -- 172.21.15.7:0/1458158483 >> 172.21.15.7:0/1458158483 conn(0x7f813006d600 msgr2=0x7f813006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.639+0000 7f8137800700 1 -- 172.21.15.7:0/1458158483 shutdown_connections 2024-06-13T06:59:36.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.640+0000 7f8137800700 1 -- 172.21.15.7:0/1458158483 wait complete. 2024-06-13T06:59:36.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.642+0000 7f8137800700 1 Processor -- start 2024-06-13T06:59:36.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.642+0000 7f8137800700 1 -- start start 2024-06-13T06:59:36.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.642+0000 7f8137800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.643+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.643+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57670/0 (socket says 172.21.15.7:57670) 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.643+0000 7f8137800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8130110c70 0x7f81301add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8137800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 0x7f81301a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8137800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81301216d0 con 0x7f813011bc70 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8137800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8130121550 con 0x7f81301099d0 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8137800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8130121850 con 0x7f8130110c70 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8134e00700 1 -- 172.21.15.7:0/1819831839 learned_addr learned my addr 172.21.15.7:0/1819831839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f812fe00700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8130110c70 0x7f81301add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.644+0000 7f8135800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 0x7f81301a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8134e00700 1 -- 172.21.15.7:0/1819831839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8130110c70 msgr2=0x7f81301add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8134e00700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8130110c70 0x7f81301add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8134e00700 1 -- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 msgr2=0x7f81301a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8134e00700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 0x7f81301a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8134e00700 1 -- 172.21.15.7:0/1819831839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a8150 con 0x7f81301099d0 2024-06-13T06:59:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.645+0000 7f8135800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 0x7f81301a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.646+0000 7f8134e00700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81280060b0 tx=0x7f8128007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.646+0000 7f812d600700 1 -- 172.21.15.7:0/1819831839 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812801a070 con 0x7f81301099d0 2024-06-13T06:59:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.647+0000 7f812d600700 1 -- 172.21.15.7:0/1819831839 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812800de20 con 0x7f81301099d0 2024-06-13T06:59:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.647+0000 7f812d600700 1 -- 172.21.15.7:0/1819831839 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128014790 con 0x7f81301099d0 2024-06-13T06:59:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.648+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a8440 con 0x7f81301099d0 2024-06-13T06:59:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.648+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301b4700 con 0x7f81301099d0 2024-06-13T06:59:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.656+0000 7f8142254b80 1 Processor -- start 2024-06-13T06:59:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.657+0000 7f8142254b80 1 -- start start 2024-06-13T06:59:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f6270b90 0x55b0f6270f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 0x55b0f626b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b0f6179a10 con 0x55b0f6270b90 2024-06-13T06:59:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b0f61bcb00 con 0x55b0f62714b0 2024-06-13T06:59:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.658+0000 7f8142254b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b0f61778f0 con 0x55b0f626b240 2024-06-13T06:59:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.660+0000 7f812d600700 1 -- 172.21.15.7:0/1819831839 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f812801d070 con 0x7f81301099d0 2024-06-13T06:59:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.662+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.662+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57684/0 (socket says 172.21.15.7:57684) 2024-06-13T06:59:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.662+0000 7f8134e00700 1 -- 172.21.15.7:0/1762223533 learned_addr learned my addr 172.21.15.7:0/1762223533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.662+0000 7f812d600700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 0x7f81140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.662+0000 7f812d600700 1 -- 172.21.15.7:0/1819831839 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f81280cec60 con 0x7f81301099d0 2024-06-13T06:59:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.663+0000 7f812fe00700 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 0x55b0f626b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f812fe00700 1 -- 172.21.15.7:0/1762223533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 msgr2=0x55b0f626ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f812fe00700 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f812fe00700 1 -- 172.21.15.7:0/1762223533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f6270b90 msgr2=0x55b0f6270f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f812fe00700 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f6270b90 0x55b0f6270f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f812fe00700 1 -- 172.21.15.7:0/1762223533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f6164f90 con 0x55b0f626b240 2024-06-13T06:59:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.665+0000 7f8134e00700 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.666+0000 7f812fe00700 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 0x55b0f626b680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f81180098c0 tx=0x7f81180095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a571be2b8733f1f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.668+0000 7f8126000700 1 -- 172.21.15.7:0/1762223533 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118027070 con 0x55b0f626b240 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.668+0000 7f8126000700 1 -- 172.21.15.7:0/1762223533 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118005520 con 0x55b0f626b240 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.668+0000 7f8126000700 1 -- 172.21.15.7:0/1762223533 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004d40 con 0x55b0f626b240 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.669+0000 7f8142254b80 1 -- 172.21.15.7:0/1762223533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 msgr2=0x55b0f626b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.669+0000 7f8142254b80 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 0x55b0f626b680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f81180098c0 tx=0x7f81180095f0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.669+0000 7f8135800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 0x7f81140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 -- 172.21.15.7:0/1762223533 shutdown_connections 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f626b240 0x55b0f626b680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f6270b90 0x55b0f6270f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 --2- 172.21.15.7:0/1762223533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f626ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 -- 172.21.15.7:0/1762223533 >> 172.21.15.7:0/1762223533 conn(0x55b0f6197170 msgr2=0x55b0f6265ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8135800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 0x7f81140a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f813006bc00 tx=0x7f8120009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.670+0000 7f8142254b80 1 -- 172.21.15.7:0/1762223533 shutdown_connections 2024-06-13T06:59:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.671+0000 7f8142254b80 1 -- 172.21.15.7:0/1762223533 wait complete. 2024-06-13T06:59:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.671+0000 7f8142254b80 1 -- start start 2024-06-13T06:59:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.672+0000 7f8142254b80 1 Processor -- start 2024-06-13T06:59:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.673+0000 7f8142254b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.674+0000 7f8142254b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f62745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.674+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.674+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57690/0 (socket says 172.21.15.7:57690) 2024-06-13T06:59:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.674+0000 7f8134e00700 1 -- 172.21.15.7:0/1881370805 learned_addr learned my addr 172.21.15.7:0/1881370805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.674+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6274b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.675+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b0f6158f60 con 0x55b0f62714b0 2024-06-13T06:59:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.675+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b0f61b8ac0 con 0x55b0f626b240 2024-06-13T06:59:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.675+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b0f6277e30 con 0x55b0f6270b90 2024-06-13T06:59:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.675+0000 7f812fe00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f62745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.676+0000 7f8135800700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6274b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.676+0000 7f812fe00700 1 -- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 msgr2=0x55b0f62772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.676+0000 7f812fe00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.676+0000 7f812fe00700 1 -- 172.21.15.7:0/1881370805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 msgr2=0x55b0f6274b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.676+0000 7f812fe00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6274b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.677+0000 7f8142254b80 10 client.0 _ll_register_callbacks cb 0x55b0f6152380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.677+0000 7f812fe00700 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f610b8d0 con 0x55b0f6270b90 2024-06-13T06:59:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.677+0000 7f8134e00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.677+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118005520 con 0x55b0f6270b90 2024-06-13T06:59:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.678+0000 7f811ea00700 10 client.25873 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.678+0000 7f812fe00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f62745d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8118009680 tx=0x7f811800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e827c5cc75ba92e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.678+0000 7f811ea00700 1 -- 172.21.15.7:0/1881370805 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118031020 con 0x55b0f6270b90 2024-06-13T06:59:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.678+0000 7f811ea00700 1 -- 172.21.15.7:0/1881370805 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f811800b9b0 con 0x55b0f6270b90 2024-06-13T06:59:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.679+0000 7f811ea00700 1 -- 172.21.15.7:0/1881370805 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8118031020 con 0x55b0f6270b90 2024-06-13T06:59:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.679+0000 7f8142254b80 10 client.25873 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.679+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b0f6282450 con 0x55b0f6270b90 2024-06-13T06:59:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.681+0000 7f811ea00700 1 -- 172.21.15.7:0/1881370805 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f811802a030 con 0x55b0f6270b90 2024-06-13T06:59:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 10 client.25873 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 10 client.25873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 20 client.25873 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f6284a60 0x55b0f6286ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 4 client.25873 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.682+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f61ca000 con 0x55b0f6284a60 2024-06-13T06:59:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.683+0000 7f8134e00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f6284a60 0x55b0f6286ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.683+0000 7f8134e00700 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f6284a60 0x55b0f6286ee0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d627b95d3b59d9b0 server_cookie=a581523a047829b9 in_seq=0 out_seq=0 2024-06-13T06:59:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.683+0000 7f811ea00700 10 client.25873 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.684+0000 7f811ea00700 1 -- 172.21.15.7:0/1881370805 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b0f61ca000 con 0x55b0f6284a60 2024-06-13T06:59:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.684+0000 7f811ea00700 10 client.25873 handle_command_reply: tid=0 2024-06-13T06:59:36.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.685+0000 7f8142254b80 1 client.25873 shutdown 2024-06-13T06:59:36.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 msgr2=0x55b0f62745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f62745d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8118009680 tx=0x7f811800b790 comp rx=0 tx=0).stop 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 shutdown_connections 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f6284a60 0x55b0f6286ee0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6274b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f62772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 --2- 172.21.15.7:0/1881370805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f62745d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f811ea00700 0 client.25873 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 >> 172.21.15.7:0/1881370805 conn(0x55b0f6197170 msgr2=0x55b0f625f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 shutdown_connections 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.686+0000 7f8142254b80 1 -- 172.21.15.7:0/1881370805 wait complete. 2024-06-13T06:59:36.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.687+0000 7f8142254b80 20 client.25873 trim_cache size 0 max 16384 2024-06-13T06:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.717+0000 7f8142254b80 1 Processor -- start 2024-06-13T06:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.718+0000 7f8142254b80 1 -- start start 2024-06-13T06:59:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.718+0000 7f8142254b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8142254b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f6274ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8142254b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6260130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8142254b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bf900 con 0x55b0f62714b0 2024-06-13T06:59:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8142254b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bd190 con 0x55b0f626b240 2024-06-13T06:59:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8142254b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bafe0 con 0x55b0f6270b90 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8134e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57704/0 (socket says 172.21.15.7:57704) 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f8134e00700 1 -- 172.21.15.7:0/3702907802 learned_addr learned my addr 172.21.15.7:0/3702907802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.719+0000 7f812fe00700 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f6274ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.720+0000 7f8134e00700 1 -- 172.21.15.7:0/3702907802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 msgr2=0x55b0f6274ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.720+0000 7f8134e00700 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f6274ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.720+0000 7f8134e00700 1 -- 172.21.15.7:0/3702907802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 msgr2=0x55b0f6260130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.720+0000 7f8134e00700 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6260130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.720+0000 7f8134e00700 1 -- 172.21.15.7:0/3702907802 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f6287560 con 0x55b0f626b240 2024-06-13T06:59:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.721+0000 7f8134e00700 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f81280cc420 tx=0x7f81280cc2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6cca8cf71c2866 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.721+0000 7f8125600700 1 -- 172.21.15.7:0/3702907802 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128099220 con 0x55b0f626b240 2024-06-13T06:59:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.721+0000 7f8125600700 1 -- 172.21.15.7:0/3702907802 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b0f61ca000 con 0x55b0f626b240 2024-06-13T06:59:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.721+0000 7f8125600700 1 -- 172.21.15.7:0/3702907802 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280bb0b0 con 0x55b0f626b240 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.723+0000 7f8142254b80 1 -- 172.21.15.7:0/3702907802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 msgr2=0x55b0f626ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.723+0000 7f8142254b80 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f81280cc420 tx=0x7f81280cc2f0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 -- 172.21.15.7:0/3702907802 shutdown_connections 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f62714b0 0x55b0f6260130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f626b240 0x55b0f626ad00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 --2- 172.21.15.7:0/3702907802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f6274ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 -- 172.21.15.7:0/3702907802 >> 172.21.15.7:0/3702907802 conn(0x55b0f6197170 msgr2=0x55b0f6274590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 -- 172.21.15.7:0/3702907802 shutdown_connections 2024-06-13T06:59:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.724+0000 7f8142254b80 1 -- 172.21.15.7:0/3702907802 wait complete. 2024-06-13T06:59:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.725+0000 7f8142254b80 1 -- start start 2024-06-13T06:59:36.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.726+0000 7f8142254b80 1 Processor -- start 2024-06-13T06:59:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f627d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f627d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bb650 con 0x55b0f626b240 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bcac0 con 0x55b0f62714b0 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81301bbd20 con 0x55b0f6270b90 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8142254b80 10 client.0 _ll_register_callbacks cb 0x55b0f6152380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8134e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8134e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55528/0 (socket says 172.21.15.7:55528) 2024-06-13T06:59:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8134e00700 1 -- 172.21.15.7:0/976315805 learned_addr learned my addr 172.21.15.7:0/976315805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.727+0000 7f8135800700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f627d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.728+0000 7f8134e00700 1 -- 172.21.15.7:0/976315805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 msgr2=0x55b0f627d0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.728+0000 7f8134e00700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f627d0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.728+0000 7f8134e00700 1 -- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 msgr2=0x55b0f627d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.728+0000 7f8134e00700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f627d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.728+0000 7f8134e00700 1 -- 172.21.15.7:0/976315805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f627db40 con 0x55b0f626b240 2024-06-13T06:59:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.730+0000 7f811ea00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T06:59:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.730+0000 7f8134e00700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f80 tx=0x7f8128009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa936efcc0a2228 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.730+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280b3070 con 0x55b0f626b240 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.730+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0f61ca000 con 0x55b0f626b240 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.730+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f812809f170 con 0x55b0f626b240 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8128015bf0 con 0x55b0f626b240 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f8142254b80 10 client.15573 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0f62ee780 con 0x55b0f626b240 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b0f61ca000 con 0x55b0f626b240 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f81280a3960 con 0x55b0f626b240 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.731+0000 7f8142254b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f8142254b80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f8142254b80 20 client.15573 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f62eebd0 0x55b0f62f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f8142254b80 4 client.15573 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T06:59:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0f61c4580 con 0x55b0f62eebd0 2024-06-13T06:59:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.732+0000 7f812fe00700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f62eebd0 0x55b0f62f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.733+0000 7f811ea00700 10 client.15573 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.734+0000 7f812fe00700 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f62eebd0 0x55b0f62f0fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdeca8d100d11ff server_cookie=7540da9c149ec59c in_seq=0 out_seq=0 2024-06-13T06:59:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f811ea00700 1 -- 172.21.15.7:0/976315805 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0f61c4580 con 0x55b0f62eebd0 2024-06-13T06:59:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f811ea00700 10 client.15573 handle_command_reply: tid=0 2024-06-13T06:59:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 client.15573 shutdown 2024-06-13T06:59:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 msgr2=0x55b0f628cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8128000f80 tx=0x7f8128009950 comp rx=0 tx=0).stop 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 shutdown_connections 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b0f62eebd0 0x55b0f62f0fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b0f62714b0 0x55b0f627d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b0f626b240 0x55b0f628cc10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 --2- 172.21.15.7:0/976315805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b0f6270b90 0x55b0f627d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f811ea00700 0 client.15573 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:36.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.735+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 >> 172.21.15.7:0/976315805 conn(0x55b0f6197170 msgr2=0x55b0f6271890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.736+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 shutdown_connections 2024-06-13T06:59:36.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.736+0000 7f8142254b80 1 -- 172.21.15.7:0/976315805 wait complete. 2024-06-13T06:59:36.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.736+0000 7f8142254b80 20 client.15573 trim_cache size 0 max 16384 2024-06-13T06:59:36.738 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T06:59:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 msgr2=0x7f81140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 0x7f81140a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f813006bc00 tx=0x7f8120009600 comp rx=0 tx=0).stop 2024-06-13T06:59:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 msgr2=0x7f81301ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81280060b0 tx=0x7f8128007f90 comp rx=0 tx=0).stop 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 shutdown_connections 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f813011bc70 0x7f81301a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f81140a2cf0 0x7f81140a51b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81301099d0 0x7f81301ad830 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 --2- 172.21.15.7:0/1819831839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8130110c70 0x7f81301add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.737+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 >> 172.21.15.7:0/1819831839 conn(0x7f813006d600 msgr2=0x7f8130119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.738+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 shutdown_connections 2024-06-13T06:59:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:36.738+0000 7f8137800700 1 -- 172.21.15.7:0/1819831839 wait complete. 2024-06-13T06:59:36.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:36 smithi007 ceph-mon[32077]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 588 op/s 2024-06-13T06:59:37.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:37.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:36 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:59:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:36 smithi189 ceph-mon[36616]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 588 op/s 2024-06-13T06:59:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:37.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:36 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:59:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:36 smithi175 ceph-mon[36886]: pgmap v616: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.6 MiB/s wr, 588 op/s 2024-06-13T06:59:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T06:59:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T06:59:37.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:36 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T06:59:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 215.0 s, 1691.4 tps, lat 58.678 ms stddev 57.427 2024-06-13T06:59:39.234 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:38 smithi007 ceph-mon[32077]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-06-13T06:59:39.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:38 smithi189 ceph-mon[36616]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-06-13T06:59:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:38 smithi175 ceph-mon[36886]: pgmap v617: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-06-13T06:59:40.716 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T06:59:40.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T06:59:41.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:40 smithi007 ceph-mon[32077]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 588 op/s 2024-06-13T06:59:41.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:40 smithi189 ceph-mon[36616]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 588 op/s 2024-06-13T06:59:41.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:40 smithi175 ceph-mon[36886]: pgmap v618: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.5 MiB/s wr, 588 op/s 2024-06-13T06:59:41.444 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 220.0 s, 1251.6 tps, lat 78.975 ms stddev 93.124 2024-06-13T06:59:43.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:42 smithi189 ceph-mon[36616]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-06-13T06:59:43.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:42 smithi175 ceph-mon[36886]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-06-13T06:59:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:42 smithi007 ceph-mon[32077]: pgmap v619: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.1 MiB/s wr, 551 op/s 2024-06-13T06:59:44.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 -- 172.21.15.7:0/4217242926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06a170 msgr2=0x7fdb3c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 --2- 172.21.15.7:0/4217242926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06a170 0x7fdb3c06a5f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009990 tx=0x7fdb24009ca0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 -- 172.21.15.7:0/4217242926 shutdown_connections 2024-06-13T06:59:44.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 --2- 172.21.15.7:0/4217242926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 --2- 172.21.15.7:0/4217242926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.625+0000 7fdb41a00700 1 --2- 172.21.15.7:0/4217242926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06a170 0x7fdb3c06a5f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.627+0000 7fdb41a00700 1 -- 172.21.15.7:0/4217242926 >> 172.21.15.7:0/4217242926 conn(0x7fdb3c078340 msgr2=0x7fdb3c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.628+0000 7fdb41a00700 1 -- 172.21.15.7:0/4217242926 shutdown_connections 2024-06-13T06:59:44.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.628+0000 7fdb41a00700 1 -- 172.21.15.7:0/4217242926 wait complete. 2024-06-13T06:59:44.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.630+0000 7fdb41a00700 1 Processor -- start 2024-06-13T06:59:44.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.631+0000 7fdb41a00700 1 -- start start 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.632+0000 7fdb41a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06a170 0x7fdb3c10b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb41a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb41a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c10beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb41a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c102000 con 0x7fdb3c06a170 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb41a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c101e80 con 0x7fdb3c108500 2024-06-13T06:59:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb41a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c102180 con 0x7fdb3c06ab30 2024-06-13T06:59:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb32000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb3aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06a170 0x7fdb3c10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb32000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35642/0 (socket says 172.21.15.7:35642) 2024-06-13T06:59:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.634+0000 7fdb32000700 1 -- 172.21.15.7:0/2554484695 learned_addr learned my addr 172.21.15.7:0/2554484695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.635+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c10beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.637+0000 7fdb3b400700 1 -- 172.21.15.7:0/2554484695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 msgr2=0x7fdb3c10b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.637+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.637+0000 7fdb3b400700 1 -- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06a170 msgr2=0x7fdb3c10b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.637+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06a170 0x7fdb3c10b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.637+0000 7fdb3b400700 1 -- 172.21.15.7:0/2554484695 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c1ab5b0 con 0x7fdb3c108500 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.638+0000 7fdb32000700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.638+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c10beb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b3f0 tx=0x7fdb3400b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.638+0000 7fdb33e00700 1 -- 172.21.15.7:0/2554484695 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34015020 con 0x7fdb3c108500 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.638+0000 7fdb33e00700 1 -- 172.21.15.7:0/2554484695 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb34011b10 con 0x7fdb3c108500 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.638+0000 7fdb33e00700 1 -- 172.21.15.7:0/2554484695 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34010be0 con 0x7fdb3c108500 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.639+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c1ab8a0 con 0x7fdb3c108500 2024-06-13T06:59:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.639+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c1abd60 con 0x7fdb3c108500 2024-06-13T06:59:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.646+0000 7fdb4c2dab80 1 Processor -- start 2024-06-13T06:59:44.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.648+0000 7fdb33e00700 1 -- 172.21.15.7:0/2554484695 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdb34018020 con 0x7fdb3c108500 2024-06-13T06:59:44.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.650+0000 7fdb33e00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 0x7fdb200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.651+0000 7fdb33e00700 1 -- 172.21.15.7:0/2554484695 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fdb340ce8d0 con 0x7fdb3c108500 2024-06-13T06:59:44.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.651+0000 7fdb4c2dab80 1 -- start start 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb32000700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 0x7fdb200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a41ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a4e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557082950a90 con 0x557082a4c380 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557082993b80 con 0x5570829bdec0 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.652+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55708294e970 con 0x557082a41630 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55804/0 (socket says 172.21.15.7:55804) 2024-06-13T06:59:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 -- 172.21.15.7:0/3656726535 learned_addr learned my addr 172.21.15.7:0/3656726535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 -- 172.21.15.7:0/3656726535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 msgr2=0x557082a41ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a41ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 -- 172.21.15.7:0/3656726535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 msgr2=0x557082a4e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a4e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.653+0000 7fdb3aa00700 1 -- 172.21.15.7:0/3656726535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708293c010 con 0x5570829bdec0 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.657+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011680 tx=0x7fdb2c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98bd32df77bb7b52 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.657+0000 7fdb2be00700 1 -- 172.21.15.7:0/3656726535 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c019030 con 0x5570829bdec0 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.657+0000 7fdb2be00700 1 -- 172.21.15.7:0/3656726535 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c0093a0 con 0x5570829bdec0 2024-06-13T06:59:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.657+0000 7fdb2be00700 1 -- 172.21.15.7:0/3656726535 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c007500 con 0x5570829bdec0 2024-06-13T06:59:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.661+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3656726535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 msgr2=0x557082a410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.661+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011680 tx=0x7fdb2c009920 comp rx=0 tx=0).stop 2024-06-13T06:59:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.661+0000 7fdb32000700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 0x7fdb200a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005e50 tx=0x7fdb24005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.661+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3656726535 shutdown_connections 2024-06-13T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.662+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a4e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.662+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a410f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.662+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3656726535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a41ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.662+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3656726535 >> 172.21.15.7:0/3656726535 conn(0x55708296e1f0 msgr2=0x557082a3d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.662+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3656726535 shutdown_connections 2024-06-13T06:59:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.663+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3656726535 wait complete. 2024-06-13T06:59:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.664+0000 7fdb4c2dab80 1 -- start start 2024-06-13T06:59:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.664+0000 7fdb4c2dab80 1 Processor -- start 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570829bdec0 0x557082a4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55708292ffe0 con 0x557082a4c380 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55708298fb40 con 0x557082a41630 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557082a4f0e0 con 0x5570829bdec0 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.665+0000 7fdb4c2dab80 10 client.0 _ll_register_callbacks cb 0x55708292d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.666+0000 7fdb32000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.666+0000 7fdb32000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55820/0 (socket says 172.21.15.7:55820) 2024-06-13T06:59:44.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.666+0000 7fdb32000700 1 -- 172.21.15.7:0/3220040333 learned_addr learned my addr 172.21.15.7:0/3220040333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.666+0000 7fdb32000700 1 -- 172.21.15.7:0/3220040333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570829bdec0 msgr2=0x557082a4bb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.667+0000 7fdb32000700 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570829bdec0 0x557082a4bb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.667+0000 7fdb32000700 1 -- 172.21.15.7:0/3220040333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 msgr2=0x557082a49380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.667+0000 7fdb32000700 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.667+0000 7fdb32000700 1 -- 172.21.15.7:0/3220040333 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570828e2950 con 0x557082a41630 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.667+0000 7fdb32000700 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005f50 tx=0x7fdb24009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa96b2388cc64102 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.668+0000 7fdb28c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.668+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400f870 con 0x557082a41630 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.668+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fdb2400fa10 con 0x557082a41630 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.668+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570829a1080 con 0x557082a41630 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.669+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb2400be60 con 0x557082a41630 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.669+0000 7fdb4c2dab80 10 client.25910 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.669+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557082a598b0 con 0x557082a41630 2024-06-13T06:59:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.669+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2400fc10 con 0x557082a41630 2024-06-13T06:59:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.672+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdb2402e070 con 0x557082a41630 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 10 client.25910 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 20 client.25910 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082a5bf90 0x557082a5e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 4 client.25910 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.673+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570829a1080 con 0x557082a5bf90 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.674+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082a5bf90 0x557082a5e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.674+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082a5bf90 0x557082a5e380 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa46f2b106b57604 server_cookie=6c23360546d4255 in_seq=0 out_seq=0 2024-06-13T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.674+0000 7fdb28c00700 10 client.25910 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.676+0000 7fdb28c00700 1 -- 172.21.15.7:0/3220040333 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570829a1080 con 0x557082a5bf90 2024-06-13T06:59:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.676+0000 7fdb28c00700 10 client.25910 handle_command_reply: tid=0 2024-06-13T06:59:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.676+0000 7fdb4c2dab80 1 client.25910 shutdown 2024-06-13T06:59:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 msgr2=0x557082a48e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005f50 tx=0x7fdb24009310 comp rx=0 tx=0).stop 2024-06-13T06:59:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 shutdown_connections 2024-06-13T06:59:44.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082a5bf90 0x557082a5e380 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a49380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570829bdec0 0x557082a4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb28c00700 0 client.25910 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/3220040333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a41630 0x557082a48e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.678+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 >> 172.21.15.7:0/3220040333 conn(0x55708296e1f0 msgr2=0x557082a3c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.679+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 shutdown_connections 2024-06-13T06:59:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.679+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/3220040333 wait complete. 2024-06-13T06:59:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.679+0000 7fdb4c2dab80 20 client.25910 trim_cache size 0 max 16384 2024-06-13T06:59:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 Processor -- start 2024-06-13T06:59:44.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 -- start start 2024-06-13T06:59:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 0x557082a3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a4b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b7130 con 0x5570829bdec0 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b49c0 con 0x557082a4c380 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.726+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b2810 con 0x557082a41630 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55830/0 (socket says 172.21.15.7:55830) 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 -- 172.21.15.7:0/2117007731 learned_addr learned my addr 172.21.15.7:0/2117007731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 0x557082a3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb32000700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a4b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 -- 172.21.15.7:0/2117007731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 msgr2=0x557082a4b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a4b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 -- 172.21.15.7:0/2117007731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 msgr2=0x557082a3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 0x557082a3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.727+0000 7fdb3b400700 1 -- 172.21.15.7:0/2117007731 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557082a42c50 con 0x557082a4c380 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.728+0000 7fdb32000700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.728+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 0x557082a3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.728+0000 7fdb3b400700 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400ba50 tx=0x7fdb3409b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c27c10c3b497e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.729+0000 7fdb2b400700 1 -- 172.21.15.7:0/2117007731 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb340a3a60 con 0x557082a4c380 2024-06-13T06:59:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.729+0000 7fdb2b400700 1 -- 172.21.15.7:0/2117007731 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3409e070 con 0x557082a4c380 2024-06-13T06:59:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.730+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/2117007731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 msgr2=0x557082a4d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.730+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400ba50 tx=0x7fdb3409b570 comp rx=0 tx=0).stop 2024-06-13T06:59:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/2117007731 shutdown_connections 2024-06-13T06:59:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557082a4c380 0x557082a4d770 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570829bdec0 0x557082a3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/2117007731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a4b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/2117007731 >> 172.21.15.7:0/2117007731 conn(0x55708296e1f0 msgr2=0x557082a3bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.731+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/2117007731 shutdown_connections 2024-06-13T06:59:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.732+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/2117007731 wait complete. 2024-06-13T06:59:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.732+0000 7fdb4c2dab80 1 -- start start 2024-06-13T06:59:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.732+0000 7fdb4c2dab80 1 Processor -- start 2024-06-13T06:59:44.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.733+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb3aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b2e80 con 0x557082a4c380 2024-06-13T06:59:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b42f0 con 0x5570829bdec0 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c1b3550 con 0x557082a41630 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb4c2dab80 10 client.0 _ll_register_callbacks cb 0x55708292d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb32000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb32000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35690/0 (socket says 172.21.15.7:35690) 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb32000700 1 -- 172.21.15.7:0/1822702619 learned_addr learned my addr 172.21.15.7:0/1822702619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.734+0000 7fdb3b400700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb32000700 1 -- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 msgr2=0x557082a54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb32000700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb32000700 1 -- 172.21.15.7:0/1822702619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 msgr2=0x557082a460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb32000700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb32000700 1 -- 172.21.15.7:0/1822702619 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557082a468e0 con 0x557082a41630 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.735+0000 7fdb3b400700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:44.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.736+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.736+0000 7fdb32000700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb240098e0 tx=0x7fdb24024f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d66ecbeaafb934 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.736+0000 7fdb28c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.736+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400fa80 con 0x557082a41630 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.736+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fdb2400fc20 con 0x557082a41630 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.737+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3409e070 con 0x557082a41630 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.737+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb24003c90 con 0x557082a41630 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.737+0000 7fdb4c2dab80 10 client.25885 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.737+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557082a63d50 con 0x557082a41630 2024-06-13T06:59:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.737+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3409e070 con 0x557082a41630 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.738+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdb2403c070 con 0x557082a41630 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 20 client.25885 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082ac6dd0 0x557082ac91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 4 client.25885 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T06:59:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.739+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55708299b600 con 0x557082ac6dd0 2024-06-13T06:59:44.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.742+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082ac6dd0 0x557082ac91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.744+0000 7fdb3aa00700 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082ac6dd0 0x557082ac91c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e141e4cdd5250518 server_cookie=64393c17c24d1457 in_seq=0 out_seq=0 2024-06-13T06:59:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.745+0000 7fdb28c00700 10 client.25885 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.745+0000 7fdb28c00700 1 -- 172.21.15.7:0/1822702619 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55708299b600 con 0x557082ac6dd0 2024-06-13T06:59:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.745+0000 7fdb28c00700 10 client.25885 handle_command_reply: tid=0 2024-06-13T06:59:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.746+0000 7fdb4c2dab80 1 client.25885 shutdown 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.748+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 msgr2=0x557082a54b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.748+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb240098e0 tx=0x7fdb24024f30 comp rx=0 tx=0).stop 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 shutdown_connections 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557082ac6dd0 0x557082ac91c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557082a4c380 0x557082a460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5570829bdec0 0x557082a54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 --2- 172.21.15.7:0/1822702619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557082a41630 0x557082a54b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 >> 172.21.15.7:0/1822702619 conn(0x55708296e1f0 msgr2=0x557082a3c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb28c00700 0 client.25885 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:44.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 shutdown_connections 2024-06-13T06:59:44.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.749+0000 7fdb4c2dab80 1 -- 172.21.15.7:0/1822702619 wait complete. 2024-06-13T06:59:44.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.750+0000 7fdb4c2dab80 20 client.25885 trim_cache size 0 max 16384 2024-06-13T06:59:44.751 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:59:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T06:59:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "26c5c870-c59a-438b-961d-558ef043c445", 2024-06-13T06:59:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T06:59:44.751 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:59:44.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.751+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 msgr2=0x7fdb200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.751+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 0x7fdb200a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005e50 tx=0x7fdb24005dc0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.751+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 msgr2=0x7fdb3c10beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:44.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.751+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c10beb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b3f0 tx=0x7fdb3400b700 comp rx=0 tx=0).stop 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 shutdown_connections 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fdb200a2cf0 0x7fdb200a51b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb3c108500 0x7fdb3c10beb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3c06a170 0x7fdb3c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 --2- 172.21.15.7:0/2554484695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb3c06ab30 0x7fdb3c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 >> 172.21.15.7:0/2554484695 conn(0x7fdb3c078340 msgr2=0x7fdb3c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 shutdown_connections 2024-06-13T06:59:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:44.753+0000 7fdb41a00700 1 -- 172.21.15.7:0/2554484695 wait complete. 2024-06-13T06:59:45.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:45 smithi189 ceph-mon[36616]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-06-13T06:59:45.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:45 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:45 smithi175 ceph-mon[36886]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-06-13T06:59:45.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:45 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:45.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:45 smithi007 ceph-mon[32077]: pgmap v620: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-06-13T06:59:45.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:45 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:45.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T06:59:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T06:59:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:46 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:46.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T06:59:46.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:46 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:46.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:46 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T06:59:47.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:47 smithi007 ceph-mon[32077]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-06-13T06:59:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:47 smithi189 ceph-mon[36616]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-06-13T06:59:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:47 smithi175 ceph-mon[36886]: pgmap v621: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 665 op/s 2024-06-13T06:59:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 225.0 s, 552.2 tps, lat 181.402 ms stddev 187.212 2024-06-13T06:59:49.036 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:49 smithi007 ceph-mon[32077]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 -- 172.21.15.7:0/435403204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964101d50 msgr2=0x7f796410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 --2- 172.21.15.7:0/435403204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964101d50 0x7f796410e240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f795c00b0a0 tx=0x7f795c00b3b0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 -- 172.21.15.7:0/435403204 shutdown_connections 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 --2- 172.21.15.7:0/435403204 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964101d50 0x7f796410e240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 --2- 172.21.15.7:0/435403204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7964107400 0x7f79641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 --2- 172.21.15.7:0/435403204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7964101390 0x7f7964101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 -- 172.21.15.7:0/435403204 >> 172.21.15.7:0/435403204 conn(0x7f7964078340 msgr2=0x7f7964078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 -- 172.21.15.7:0/435403204 shutdown_connections 2024-06-13T06:59:49.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.046+0000 7f796b400700 1 -- 172.21.15.7:0/435403204 wait complete. 2024-06-13T06:59:49.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.048+0000 7f796b400700 1 Processor -- start 2024-06-13T06:59:49.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.048+0000 7f796b400700 1 -- start start 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7964101390 0x7f796419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7964101d50 0x7f796419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7964110580 con 0x7f7964101390 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7964110400 con 0x7f7964107400 2024-06-13T06:59:49.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f796b400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7964110700 con 0x7f7964101d50 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f7969400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f7969400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55850/0 (socket says 172.21.15.7:55850) 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.049+0000 7f7969400700 1 -- 172.21.15.7:0/902588657 learned_addr learned my addr 172.21.15.7:0/902588657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.050+0000 7f7969400700 1 -- 172.21.15.7:0/902588657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7964101d50 msgr2=0x7f796419cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.050+0000 7f7969400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7964101d50 0x7f796419cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.050+0000 7f7969400700 1 -- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7964101390 msgr2=0x7f796419c670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T06:59:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.052+0000 7f7969400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7964101390 0x7f796419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.052+0000 7f7969400700 1 -- 172.21.15.7:0/902588657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7964197020 con 0x7f7964107400 2024-06-13T06:59:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.052+0000 7f7969400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f795c0062a0 tx=0x7f795c004e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.053+0000 7f7961600700 1 -- 172.21.15.7:0/902588657 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c021070 con 0x7f7964107400 2024-06-13T06:59:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.053+0000 7f7961600700 1 -- 172.21.15.7:0/902588657 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f795c007c80 con 0x7f7964107400 2024-06-13T06:59:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.053+0000 7f7961600700 1 -- 172.21.15.7:0/902588657 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c01d450 con 0x7f7964107400 2024-06-13T06:59:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.053+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641972b0 con 0x7f7964107400 2024-06-13T06:59:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.054+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a78e0 con 0x7f7964107400 2024-06-13T06:59:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.059+0000 7f7961600700 1 -- 172.21.15.7:0/902588657 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f795c004000 con 0x7f7964107400 2024-06-13T06:59:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.061+0000 7f7961600700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 0x7f794c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.061+0000 7f7961600700 1 -- 172.21.15.7:0/902588657 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f795c0cfd50 con 0x7f7964107400 2024-06-13T06:59:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.062+0000 7f7975cbab80 1 Processor -- start 2024-06-13T06:59:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.062+0000 7f7975cbab80 1 -- start start 2024-06-13T06:59:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.062+0000 7f7968a00700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 0x7f794c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 0x560a64829880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a64836f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560a64730a10 con 0x560a64834b10 2024-06-13T06:59:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a64773b00 con 0x560a64829dc0 2024-06-13T06:59:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.063+0000 7f7975cbab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a6472e8f0 con 0x560a6479de40 2024-06-13T06:59:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7963e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7963e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55854/0 (socket says 172.21.15.7:55854) 2024-06-13T06:59:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7963e00700 1 -- 172.21.15.7:0/2875876985 learned_addr learned my addr 172.21.15.7:0/2875876985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7969400700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a64836f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7968a00700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 0x560a64829880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.064+0000 7f7968a00700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 0x7f794c0a5290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7958005e90 tx=0x7f7958005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7968a00700 1 -- 172.21.15.7:0/2875876985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 msgr2=0x560a6482a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7968a00700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7968a00700 1 -- 172.21.15.7:0/2875876985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 msgr2=0x560a64836f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7968a00700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a64836f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7968a00700 1 -- 172.21.15.7:0/2875876985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6471bf90 con 0x560a6479de40 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7969400700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a64836f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.065+0000 7f7963e00700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.066+0000 7f7968a00700 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 0x560a64829880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f795800d5c0 tx=0x7f7958010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8a91d886b095bc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.066+0000 7f7952000700 1 -- 172.21.15.7:0/2875876985 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580108f0 con 0x560a6479de40 2024-06-13T06:59:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.067+0000 7f7952000700 1 -- 172.21.15.7:0/2875876985 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958016030 con 0x560a6479de40 2024-06-13T06:59:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.067+0000 7f7952000700 1 -- 172.21.15.7:0/2875876985 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958005280 con 0x560a6479de40 2024-06-13T06:59:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 -- 172.21.15.7:0/2875876985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 msgr2=0x560a64829880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 0x560a64829880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f795800d5c0 tx=0x7f7958010000 comp rx=0 tx=0).stop 2024-06-13T06:59:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 -- 172.21.15.7:0/2875876985 shutdown_connections 2024-06-13T06:59:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a64836f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a6479de40 0x560a64829880 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2875876985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64829dc0 0x560a6482a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.068+0000 7f7975cbab80 1 -- 172.21.15.7:0/2875876985 >> 172.21.15.7:0/2875876985 conn(0x560a6474e170 msgr2=0x560a6481d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.070+0000 7f7975cbab80 1 -- 172.21.15.7:0/2875876985 shutdown_connections 2024-06-13T06:59:49.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.070+0000 7f7975cbab80 1 -- 172.21.15.7:0/2875876985 wait complete. 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.070+0000 7f9814200640 1 -- 172.21.15.7:0/2336045516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c1057e0 msgr2=0x7f980c10c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.070+0000 7f9814200640 1 --2- 172.21.15.7:0/2336045516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c1057e0 0x7f980c10c070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9804009860 tx=0x7f980402a340 comp rx=0 tx=0).stop 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 -- 172.21.15.7:0/2336045516 shutdown_connections 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 --2- 172.21.15.7:0/2336045516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c1057e0 0x7f980c10c070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 --2- 172.21.15.7:0/2336045516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c104e20 0x7f980c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 --2- 172.21.15.7:0/2336045516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c103c30 0x7f980c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 -- 172.21.15.7:0/2336045516 >> 172.21.15.7:0/2336045516 conn(0x7f980c0ff1c0 msgr2=0x7f980c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f9814200640 1 -- 172.21.15.7:0/2336045516 shutdown_connections 2024-06-13T06:59:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.073+0000 7f9814200640 1 -- 172.21.15.7:0/2336045516 wait complete. 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.071+0000 7f7975cbab80 1 -- start start 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.071+0000 7f7975cbab80 1 Processor -- start 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.071+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64834240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64829dc0 0x560a64831540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560a6470ff60 con 0x560a64829dc0 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560a6476fac0 con 0x560a6479de40 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a64837770 con 0x560a64834b10 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35754/0 (socket says 172.21.15.7:35754) 2024-06-13T06:59:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 -- 172.21.15.7:0/2814435823 learned_addr learned my addr 172.21.15.7:0/2814435823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7975cbab80 10 client.0 _ll_register_callbacks cb 0x560a6470d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 -- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 msgr2=0x560a64834240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64834240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 -- 172.21.15.7:0/2814435823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64829dc0 msgr2=0x560a64831540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64829dc0 0x560a64831540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.072+0000 7f7969400700 1 -- 172.21.15.7:0/2814435823 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a646c28d0 con 0x560a64834b10 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.073+0000 7f7969400700 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f795c09c3c0 tx=0x7f795c007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcf6abb55943593 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.073+0000 7f7946a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.074+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c09c430 con 0x560a64834b10 2024-06-13T06:59:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.074+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f795c0a4b80 con 0x560a64834b10 2024-06-13T06:59:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.074+0000 7f9814200640 1 Processor -- start 2024-06-13T06:59:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.074+0000 7f9814200640 1 -- start start 2024-06-13T06:59:49.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7958016030 con 0x560a64834b10 2024-06-13T06:59:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f9814200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f9814200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f9814200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 0x7f980c1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f9810e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f795c0a07e0 con 0x560a64834b10 2024-06-13T06:59:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f7975cbab80 10 client.25897 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a64838070 con 0x560a64834b10 2024-06-13T06:59:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9810e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55874/0 (socket says 172.21.15.7:55874) 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9811800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9814200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f980c076be0 con 0x7f980c1057e0 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9814200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f980c076a60 con 0x7f980c104e20 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9814200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f980c076d60 con 0x7f980c103c30 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9811800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35770/0 (socket says 172.21.15.7:35770) 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9811800640 1 -- 172.21.15.7:0/2579059450 learned_addr learned my addr 172.21.15.7:0/2579059450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 0x7f980c1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 -- 172.21.15.7:0/2579059450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 msgr2=0x7f980c110d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 -- 172.21.15.7:0/2579059450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 msgr2=0x7f980c111280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 -- 172.21.15.7:0/2579059450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c116950 con 0x7f980c1057e0 2024-06-13T06:59:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9810e00640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9811800640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.075+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f795c0a5010 con 0x560a64834b10 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f795c011070 con 0x560a64834b10 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 10 client.25897 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 20 client.25897 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a6483a600 0x560a6483ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 4 client.25897 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:49.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.076+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7958016030 con 0x560a6483a600 2024-06-13T06:59:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 0x7f980c1162e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f980402a860 tx=0x7f9804033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.077+0000 7f7968a00700 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a6483a600 0x560a6483ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.078+0000 7f980a000640 1 -- 172.21.15.7:0/2579059450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804037040 con 0x7f980c1057e0 2024-06-13T06:59:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.078+0000 7f7968a00700 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a6483a600 0x560a6483ca80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c37370bc943fa4 server_cookie=ed6670b38b4c3409 in_seq=0 out_seq=0 2024-06-13T06:59:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.078+0000 7f7946a00700 10 client.25897 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.078+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c116be0 con 0x7f980c1057e0 2024-06-13T06:59:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.078+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c1170f0 con 0x7f980c1057e0 2024-06-13T06:59:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.079+0000 7f980a000640 1 -- 172.21.15.7:0/2579059450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9804004050 con 0x7f980c1057e0 2024-06-13T06:59:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.079+0000 7f980a000640 1 -- 172.21.15.7:0/2579059450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804037040 con 0x7f980c1057e0 2024-06-13T06:59:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7946a00700 1 -- 172.21.15.7:0/2814435823 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f7958016030 con 0x560a6483a600 2024-06-13T06:59:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7946a00700 10 client.25897 handle_command_reply: tid=0 2024-06-13T06:59:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 client.25897 shutdown 2024-06-13T06:59:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 msgr2=0x560a64831a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f795c09c3c0 tx=0x7f795c007e30 comp rx=0 tx=0).stop 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 shutdown_connections 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a6483a600 0x560a6483ca80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64834b10 0x560a64831a80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64834240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7946a00700 0 client.25897 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 --2- 172.21.15.7:0/2814435823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64829dc0 0x560a64831540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.080+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 >> 172.21.15.7:0/2814435823 conn(0x560a6474e170 msgr2=0x560a6481c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.081+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 shutdown_connections 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.081+0000 7f7975cbab80 1 -- 172.21.15.7:0/2814435823 wait complete. 2024-06-13T06:59:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.081+0000 7f7975cbab80 20 client.25897 trim_cache size 0 max 16384 2024-06-13T06:59:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.083+0000 7f98171c1740 1 Processor -- start 2024-06-13T06:59:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.084+0000 7f980a000640 1 -- 172.21.15.7:0/2579059450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9804042030 con 0x7f980c1057e0 2024-06-13T06:59:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 -- start start 2024-06-13T06:59:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a034020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a0390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f29f97420 con 0x559f2a039760 2024-06-13T06:59:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f29f3ec50 con 0x559f2a0345f0 2024-06-13T06:59:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.085+0000 7f98171c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f29f933e0 con 0x559f2a033c00 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f9810e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a034020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f980a000640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 0x7f97f40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f9811800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a0390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f9810e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a034020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35778/0 (socket says 172.21.15.7:35778) 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f9810e00640 1 -- 172.21.15.7:0/582560384 learned_addr learned my addr 172.21.15.7:0/582560384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f9810e00640 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.086+0000 7f980a000640 1 -- 172.21.15.7:0/2579059450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9804040020 con 0x7f980c1057e0 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 0x7f97f40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9810e00640 1 -- 172.21.15.7:0/582560384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 msgr2=0x559f2a034020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9810e00640 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a034020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9810e00640 1 -- 172.21.15.7:0/582560384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 msgr2=0x559f2a0390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9810e00640 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a0390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9810e00640 1 -- 172.21.15.7:0/582560384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f29cab210 con 0x559f2a039760 2024-06-13T06:59:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.087+0000 7f9812200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 0x7f97f40a54f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9804031000 tx=0x7f9804046000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.088+0000 7f9810e00640 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a03bbf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x559f2a034e50 tx=0x7f97f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48297da0174880de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.089+0000 7f97fea00640 1 -- 172.21.15.7:0/582560384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8014310 con 0x559f2a039760 2024-06-13T06:59:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.089+0000 7f97fea00640 1 -- 172.21.15.7:0/582560384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8005020 con 0x559f2a039760 2024-06-13T06:59:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.089+0000 7f97fea00640 1 -- 172.21.15.7:0/582560384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8004b00 con 0x559f2a039760 2024-06-13T06:59:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.090+0000 7f98171c1740 1 -- 172.21.15.7:0/582560384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 msgr2=0x559f2a03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.090+0000 7f98171c1740 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a03bbf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x559f2a034e50 tx=0x7f97f8009260 comp rx=0 tx=0).stop 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.091+0000 7f98171c1740 1 -- 172.21.15.7:0/582560384 shutdown_connections 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.091+0000 7f98171c1740 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a03bbf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.091+0000 7f98171c1740 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a0390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.091+0000 7f98171c1740 1 --2- 172.21.15.7:0/582560384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a034020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.092+0000 7f98171c1740 1 -- 172.21.15.7:0/582560384 >> 172.21.15.7:0/582560384 conn(0x559f29ffbfb0 msgr2=0x559f2a02f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.092+0000 7f98171c1740 1 -- 172.21.15.7:0/582560384 shutdown_connections 2024-06-13T06:59:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.092+0000 7f98171c1740 1 -- 172.21.15.7:0/582560384 wait complete. 2024-06-13T06:59:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.093+0000 7f98171c1740 1 -- start start 2024-06-13T06:59:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.093+0000 7f98171c1740 1 Processor -- start 2024-06-13T06:59:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.094+0000 7f98171c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a03dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.095+0000 7f98171c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a03e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.095+0000 7f98171c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a045520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f98171c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f2a03c710 con 0x559f2a033c00 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f98171c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559f29f520b0 con 0x559f2a039760 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f98171c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f2a03c890 con 0x559f2a0345f0 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f98171c1740 10 client.0 _ll_register_callbacks cb 0x559f29f3df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f9810e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a03dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f9810e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a03dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37396/0 (socket says 172.21.15.7:37396) 2024-06-13T06:59:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f9810e00640 1 -- 172.21.15.7:0/162056963 learned_addr learned my addr 172.21.15.7:0/162056963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.096+0000 7f9811800640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a03e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9810e00640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a045520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9811800640 1 -- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 msgr2=0x559f2a045520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9811800640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a045520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9811800640 1 -- 172.21.15.7:0/162056963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 msgr2=0x559f2a03dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9811800640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a03dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.097+0000 7f9811800640 1 -- 172.21.15.7:0/162056963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f29d77620 con 0x559f2a0345f0 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.098+0000 7f9810e00640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a045520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.098+0000 7f9811800640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a03e4e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x559f2a0349f0 tx=0x7f9800012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57f2e894b973bbb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.098+0000 7f97f3400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.099+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800004d20 con 0x559f2a0345f0 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.099+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9800010b30 con 0x559f2a0345f0 2024-06-13T06:59:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.099+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f8005020 con 0x559f2a0345f0 2024-06-13T06:59:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.100+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9800010d30 con 0x559f2a0345f0 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.100+0000 7f98171c1740 10 client.25903 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.100+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f2a048990 con 0x559f2a0345f0 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.100+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8005020 con 0x559f2a0345f0 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.100+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9800019070 con 0x559f2a0345f0 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 10 client.25903 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 20 client.25903 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a04a0f0 0x559f2a04c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 4 client.25903 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T06:59:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f29fa4b50 con 0x559f2a04a0f0 2024-06-13T06:59:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.101+0000 7f9810e00640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a04a0f0 0x559f2a04c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.102+0000 7f9810e00640 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a04a0f0 0x559f2a04c520 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eff0b539885b424 server_cookie=b7325660b6c36425 in_seq=0 out_seq=0 2024-06-13T06:59:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.102+0000 7f97f3400640 10 client.25903 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.103+0000 7f97f3400640 1 -- 172.21.15.7:0/162056963 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f29fa4b50 con 0x559f2a04a0f0 2024-06-13T06:59:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.103+0000 7f97f3400640 10 client.25903 handle_command_reply: tid=0 2024-06-13T06:59:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.103+0000 7f98171c1740 1 client.25903 shutdown 2024-06-13T06:59:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 msgr2=0x559f2a03e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a03e4e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x559f2a0349f0 tx=0x7f9800012b60 comp rx=0 tx=0).stop 2024-06-13T06:59:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 shutdown_connections 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a04a0f0 0x559f2a04c520 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a045520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a03e4e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 --2- 172.21.15.7:0/162056963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a03dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 >> 172.21.15.7:0/162056963 conn(0x559f29ffbfb0 msgr2=0x559f2a02e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f97f3400640 0 client.25903 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.104+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 shutdown_connections 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.105+0000 7f98171c1740 1 -- 172.21.15.7:0/162056963 wait complete. 2024-06-13T06:59:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.105+0000 7f98171c1740 20 client.25903 trim_cache size 0 max 16384 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 Processor -- start 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 -- start start 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64833d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a6482c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a6482cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7964072c00 con 0x560a64834b10 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7964070490 con 0x560a6479de40 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.115+0000 7f7975cbab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f796406e2e0 con 0x560a64829dc0 2024-06-13T06:59:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.116+0000 7f7968a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64833d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.116+0000 7f7963e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a6482c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.116+0000 7f7963e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a6482c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35808/0 (socket says 172.21.15.7:35808) 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.116+0000 7f7963e00700 1 -- 172.21.15.7:0/647523401 learned_addr learned my addr 172.21.15.7:0/647523401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.116+0000 7f7969400700 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a6482cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 -- 172.21.15.7:0/647523401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 msgr2=0x560a6482c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a6482c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 -- 172.21.15.7:0/647523401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 msgr2=0x560a6482cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a6482cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 -- 172.21.15.7:0/647523401 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6482a7e0 con 0x560a6479de40 2024-06-13T06:59:49.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7968a00700 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64833d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560a6483d340 tx=0x7f795800d7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eca12074da80ba server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.117+0000 7f7951600700 1 -- 172.21.15.7:0/647523401 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958004020 con 0x560a6479de40 2024-06-13T06:59:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.118+0000 7f7951600700 1 -- 172.21.15.7:0/647523401 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958016040 con 0x560a6479de40 2024-06-13T06:59:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.118+0000 7f7951600700 1 -- 172.21.15.7:0/647523401 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580038e0 con 0x560a6479de40 2024-06-13T06:59:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.119+0000 7f7975cbab80 1 -- 172.21.15.7:0/647523401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 msgr2=0x560a64833d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.119+0000 7f7975cbab80 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64833d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560a6483d340 tx=0x7f795800d7d0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 -- 172.21.15.7:0/647523401 shutdown_connections 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a64834b10 0x560a6482cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a6479de40 0x560a64833d80 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x560a6483d340 tx=0x7f795800d7d0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 --2- 172.21.15.7:0/647523401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a6482c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 -- 172.21.15.7:0/647523401 >> 172.21.15.7:0/647523401 conn(0x560a6474e170 msgr2=0x560a6481cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.121+0000 7f7975cbab80 1 -- 172.21.15.7:0/647523401 shutdown_connections 2024-06-13T06:59:49.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.122+0000 7f7975cbab80 1 -- 172.21.15.7:0/647523401 wait complete. 2024-06-13T06:59:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.122+0000 7f7975cbab80 1 -- start start 2024-06-13T06:59:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.122+0000 7f7975cbab80 1 Processor -- start 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.123+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a64842a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.123+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 0x560a6483ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.123+0000 7f7975cbab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a6483f230 0x560a6483f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7975cbab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f796406e950 con 0x560a6483f230 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7975cbab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f796406fdc0 con 0x560a64834b10 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7975cbab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f796406f020 con 0x560a64829dc0 2024-06-13T06:59:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7975cbab80 10 client.0 _ll_register_callbacks cb 0x560a6470d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7968a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a64842a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7968a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a64842a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35824/0 (socket says 172.21.15.7:35824) 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7968a00700 1 -- 172.21.15.7:0/994186559 learned_addr learned my addr 172.21.15.7:0/994186559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 0x560a6483ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7969400700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a6483f230 0x560a6483f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 -- 172.21.15.7:0/994186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 msgr2=0x560a64842a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a64842a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 -- 172.21.15.7:0/994186559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a6483f230 msgr2=0x560a6483f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a6483f230 0x560a6483f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.124+0000 7f7963e00700 1 -- 172.21.15.7:0/994186559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a648ae9e0 con 0x560a64834b10 2024-06-13T06:59:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.125+0000 7f7963e00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 0x560a6483ecf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x560a64851c60 tx=0x7f7954012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb85be1bc4f467b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.125+0000 7f7946a00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.125+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954010930 con 0x560a64834b10 2024-06-13T06:59:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.125+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7958016040 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.125+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7954010ad0 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.126+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7954010cd0 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.126+0000 7f7975cbab80 10 client.25937 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.126+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a648afd90 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.126+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958016040 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7954019070 con 0x560a64834b10 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 10 client.25937 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 20 client.25937 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a648b0470 0x560a648b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 4 client.25937 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T06:59:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.127+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a64781000 con 0x560a648b0470 2024-06-13T06:59:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.128+0000 7f7968a00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a648b0470 0x560a648b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.129+0000 7f7968a00700 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a648b0470 0x560a648b2860 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f2fa270640a586 server_cookie=b11ee011ff98be13 in_seq=0 out_seq=0 2024-06-13T06:59:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.129+0000 7f7946a00700 10 client.25937 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.129+0000 7f7946a00700 1 -- 172.21.15.7:0/994186559 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a64781000 con 0x560a648b0470 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.129+0000 7f7946a00700 10 client.25937 handle_command_reply: tid=0 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.129+0000 7f7975cbab80 1 client.25937 shutdown 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 msgr2=0x560a6483ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 0x560a6483ecf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x560a64851c60 tx=0x7f7954012730 comp rx=0 tx=0).stop 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 shutdown_connections 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560a648b0470 0x560a648b2860 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560a6483f230 0x560a6483f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a64829dc0 0x560a64842a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 --2- 172.21.15.7:0/994186559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560a64834b10 0x560a6483ecf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 >> 172.21.15.7:0/994186559 conn(0x560a6474e170 msgr2=0x560a6482fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7946a00700 0 client.25937 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 shutdown_connections 2024-06-13T06:59:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 1 -- 172.21.15.7:0/994186559 wait complete. 2024-06-13T06:59:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.130+0000 7f7975cbab80 20 client.25937 trim_cache size 0 max 16384 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "26c5c870-c59a-438b-961d-558ef043c445": { 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "26c5c870-c59a-438b-961d-558ef043c445", 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T06:59:49.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.131+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 msgr2=0x7f794c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.131+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 0x7f794c0a5290 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7958005e90 tx=0x7f7958005d00 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.131+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 msgr2=0x7f7964196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.131+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f795c0062a0 tx=0x7f795c004e20 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.131+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 shutdown_connections 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f794c0a2dd0 0x7f794c0a5290 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7964107400 0x7f7964196840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7964101390 0x7f796419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 --2- 172.21.15.7:0/902588657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7964101d50 0x7f796419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 >> 172.21.15.7:0/902588657 conn(0x7f7964078340 msgr2=0x7f79640ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 shutdown_connections 2024-06-13T06:59:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T06:59:49.132+0000 7f796b400700 1 -- 172.21.15.7:0/902588657 wait complete. 2024-06-13T06:59:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.138+0000 7f98171c1740 1 Processor -- start 2024-06-13T06:59:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 -- start start 2024-06-13T06:59:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a02f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a03dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec012310 con 0x559f2a033c00 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00fbe0 con 0x559f2a039760 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.139+0000 7f98171c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00da30 con 0x559f2a0345f0 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9810e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a02f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9810e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a02f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37442/0 (socket says 172.21.15.7:37442) 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9810e00640 1 -- 172.21.15.7:0/3039508955 learned_addr learned my addr 172.21.15.7:0/3039508955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35840/0 (socket says 172.21.15.7:35840) 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9810e00640 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a03dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 -- 172.21.15.7:0/3039508955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 msgr2=0x559f2a03dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a03dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 -- 172.21.15.7:0/3039508955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 msgr2=0x559f2a02f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a02f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.140+0000 7f9811800640 1 -- 172.21.15.7:0/3039508955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2a02f640 con 0x559f2a0345f0 2024-06-13T06:59:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.141+0000 7f9811800640 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f980001a7f0 tx=0x7f9800005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c530a420e43188f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.142+0000 7f97fe000640 1 -- 172.21.15.7:0/3039508955 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980001a3f0 con 0x559f2a0345f0 2024-06-13T06:59:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.142+0000 7f97fe000640 1 -- 172.21.15.7:0/3039508955 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800029070 con 0x559f2a0345f0 2024-06-13T06:59:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.142+0000 7f97fe000640 1 -- 172.21.15.7:0/3039508955 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000053b0 con 0x559f2a0345f0 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 -- 172.21.15.7:0/3039508955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 msgr2=0x559f2a0454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f980001a7f0 tx=0x7f9800005a90 comp rx=0 tx=0).stop 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 -- 172.21.15.7:0/3039508955 shutdown_connections 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a039760 0x559f2a03dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a0345f0 0x559f2a0454e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 --2- 172.21.15.7:0/3039508955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a033c00 0x559f2a02f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 -- 172.21.15.7:0/3039508955 >> 172.21.15.7:0/3039508955 conn(0x559f29ffbfb0 msgr2=0x559f2a038f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 -- 172.21.15.7:0/3039508955 shutdown_connections 2024-06-13T06:59:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.143+0000 7f98171c1740 1 -- 172.21.15.7:0/3039508955 wait complete. 2024-06-13T06:59:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 -- start start 2024-06-13T06:59:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 Processor -- start 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a041fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00e0a0 con 0x559f2a039760 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00f510 con 0x559f2a0345f0 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00e770 con 0x559f2a033c00 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.144+0000 7f98171c1740 10 client.0 _ll_register_callbacks cb 0x7f9800009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.145+0000 7f9811800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.145+0000 7f9811800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55934/0 (socket says 172.21.15.7:55934) 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.145+0000 7f9811800640 1 -- 172.21.15.7:0/4178584877 learned_addr learned my addr 172.21.15.7:0/4178584877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T06:59:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.145+0000 7f9810e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a041fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.145+0000 7f9810e00640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9811800640 1 -- 172.21.15.7:0/4178584877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 msgr2=0x559f2a041fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9811800640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a041fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9811800640 1 -- 172.21.15.7:0/4178584877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 msgr2=0x559f2a0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T06:59:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9811800640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a0b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9811800640 1 -- 172.21.15.7:0/4178584877 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2a0455e0 con 0x559f2a0345f0 2024-06-13T06:59:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.146+0000 7f9810e00640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a041fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T06:59:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.147+0000 7f9811800640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f980001a890 tx=0x7f980001abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df9e345e989dad1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T06:59:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.147+0000 7f97f3400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T06:59:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.148+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800010b30 con 0x559f2a0345f0 2024-06-13T06:59:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.148+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9800004c10 con 0x559f2a0345f0 2024-06-13T06:59:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.149+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9800029070 con 0x559f2a0345f0 2024-06-13T06:59:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.150+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9800004e30 con 0x559f2a0345f0 2024-06-13T06:59:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.152+0000 7f98171c1740 10 client.25943 fetch_fsmap learned FSMap version 32 2024-06-13T06:59:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.152+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f2a0b4200 con 0x559f2a0345f0 2024-06-13T06:59:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.153+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800029070 con 0x559f2a0345f0 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f9800019070 con 0x559f2a0345f0 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 10 client.25943 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 20 client.25943 populate_metadata read hostname 'smithi007' 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a0b4b10 0x559f2a0b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 4 client.25943 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T06:59:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.154+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f29f9eea0 con 0x559f2a0b4b10 2024-06-13T06:59:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.155+0000 7f9810e00640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a0b4b10 0x559f2a0b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T06:59:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.156+0000 7f9810e00640 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a0b4b10 0x559f2a0b6f30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871f3a5c09e648bd server_cookie=174a0c091f906f24 in_seq=0 out_seq=0 2024-06-13T06:59:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.156+0000 7f97f3400640 10 client.25943 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.173+0000 7f97f3400640 1 -- 172.21.15.7:0/4178584877 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559f29f9eea0 con 0x559f2a0b4b10 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.173+0000 7f97f3400640 10 client.25943 handle_command_reply: tid=0 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 client.25943 shutdown 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 msgr2=0x559f2a042510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f980001a890 tx=0x7f980001abc0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 shutdown_connections 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559f2a0b4b10 0x559f2a0b6f30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f2a039760 0x559f2a0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559f2a0345f0 0x559f2a042510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 --2- 172.21.15.7:0/4178584877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f2a033c00 0x559f2a041fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 >> 172.21.15.7:0/4178584877 conn(0x559f29ffbfb0 msgr2=0x559f2a0358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f97f3400640 0 client.25943 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 shutdown_connections 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.174+0000 7f98171c1740 1 -- 172.21.15.7:0/4178584877 wait complete. 2024-06-13T06:59:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.175+0000 7f98171c1740 20 client.25943 trim_cache size 0 max 16384 2024-06-13T06:59:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T06:59:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T06:59:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 msgr2=0x7f97f40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 0x7f97f40a54f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9804031000 tx=0x7f9804046000 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 msgr2=0x7f980c1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 0x7f980c1162e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f980402a860 tx=0x7f9804033d10 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 shutdown_connections 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97f40a3030 0x7f97f40a54f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980c1057e0 0x7f980c1162e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f980c104e20 0x7f980c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 --2- 172.21.15.7:0/2579059450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f980c103c30 0x7f980c110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.176+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 >> 172.21.15.7:0/2579059450 conn(0x7f980c0ff1c0 msgr2=0x7f980c079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.177+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 shutdown_connections 2024-06-13T06:59:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T06:59:49.177+0000 7f9814200640 1 -- 172.21.15.7:0/2579059450 wait complete. 2024-06-13T06:59:49.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:49 smithi189 ceph-mon[36616]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-06-13T06:59:49.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:49 smithi175 ceph-mon[36886]: pgmap v622: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.6 MiB/s wr, 777 op/s 2024-06-13T06:59:49.955 INFO:tasks.cephfs.filesystem:scrub status for tag:26c5c870-c59a-438b-961d-558ef043c445 - {'path': '/', 'tag': '26c5c870-c59a-438b-961d-558ef043c445', 'options': 'recursive,force'} 2024-06-13T06:59:51.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:51 smithi189 ceph-mon[36616]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 716 op/s 2024-06-13T06:59:51.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:51 smithi175 ceph-mon[36886]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 716 op/s 2024-06-13T06:59:51.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:51 smithi007 ceph-mon[32077]: pgmap v623: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 716 op/s 2024-06-13T06:59:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 230.0 s, 577.2 tps, lat 174.259 ms stddev 239.273 2024-06-13T06:59:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:53 smithi189 ceph-mon[36616]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-06-13T06:59:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:53 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:59:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:53 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:59:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:53 smithi175 ceph-mon[36886]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-06-13T06:59:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:53 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:59:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:53 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:59:53.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:53 smithi007 ceph-mon[32077]: pgmap v624: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.9 MiB/s wr, 707 op/s 2024-06-13T06:59:53.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:53 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:59:53.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:53 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:59:54.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:54 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T06:59:54.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:54 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T06:59:54.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:54 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T06:59:55.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:55 smithi189 ceph-mon[36616]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 762 op/s 2024-06-13T06:59:55.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:55 smithi175 ceph-mon[36886]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 762 op/s 2024-06-13T06:59:55.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:55 smithi007 ceph-mon[32077]: pgmap v625: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.4 MiB/s wr, 762 op/s 2024-06-13T06:59:57.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:57 smithi189 ceph-mon[36616]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 704 op/s 2024-06-13T06:59:57.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:57 smithi175 ceph-mon[36886]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 704 op/s 2024-06-13T06:59:57.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:57 smithi007 ceph-mon[32077]: pgmap v626: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 704 op/s 2024-06-13T06:59:57.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 235.0 s, 622.8 tps, lat 159.681 ms stddev 175.970 2024-06-13T06:59:59.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 06:59:59 smithi175 ceph-mon[36886]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.9 MiB/s wr, 706 op/s 2024-06-13T06:59:59.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 06:59:59 smithi007 ceph-mon[32077]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.9 MiB/s wr, 706 op/s 2024-06-13T06:59:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 06:59:59 smithi189 ceph-mon[36616]: pgmap v627: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.9 MiB/s wr, 706 op/s 2024-06-13T07:00:00.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:00 smithi175 ceph-mon[36886]: overall HEALTH_OK 2024-06-13T07:00:00.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:00 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T07:00:00.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:00 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T07:00:01.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:01 smithi175 ceph-mon[36886]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-06-13T07:00:01.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:01 smithi007 ceph-mon[32077]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-06-13T07:00:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:01 smithi189 ceph-mon[36616]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-06-13T07:00:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.379+0000 7f4afe000640 1 -- 172.21.15.7:0/1283456948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 msgr2=0x7f4af8114550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.379+0000 7f4afe000640 1 --2- 172.21.15.7:0/1283456948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af8114550 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009900 tx=0x7f4af002a410 comp rx=0 tx=0).stop 2024-06-13T07:00:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 -- 172.21.15.7:0/1283456948 shutdown_connections 2024-06-13T07:00:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 --2- 172.21.15.7:0/1283456948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af8114550 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 --2- 172.21.15.7:0/1283456948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806bb50 0x7f4af806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 --2- 172.21.15.7:0/1283456948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806d130 0x7f4af806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 -- 172.21.15.7:0/1283456948 >> 172.21.15.7:0/1283456948 conn(0x7f4af80709c0 msgr2=0x7f4af8072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 -- 172.21.15.7:0/1283456948 shutdown_connections 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.380+0000 7f4afe000640 1 -- 172.21.15.7:0/1283456948 wait complete. 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.381+0000 7f4afe000640 1 Processor -- start 2024-06-13T07:00:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.381+0000 7f4afe000640 1 -- start start 2024-06-13T07:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806bb50 0x7f4af81a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 0x7f4af81a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af81a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81180e0 con 0x7f4af806bb50 2024-06-13T07:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8117f60 con 0x7f4af806d130 2024-06-13T07:00:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.382+0000 7f4afe000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8118260 con 0x7f4af81120c0 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.383+0000 7f4af6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 0x7f4af81a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.383+0000 7f4af7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806bb50 0x7f4af81a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.383+0000 7f4af6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 0x7f4af81a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44294/0 (socket says 172.21.15.7:44294) 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.383+0000 7f4af6a00640 1 -- 172.21.15.7:0/4163900778 learned_addr learned my addr 172.21.15.7:0/4163900778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.383+0000 7f4af7e00640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af81a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 -- 172.21.15.7:0/4163900778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 msgr2=0x7f4af81a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 0x7f4af81a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 -- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806bb50 msgr2=0x7f4af81a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806bb50 0x7f4af81a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 -- 172.21.15.7:0/4163900778 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af81aa200 con 0x7f4af81120c0 2024-06-13T07:00:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.384+0000 7f4af7e00640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af81a9b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009850 tx=0x7f4af0032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.385+0000 7f4aefe00640 1 -- 172.21.15.7:0/4163900778 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af003f030 con 0x7f4af81120c0 2024-06-13T07:00:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.385+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af8109bb0 con 0x7f4af81120c0 2024-06-13T07:00:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.386+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af810a170 con 0x7f4af81120c0 2024-06-13T07:00:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.386+0000 7f4aefe00640 1 -- 172.21.15.7:0/4163900778 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4af0003ce0 con 0x7f4af81120c0 2024-06-13T07:00:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.386+0000 7f4aefe00640 1 -- 172.21.15.7:0/4163900778 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0035070 con 0x7f4af81120c0 2024-06-13T07:00:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.393+0000 7f4aefe00640 1 -- 172.21.15.7:0/4163900778 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4af0042030 con 0x7f4af81120c0 2024-06-13T07:00:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.394+0000 7f4b00e52740 1 Processor -- start 2024-06-13T07:00:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.394+0000 7f4b00e52740 1 -- start start 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb787e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb787e760 0x561eb7880bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561eb77dc420 con 0x561eb78795f0 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561eb7783c50 con 0x561eb787e760 2024-06-13T07:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4b00e52740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561eb77d83e0 con 0x561eb7878c00 2024-06-13T07:00:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4af6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4af6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55154/0 (socket says 172.21.15.7:55154) 2024-06-13T07:00:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4af6a00640 1 -- 172.21.15.7:0/184085362 learned_addr learned my addr 172.21.15.7:0/184085362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.395+0000 7f4af6a00640 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb787e760 0x561eb7880bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 -- 172.21.15.7:0/184085362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb787e760 msgr2=0x561eb7880bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb787e760 0x561eb7880bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 -- 172.21.15.7:0/184085362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 msgr2=0x561eb787e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb787e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 -- 172.21.15.7:0/184085362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb74f0210 con 0x561eb7878c00 2024-06-13T07:00:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.396+0000 7f4af6a00640 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40165d0 tx=0x7f4ae400af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873b31c2619646a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.397+0000 7f4ae3e00640 1 -- 172.21.15.7:0/184085362 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4018440 con 0x561eb7878c00 2024-06-13T07:00:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.397+0000 7f4ae3e00640 1 -- 172.21.15.7:0/184085362 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae40178c0 con 0x561eb7878c00 2024-06-13T07:00:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.397+0000 7f4ae3e00640 1 -- 172.21.15.7:0/184085362 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4012710 con 0x561eb7878c00 2024-06-13T07:00:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.397+0000 7f4aefe00640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 0x7f4adc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.397+0000 7f4aefe00640 1 -- 172.21.15.7:0/4163900778 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4af00ef980 con 0x7f4af81120c0 2024-06-13T07:00:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.399+0000 7f4b00e52740 1 -- 172.21.15.7:0/184085362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 msgr2=0x561eb7879020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.399+0000 7f4b00e52740 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40165d0 tx=0x7f4ae400af90 comp rx=0 tx=0).stop 2024-06-13T07:00:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 -- 172.21.15.7:0/184085362 shutdown_connections 2024-06-13T07:00:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb787e760 0x561eb7880bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb787e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 --2- 172.21.15.7:0/184085362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb7878c00 0x561eb7879020 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 -- 172.21.15.7:0/184085362 >> 172.21.15.7:0/184085362 conn(0x561eb7840fb0 msgr2=0x561eb7874680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4af7400640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 0x7f4adc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.400+0000 7f4b00e52740 1 -- 172.21.15.7:0/184085362 shutdown_connections 2024-06-13T07:00:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.401+0000 7f4b00e52740 1 -- 172.21.15.7:0/184085362 wait complete. 2024-06-13T07:00:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.401+0000 7f4b00e52740 1 -- start start 2024-06-13T07:00:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.402+0000 7f4af7400640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 0x7f4adc0a5240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561eb7875530 tx=0x7f4ae8014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.402+0000 7f4b00e52740 1 Processor -- start 2024-06-13T07:00:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.403+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb7883510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb788a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561eb7881710 con 0x561eb78795f0 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561eb77970b0 con 0x561eb7878c00 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561eb7881890 con 0x561eb787e760 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4b00e52740 10 client.0 _ll_register_callbacks cb 0x561eb7782f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44306/0 (socket says 172.21.15.7:44306) 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.404+0000 7f4af7400640 1 -- 172.21.15.7:0/3622643471 learned_addr learned my addr 172.21.15.7:0/3622643471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af6a00640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb7883510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af7400640 1 -- 172.21.15.7:0/3622643471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 msgr2=0x561eb788a550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af7400640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb788a550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af7400640 1 -- 172.21.15.7:0/3622643471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 msgr2=0x561eb7883510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af7400640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb7883510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.405+0000 7f4af7400640 1 -- 172.21.15.7:0/3622643471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb75bc620 con 0x561eb7878c00 2024-06-13T07:00:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.407+0000 7f4af7400640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8017b90 tx=0x7f4ae8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873383c571547dd1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.407+0000 7f4ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.408+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8029430 con 0x561eb7878c00 2024-06-13T07:00:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.408+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae40178c0 con 0x561eb7878c00 2024-06-13T07:00:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.408+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4ae80127d0 con 0x561eb7878c00 2024-06-13T07:00:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.408+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ae8036050 con 0x561eb7878c00 2024-06-13T07:00:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.409+0000 7f4b00e52740 10 client.25952 fetch_fsmap learned FSMap version 32 2024-06-13T07:00:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.409+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561eb788d980 con 0x561eb7878c00 2024-06-13T07:00:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.409+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae40178c0 con 0x561eb7878c00 2024-06-13T07:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae8031030 con 0x561eb7878c00 2024-06-13T07:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 10 client.25952 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 20 client.25952 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb788f180 0x561eb7891620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 4 client.25952 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.410+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb77e9b50 con 0x561eb788f180 2024-06-13T07:00:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.411+0000 7f4af6a00640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb788f180 0x561eb7891620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.412+0000 7f4af6a00640 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb788f180 0x561eb7891620 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36ef64ba61e9991 server_cookie=ba3e0a5f9540aa55 in_seq=0 out_seq=0 2024-06-13T07:00:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.412+0000 7f4ae0c00640 10 client.25952 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.413+0000 7f4ae0c00640 1 -- 172.21.15.7:0/3622643471 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561eb77e9b50 con 0x561eb788f180 2024-06-13T07:00:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.413+0000 7f4ae0c00640 10 client.25952 handle_command_reply: tid=0 2024-06-13T07:00:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.413+0000 7f4b00e52740 1 client.25952 shutdown 2024-06-13T07:00:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 msgr2=0x561eb7882fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8017b90 tx=0x7f4ae8004ab0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 shutdown_connections 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb788f180 0x561eb7891620 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb788a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb7883510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 --2- 172.21.15.7:0/3622643471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7882fd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.415+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 >> 172.21.15.7:0/3622643471 conn(0x561eb7840fb0 msgr2=0x561eb7873840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.419+0000 7f4ae0c00640 0 client.25952 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.419+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 shutdown_connections 2024-06-13T07:00:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.421+0000 7f4b00e52740 1 -- 172.21.15.7:0/3622643471 wait complete. 2024-06-13T07:00:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.421+0000 7f4b00e52740 20 client.25952 trim_cache size 0 max 16384 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.448+0000 7f4b00e52740 1 Processor -- start 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 -- start start 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb786f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb788a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb7877570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8012360 con 0x561eb78795f0 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800fc30 con 0x561eb7878c00 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4b00e52740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800da80 con 0x561eb787e760 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb786f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4af6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb788a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb786f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44312/0 (socket says 172.21.15.7:44312) 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.449+0000 7f4af7400640 1 -- 172.21.15.7:0/2175408152 learned_addr learned my addr 172.21.15.7:0/2175408152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb7877570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 -- 172.21.15.7:0/2175408152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 msgr2=0x561eb7877570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb7877570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 -- 172.21.15.7:0/2175408152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 msgr2=0x561eb786f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb786f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 -- 172.21.15.7:0/2175408152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb787b080 con 0x561eb78795f0 2024-06-13T07:00:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.450+0000 7f4af6a00640 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb788a4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4006d40 tx=0x7f4ae400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9702e02020c8e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.451+0000 7f4ae3400640 1 -- 172.21.15.7:0/2175408152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4027070 con 0x561eb78795f0 2024-06-13T07:00:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.451+0000 7f4ae3400640 1 -- 172.21.15.7:0/2175408152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae4004520 con 0x561eb78795f0 2024-06-13T07:00:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.451+0000 7f4ae3400640 1 -- 172.21.15.7:0/2175408152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae401d040 con 0x561eb78795f0 2024-06-13T07:00:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.451+0000 7f4b00e52740 1 -- 172.21.15.7:0/2175408152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 msgr2=0x561eb788a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.451+0000 7f4b00e52740 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb788a4e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4006d40 tx=0x7f4ae400a430 comp rx=0 tx=0).stop 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 -- 172.21.15.7:0/2175408152 shutdown_connections 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb787e760 0x561eb7877570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb78795f0 0x561eb788a4e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 --2- 172.21.15.7:0/2175408152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb786f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 -- 172.21.15.7:0/2175408152 >> 172.21.15.7:0/2175408152 conn(0x561eb7840fb0 msgr2=0x561eb787a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 -- 172.21.15.7:0/2175408152 shutdown_connections 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 -- 172.21.15.7:0/2175408152 wait complete. 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 -- start start 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.452+0000 7f4b00e52740 1 Processor -- start 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb78795f0 0x561eb78875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb787e760 0x561eb78f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800e0f0 con 0x561eb787e760 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800f560 con 0x561eb7878c00 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800e7c0 con 0x561eb78795f0 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4b00e52740 10 client.0 _ll_register_callbacks cb 0x7f4ae80075c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44322/0 (socket says 172.21.15.7:44322) 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 -- 172.21.15.7:0/844914010 learned_addr learned my addr 172.21.15.7:0/844914010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7e00640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb787e760 0x561eb78f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af6a00640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb78795f0 0x561eb78875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 -- 172.21.15.7:0/844914010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb78795f0 msgr2=0x561eb78875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb78795f0 0x561eb78875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 -- 172.21.15.7:0/844914010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb787e760 msgr2=0x561eb78f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb787e760 0x561eb78f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.453+0000 7f4af7400640 1 -- 172.21.15.7:0/844914010 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb788a690 con 0x561eb7878c00 2024-06-13T07:00:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.454+0000 7f4af7400640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8004a80 tx=0x7f4ae801e620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ce2c0f7627144a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.454+0000 7f4ae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.454+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8029430 con 0x561eb7878c00 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.454+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4ae801eaf0 con 0x561eb7878c00 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.454+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae4004520 con 0x561eb7878c00 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ae804a050 con 0x561eb7878c00 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 10 client.25958 fetch_fsmap learned FSMap version 32 2024-06-13T07:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561eb78f9420 con 0x561eb7878c00 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae4004520 con 0x561eb7878c00 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae8031040 con 0x561eb7878c00 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 10 client.25958 fetch_fsmap finished waiting for FSMap version 32 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 10 client.25958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 20 client.25958 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb78f9df0 0x561eb78fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 4 client.25958 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.455+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561eb786fdb0 con 0x561eb78f9df0 2024-06-13T07:00:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.456+0000 7f4af6a00640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb78f9df0 0x561eb78fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.456+0000 7f4af6a00640 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb78f9df0 0x561eb78fc210 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19d03ae7048c371 server_cookie=976bf050812f0fa2 in_seq=0 out_seq=0 2024-06-13T07:00:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.456+0000 7f4ae0c00640 10 client.25958 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4ae0c00640 1 -- 172.21.15.7:0/844914010 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561eb786fdb0 con 0x561eb78f9df0 2024-06-13T07:00:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4ae0c00640 10 client.25958 handle_command_reply: tid=0 2024-06-13T07:00:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 client.25958 shutdown 2024-06-13T07:00:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 msgr2=0x561eb7887080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8004a80 tx=0x7f4ae801e620 comp rx=0 tx=0).stop 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 shutdown_connections 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561eb78f9df0 0x561eb78fc210 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561eb787e760 0x561eb78f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561eb78795f0 0x561eb78875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 --2- 172.21.15.7:0/844914010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561eb7878c00 0x561eb7887080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4ae0c00640 0 client.25958 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 >> 172.21.15.7:0/844914010 conn(0x561eb7840fb0 msgr2=0x561eb7877570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 shutdown_connections 2024-06-13T07:00:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.495+0000 7f4b00e52740 1 -- 172.21.15.7:0/844914010 wait complete. 2024-06-13T07:00:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.496+0000 7f4b00e52740 20 client.25958 trim_cache size 0 max 16384 2024-06-13T07:00:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:00:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:00:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:00:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 msgr2=0x7f4adc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 0x7f4adc0a5240 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561eb7875530 tx=0x7f4ae8014040 comp rx=0 tx=0).stop 2024-06-13T07:00:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 msgr2=0x7f4af81a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af81a9b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009850 tx=0x7f4af0032da0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 shutdown_connections 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4adc0a2d80 0x7f4adc0a5240 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4af81120c0 0x7f4af81a9b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4af806d130 0x7f4af81a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 --2- 172.21.15.7:0/4163900778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806bb50 0x7f4af81a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 >> 172.21.15.7:0/4163900778 conn(0x7f4af80709c0 msgr2=0x7f4af810dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.497+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 shutdown_connections 2024-06-13T07:00:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:02.498+0000 7f4afe000640 1 -- 172.21.15.7:0/4163900778 wait complete. 2024-06-13T07:00:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 240.0 s, 759.4 tps, lat 132.407 ms stddev 159.589 2024-06-13T07:00:03.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:03 smithi007 ceph-mon[32077]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-06-13T07:00:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:03 smithi189 ceph-mon[36616]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-06-13T07:00:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:03 smithi175 ceph-mon[36886]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-06-13T07:00:05.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:05 smithi007 ceph-mon[32077]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 770 op/s 2024-06-13T07:00:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:05 smithi189 ceph-mon[36616]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 770 op/s 2024-06-13T07:00:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:05 smithi175 ceph-mon[36886]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 770 op/s 2024-06-13T07:00:06.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:06 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:06.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:06 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:06.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:06 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:07.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:07 smithi189 ceph-mon[36616]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 747 op/s 2024-06-13T07:00:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:07 smithi175 ceph-mon[36886]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 747 op/s 2024-06-13T07:00:07.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:07 smithi007 ceph-mon[32077]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 747 op/s 2024-06-13T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 245.0 s, 857.4 tps, lat 116.366 ms stddev 121.214 2024-06-13T07:00:09.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:09 smithi189 ceph-mon[36616]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-13T07:00:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:09 smithi175 ceph-mon[36886]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-13T07:00:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:09 smithi007 ceph-mon[32077]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 795 op/s 2024-06-13T07:00:11.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:11 smithi189 ceph-mon[36616]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 667 op/s 2024-06-13T07:00:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:11 smithi175 ceph-mon[36886]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 667 op/s 2024-06-13T07:00:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:11 smithi007 ceph-mon[32077]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 MiB/s wr, 667 op/s 2024-06-13T07:00:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 250.0 s, 695.6 tps, lat 143.855 ms stddev 208.447 2024-06-13T07:00:13.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:13 smithi189 ceph-mon[36616]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-06-13T07:00:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:13 smithi175 ceph-mon[36886]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-06-13T07:00:13.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:13 smithi007 ceph-mon[32077]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-06-13T07:00:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:15 smithi189 ceph-mon[36616]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 677 op/s 2024-06-13T07:00:15.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:15 smithi175 ceph-mon[36886]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 677 op/s 2024-06-13T07:00:15.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:15 smithi007 ceph-mon[32077]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.3 MiB/s wr, 677 op/s 2024-06-13T07:00:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.707+0000 7fa58ee00640 1 -- 172.21.15.7:0/140906486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 msgr2=0x7fa5881071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.707+0000 7fa58ee00640 1 --2- 172.21.15.7:0/140906486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881071a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa574009860 tx=0x7fa57402a340 comp rx=0 tx=0).stop 2024-06-13T07:00:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.708+0000 7fa58ee00640 1 -- 172.21.15.7:0/140906486 shutdown_connections 2024-06-13T07:00:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.708+0000 7fa58ee00640 1 --2- 172.21.15.7:0/140906486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa588109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.708+0000 7fa58ee00640 1 --2- 172.21.15.7:0/140906486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881071a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.708+0000 7fa58ee00640 1 --2- 172.21.15.7:0/140906486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 0x7fa588104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.708+0000 7fa58ee00640 1 -- 172.21.15.7:0/140906486 >> 172.21.15.7:0/140906486 conn(0x7fa5880fbe70 msgr2=0x7fa5880fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.709+0000 7fa58ee00640 1 -- 172.21.15.7:0/140906486 shutdown_connections 2024-06-13T07:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.709+0000 7fa58ee00640 1 -- 172.21.15.7:0/140906486 wait complete. 2024-06-13T07:00:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.710+0000 7fa58ee00640 1 Processor -- start 2024-06-13T07:00:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.710+0000 7fa58ee00640 1 -- start start 2024-06-13T07:00:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.710+0000 7fa58ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 0x7fa588110d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa5881ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa588101f90 con 0x7fa588102480 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa588101e10 con 0x7fa588104db0 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa588102110 con 0x7fa5881076e0 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59298/0 (socket says 172.21.15.7:59298) 2024-06-13T07:00:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 -- 172.21.15.7:0/587839649 learned_addr learned my addr 172.21.15.7:0/587839649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587e00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 0x7fa588110d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa58ce00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa5881ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 -- 172.21.15.7:0/587839649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 msgr2=0x7fa5881ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa5881ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 -- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 msgr2=0x7fa588110d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.711+0000 7fa587400640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 0x7fa588110d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.712+0000 7fa587400640 1 -- 172.21.15.7:0/587839649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5881ae7d0 con 0x7fa588104db0 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.712+0000 7fa58ce00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa5881ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.712+0000 7fa587400640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5740305b0 tx=0x7fa574033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.712+0000 7fa584c00640 1 -- 172.21.15.7:0/587839649 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57403f020 con 0x7fa588104db0 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.713+0000 7fa584c00640 1 -- 172.21.15.7:0/587839649 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa574030850 con 0x7fa588104db0 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.713+0000 7fa584c00640 1 -- 172.21.15.7:0/587839649 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574037040 con 0x7fa588104db0 2024-06-13T07:00:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.713+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5881ae9a0 con 0x7fa588104db0 2024-06-13T07:00:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.713+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5881aee10 con 0x7fa588104db0 2024-06-13T07:00:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.718+0000 7fa584c00640 1 -- 172.21.15.7:0/587839649 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa574042020 con 0x7fa588104db0 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.720+0000 7fa591c70740 1 Processor -- start 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.720+0000 7fa591c70740 1 -- start start 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b52f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52f770 0x55e86b531c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e86b47c420 con 0x55e86b52f770 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e86b423c50 con 0x55e86b52a600 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa591c70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e86b4783e0 con 0x55e86b529c10 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:60150/0 (socket says 172.21.15.7:60150) 2024-06-13T07:00:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa58ce00640 1 -- 172.21.15.7:0/2791496112 learned_addr learned my addr 172.21.15.7:0/2791496112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa587e00640 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b52f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.721+0000 7fa587400640 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52f770 0x55e86b531c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 -- 172.21.15.7:0/2791496112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 msgr2=0x55e86b52f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b52f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 -- 172.21.15.7:0/2791496112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52f770 msgr2=0x55e86b531c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52f770 0x55e86b531c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 -- 172.21.15.7:0/2791496112 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86b190210 con 0x55e86b529c10 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.722+0000 7fa58ce00640 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa580009f60 tx=0x7fa580009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a1852725d5c9f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.723+0000 7fa57d600640 1 -- 172.21.15.7:0/2791496112 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580033650 con 0x55e86b529c10 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.723+0000 7fa57d600640 1 -- 172.21.15.7:0/2791496112 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa580033cd0 con 0x55e86b529c10 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.723+0000 7fa57d600640 1 -- 172.21.15.7:0/2791496112 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580032df0 con 0x55e86b529c10 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.724+0000 7fa591c70740 1 -- 172.21.15.7:0/2791496112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 msgr2=0x55e86b52a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.724+0000 7fa591c70740 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa580009f60 tx=0x7fa580009f90 comp rx=0 tx=0).stop 2024-06-13T07:00:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.724+0000 7fa584c00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 0x7fa56c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.724+0000 7fa584c00640 1 -- 172.21.15.7:0/587839649 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa5740ef4e0 con 0x7fa588104db0 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 -- 172.21.15.7:0/2791496112 shutdown_connections 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52f770 0x55e86b531c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b52f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 --2- 172.21.15.7:0/2791496112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b529c10 0x55e86b52a030 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 -- 172.21.15.7:0/2791496112 >> 172.21.15.7:0/2791496112 conn(0x55e86b4e0fb0 msgr2=0x55e86b514680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 -- 172.21.15.7:0/2791496112 shutdown_connections 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.726+0000 7fa591c70740 1 -- 172.21.15.7:0/2791496112 wait complete. 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.727+0000 7fa591c70740 1 -- start start 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.727+0000 7fa58ce00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 0x7fa56c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.727+0000 7fa591c70740 1 Processor -- start 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b51a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52f770 0x55e86b5219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e86b518b60 con 0x55e86b529c10 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e86b4370b0 con 0x55e86b52f770 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e86b518ce0 con 0x55e86b52a600 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa591c70740 10 client.0 _ll_register_callbacks cb 0x55e86b422f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:60164/0 (socket says 172.21.15.7:60164) 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 -- 172.21.15.7:0/3393691230 learned_addr learned my addr 172.21.15.7:0/3393691230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 -- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52f770 msgr2=0x55e86b5219a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52f770 0x55e86b5219a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 -- 172.21.15.7:0/3393691230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 msgr2=0x55e86b51a420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b51a420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.728+0000 7fa587400640 1 -- 172.21.15.7:0/3393691230 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86b25c620 con 0x55e86b52a600 2024-06-13T07:00:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.729+0000 7fa587400640 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa5740b83d0 tx=0x7fa57402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e969820876acb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.729+0000 7fa572000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740bc100 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa5740ef260 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa580033cd0 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa574003e20 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa591c70740 10 client.25939 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e86b518c20 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.730+0000 7fa58ce00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 0x7fa56c0a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa580032890 tx=0x7fa580000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.731+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa580033cd0 con 0x55e86b52a600 2024-06-13T07:00:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.732+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa57403a030 con 0x55e86b52a600 2024-06-13T07:00:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 10 client.25939 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 20 client.25939 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b526400 0x55e86b528810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 4 client.25939 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e86b489b50 con 0x55e86b526400 2024-06-13T07:00:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.733+0000 7fa587e00640 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b526400 0x55e86b528810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.735+0000 7fa587e00640 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b526400 0x55e86b528810 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5843a7f822be4eb server_cookie=ae5c3eb81ac5093a in_seq=0 out_seq=0 2024-06-13T07:00:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.735+0000 7fa572000640 10 client.25939 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.737+0000 7fa572000640 1 -- 172.21.15.7:0/3393691230 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e86b489b50 con 0x55e86b526400 2024-06-13T07:00:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.737+0000 7fa572000640 10 client.25939 handle_command_reply: tid=0 2024-06-13T07:00:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.737+0000 7fa591c70740 1 client.25939 shutdown 2024-06-13T07:00:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 msgr2=0x55e86b51a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa5740b83d0 tx=0x7fa57402a340 comp rx=0 tx=0).stop 2024-06-13T07:00:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 shutdown_connections 2024-06-13T07:00:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b526400 0x55e86b528810 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52f770 0x55e86b5219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52a600 0x55e86b51a960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 --2- 172.21.15.7:0/3393691230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b51a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa572000640 0 client.25939 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.738+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 >> 172.21.15.7:0/3393691230 conn(0x55e86b4e0fb0 msgr2=0x55e86b513840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.739+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 shutdown_connections 2024-06-13T07:00:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.739+0000 7fa591c70740 1 -- 172.21.15.7:0/3393691230 wait complete. 2024-06-13T07:00:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.739+0000 7fa591c70740 20 client.25939 trim_cache size 0 max 16384 2024-06-13T07:00:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 Processor -- start 2024-06-13T07:00:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 -- start start 2024-06-13T07:00:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b5190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b521960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5881b3480 con 0x55e86b529c10 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5881b0cf0 con 0x55e86b52a600 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.762+0000 7fa591c70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa588073610 con 0x55e86b52f770 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa587e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b5190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:60178/0 (socket says 172.21.15.7:60178) 2024-06-13T07:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 -- 172.21.15.7:0/16734582 learned_addr learned my addr 172.21.15.7:0/16734582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa587400640 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b521960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 -- 172.21.15.7:0/16734582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 msgr2=0x55e86b521960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b521960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 -- 172.21.15.7:0/16734582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 msgr2=0x55e86b5190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b5190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.763+0000 7fa58ce00640 1 -- 172.21.15.7:0/16734582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86b52ae90 con 0x55e86b52f770 2024-06-13T07:00:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.764+0000 7fa58ce00640 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa580009f30 tx=0x7fa580043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7fe0694cd05e13 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.765+0000 7fa57cc00640 1 -- 172.21.15.7:0/16734582 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580003e40 con 0x55e86b52f770 2024-06-13T07:00:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.765+0000 7fa57cc00640 1 -- 172.21.15.7:0/16734582 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa580050070 con 0x55e86b52f770 2024-06-13T07:00:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.765+0000 7fa57cc00640 1 -- 172.21.15.7:0/16734582 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800052a0 con 0x55e86b52f770 2024-06-13T07:00:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 -- 172.21.15.7:0/16734582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 msgr2=0x55e86b514100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa580009f30 tx=0x7fa580043000 comp rx=0 tx=0).stop 2024-06-13T07:00:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 -- 172.21.15.7:0/16734582 shutdown_connections 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b514100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b52a600 0x55e86b521960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 --2- 172.21.15.7:0/16734582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b529c10 0x55e86b5190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.767+0000 7fa591c70740 1 -- 172.21.15.7:0/16734582 >> 172.21.15.7:0/16734582 conn(0x55e86b4e0fb0 msgr2=0x55e86b52b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.768+0000 7fa591c70740 1 -- 172.21.15.7:0/16734582 shutdown_connections 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.768+0000 7fa591c70740 1 -- 172.21.15.7:0/16734582 wait complete. 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 -- start start 2024-06-13T07:00:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 Processor -- start 2024-06-13T07:00:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52a600 0x55e86b51eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b5984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5881af1d0 con 0x55e86b52a600 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5881b0620 con 0x55e86b529c10 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5881af880 con 0x55e86b52f770 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.769+0000 7fa591c70740 10 client.0 _ll_register_callbacks cb 0x7fa574005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59330/0 (socket says 172.21.15.7:59330) 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 -- 172.21.15.7:0/3677992164 learned_addr learned my addr 172.21.15.7:0/3677992164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa58ce00640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b5984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587400640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52a600 0x55e86b51eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 -- 172.21.15.7:0/3677992164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 msgr2=0x55e86b5984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b5984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 -- 172.21.15.7:0/3677992164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52a600 msgr2=0x55e86b51eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52a600 0x55e86b51eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 -- 172.21.15.7:0/3677992164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86b598a10 con 0x55e86b529c10 2024-06-13T07:00:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa587e00640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa57800de00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e81a0e5c6cee77d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.770+0000 7fa572000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578011440 con 0x55e86b529c10 2024-06-13T07:00:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa580050070 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa5780115e0 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa5780117e0 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 10 client.25973 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e86b599d00 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa580050070 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa578021070 con 0x55e86b529c10 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 10 client.25973 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 20 client.25973 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b59a6e0 0x55e86b59cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 4 client.25973 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.771+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e86b519880 con 0x55e86b59a6e0 2024-06-13T07:00:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.772+0000 7fa587400640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b59a6e0 0x55e86b59cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.772+0000 7fa587400640 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b59a6e0 0x55e86b59cb00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e85d12b8cb9c4bf server_cookie=627adfbc58038d9a in_seq=0 out_seq=0 2024-06-13T07:00:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.772+0000 7fa572000640 10 client.25973 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa572000640 1 -- 172.21.15.7:0/3677992164 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e86b519880 con 0x55e86b59a6e0 2024-06-13T07:00:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa572000640 10 client.25973 handle_command_reply: tid=0 2024-06-13T07:00:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 client.25973 shutdown 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 msgr2=0x55e86b51e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa578000c00 tx=0x7fa57800de00 comp rx=0 tx=0).stop 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 shutdown_connections 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e86b59a6e0 0x55e86b59cb00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86b52f770 0x55e86b5984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86b52a600 0x55e86b51eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 --2- 172.21.15.7:0/3677992164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e86b529c10 0x55e86b51e760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 >> 172.21.15.7:0/3677992164 conn(0x55e86b4e0fb0 msgr2=0x55e86b514100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa572000640 0 client.25973 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 shutdown_connections 2024-06-13T07:00:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 1 -- 172.21.15.7:0/3677992164 wait complete. 2024-06-13T07:00:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.782+0000 7fa591c70740 20 client.25973 trim_cache size 0 max 16384 2024-06-13T07:00:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:00:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:00:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.783+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 msgr2=0x7fa56c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.783+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 0x7fa56c0a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa580032890 tx=0x7fa580000f40 comp rx=0 tx=0).stop 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.783+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 msgr2=0x7fa5881112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.783+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5740305b0 tx=0x7fa574033d90 comp rx=0 tx=0).stop 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 shutdown_connections 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa56c0a2cb0 0x7fa56c0a5170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5881076e0 0x7fa5881ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa588104db0 0x7fa5881112b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 --2- 172.21.15.7:0/587839649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa588102480 0x7fa588110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 >> 172.21.15.7:0/587839649 conn(0x7fa5880fbe70 msgr2=0x7fa588102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.785+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 shutdown_connections 2024-06-13T07:00:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:15.786+0000 7fa58ee00640 1 -- 172.21.15.7:0/587839649 wait complete. 2024-06-13T07:00:16.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:16 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:16.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:16 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:16.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:16 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:00:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:17 smithi189 ceph-mon[36616]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-06-13T07:00:17.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:17 smithi175 ceph-mon[36886]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-06-13T07:00:17.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:17 smithi007 ceph-mon[32077]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 561 op/s 2024-06-13T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 255.0 s, 797.4 tps, lat 123.706 ms stddev 137.708 2024-06-13T07:00:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:19 smithi189 ceph-mon[36616]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-06-13T07:00:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:19 smithi175 ceph-mon[36886]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-06-13T07:00:19.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:19 smithi007 ceph-mon[32077]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-06-13T07:00:19.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:00:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:21.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:21 smithi007 ceph-mon[32077]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-06-13T07:00:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:21 smithi189 ceph-mon[36616]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-06-13T07:00:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:21 smithi175 ceph-mon[36886]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-06-13T07:00:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 260.0 s, 1083.6 tps, lat 94.481 ms stddev 132.494 2024-06-13T07:00:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 -- 172.21.15.7:0/1173283403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 msgr2=0x7f1058107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 --2- 172.21.15.7:0/1173283403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058107110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1040009990 tx=0x7f1040009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 -- 172.21.15.7:0/1173283403 shutdown_connections 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 --2- 172.21.15.7:0/1173283403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058107840 0x7f105810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 --2- 172.21.15.7:0/1173283403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1058102740 0x7f1058102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 --2- 172.21.15.7:0/1173283403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058107110 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.192+0000 7f105f200700 1 -- 172.21.15.7:0/1173283403 >> 172.21.15.7:0/1173283403 conn(0x7f10580795f0 msgr2=0x7f1058079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.193+0000 7f105f200700 1 -- 172.21.15.7:0/1173283403 shutdown_connections 2024-06-13T07:00:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.193+0000 7f105f200700 1 -- 172.21.15.7:0/1173283403 wait complete. 2024-06-13T07:00:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.195+0000 7f105f200700 1 Processor -- start 2024-06-13T07:00:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.195+0000 7f105f200700 1 -- start start 2024-06-13T07:00:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.198+0000 7f105f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 0x7f1058073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.198+0000 7f105f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.198+0000 7f105f200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1058107840 0x7f105806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f104f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f104f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40552/0 (socket says 172.21.15.7:40552) 2024-06-13T07:00:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f104f400700 1 -- 172.21.15.7:0/1545237124 learned_addr learned my addr 172.21.15.7:0/1545237124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f105f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1058114a30 con 0x7f1058102740 2024-06-13T07:00:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10581148b0 con 0x7f1058107840 2024-06-13T07:00:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.199+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1058114bb0 con 0x7f10581030f0 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f1057e00700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 0x7f1058073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f104f400700 1 -- 172.21.15.7:0/1545237124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1058107840 msgr2=0x7f105806ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f104f400700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1058107840 0x7f105806ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f104f400700 1 -- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 msgr2=0x7f1058073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f104f400700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 0x7f1058073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f104f400700 1 -- 172.21.15.7:0/1545237124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105806d770 con 0x7f10581030f0 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.200+0000 7f1057e00700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 0x7f1058073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.201+0000 7f104f400700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.205+0000 7f1055600700 1 -- 172.21.15.7:0/1545237124 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040027070 con 0x7f10581030f0 2024-06-13T07:00:23.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.205+0000 7f1055600700 1 -- 172.21.15.7:0/1545237124 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1040004110 con 0x7f10581030f0 2024-06-13T07:00:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.205+0000 7f1055600700 1 -- 172.21.15.7:0/1545237124 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040022da0 con 0x7f10581030f0 2024-06-13T07:00:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.205+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105806d9a0 con 0x7f10581030f0 2024-06-13T07:00:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.206+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10580718e0 con 0x7f10581030f0 2024-06-13T07:00:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.214+0000 7f1055600700 1 -- 172.21.15.7:0/1545237124 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1040022500 con 0x7f10581030f0 2024-06-13T07:00:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.216+0000 7f1055600700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 0x7f10380a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.216+0000 7f1055600700 1 -- 172.21.15.7:0/1545237124 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f10400d03c0 con 0x7f10581030f0 2024-06-13T07:00:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.217+0000 7f1057e00700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 0x7f10380a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.218+0000 7f1069c71b80 1 Processor -- start 2024-06-13T07:00:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.218+0000 7f1057e00700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 0x7f10380a52b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f10480094f0 tx=0x7f1048009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.218+0000 7f1069c71b80 1 -- start start 2024-06-13T07:00:23.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.220+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc488070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.220+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc493300 0x5599fc4956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f1069c71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5599fc397a10 con 0x5599fc404e40 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f1069c71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5599fc3dab00 con 0x5599fc493300 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f1069c71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599fc3958f0 con 0x5599fc4885b0 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40566/0 (socket says 172.21.15.7:40566) 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 -- 172.21.15.7:0/3381521520 learned_addr learned my addr 172.21.15.7:0/3381521520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:23.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f1057e00700 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc488070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 -- 172.21.15.7:0/3381521520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc493300 msgr2=0x5599fc4956f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc493300 0x5599fc4956f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 -- 172.21.15.7:0/3381521520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 msgr2=0x5599fc488070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.221+0000 7f104f400700 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc488070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.222+0000 7f104f400700 1 -- 172.21.15.7:0/3381521520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fc382f90 con 0x5599fc4885b0 2024-06-13T07:00:23.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.222+0000 7f104f400700 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1040098d70 tx=0x7f1040003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e3b80aef03bee server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.222+0000 7f104d600700 1 -- 172.21.15.7:0/3381521520 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104009d010 con 0x5599fc4885b0 2024-06-13T07:00:23.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.222+0000 7f104d600700 1 -- 172.21.15.7:0/3381521520 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10400a1e50 con 0x5599fc4885b0 2024-06-13T07:00:23.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.222+0000 7f104d600700 1 -- 172.21.15.7:0/3381521520 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10400cdd00 con 0x5599fc4885b0 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 -- 172.21.15.7:0/3381521520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 msgr2=0x5599fc488a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1040098d70 tx=0x7f1040003890 comp rx=0 tx=0).stop 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 -- 172.21.15.7:0/3381521520 shutdown_connections 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc493300 0x5599fc4956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc488070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 --2- 172.21.15.7:0/3381521520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc4885b0 0x5599fc488a30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 -- 172.21.15.7:0/3381521520 >> 172.21.15.7:0/3381521520 conn(0x5599fc3b5170 msgr2=0x5599fc4844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 -- 172.21.15.7:0/3381521520 shutdown_connections 2024-06-13T07:00:23.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.223+0000 7f1069c71b80 1 -- 172.21.15.7:0/3381521520 wait complete. 2024-06-13T07:00:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.224+0000 7f1069c71b80 1 -- start start 2024-06-13T07:00:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.224+0000 7f1069c71b80 1 Processor -- start 2024-06-13T07:00:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 0x5599fc490360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5599fc376f60 con 0x5599fc493300 2024-06-13T07:00:23.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5599fc3d6ac0 con 0x5599fc4885b0 2024-06-13T07:00:23.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599fc495d70 con 0x5599fc404e40 2024-06-13T07:00:23.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1069c71b80 10 client.0 _ll_register_callbacks cb 0x5599fc374720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1057e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1057e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40574/0 (socket says 172.21.15.7:40574) 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f1057e00700 1 -- 172.21.15.7:0/633418909 learned_addr learned my addr 172.21.15.7:0/633418909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.225+0000 7f104f400700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f105d200700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 0x5599fc490360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 -- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 msgr2=0x5599fc48fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 -- 172.21.15.7:0/633418909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 msgr2=0x5599fc490360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 0x5599fc490360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 -- 172.21.15.7:0/633418909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fc3298d0 con 0x5599fc404e40 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f105d200700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 0x5599fc490360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.226+0000 7f1057e00700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1048018dd0 tx=0x7f1048011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8d2ddf2b4a670f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.227+0000 7f1046000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:23.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.227+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048015d50 con 0x5599fc404e40 2024-06-13T07:00:23.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.227+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1048026940 con 0x5599fc404e40 2024-06-13T07:00:23.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.227+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10400a1e50 con 0x5599fc404e40 2024-06-13T07:00:23.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.228+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1048026b40 con 0x5599fc404e40 2024-06-13T07:00:23.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.228+0000 7f1069c71b80 10 client.25957 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:23.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.228+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599fc4a04c0 con 0x5599fc404e40 2024-06-13T07:00:23.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.228+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10400a1e50 con 0x5599fc404e40 2024-06-13T07:00:23.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1048016c60 con 0x5599fc404e40 2024-06-13T07:00:23.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 10 client.25957 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:23.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:23.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 20 client.25957 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:23.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc4a2b00 0x5599fc4a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 4 client.25957 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:23.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.230+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599fc3e8000 con 0x5599fc4a2b00 2024-06-13T07:00:23.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.231+0000 7f104f400700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc4a2b00 0x5599fc4a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.231+0000 7f104f400700 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc4a2b00 0x5599fc4a4ef0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6558aadaab10e2dc server_cookie=34129ef094b7bbbb in_seq=0 out_seq=0 2024-06-13T07:00:23.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.231+0000 7f1046000700 10 client.25957 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:23.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.232+0000 7f1046000700 1 -- 172.21.15.7:0/633418909 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5599fc3e8000 con 0x5599fc4a2b00 2024-06-13T07:00:23.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.232+0000 7f1046000700 10 client.25957 handle_command_reply: tid=0 2024-06-13T07:00:23.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 client.25957 shutdown 2024-06-13T07:00:23.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 msgr2=0x5599fc492b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1048018dd0 tx=0x7f1048011000 comp rx=0 tx=0).stop 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 shutdown_connections 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc4a2b00 0x5599fc4a4ef0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc493300 0x5599fc490360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc404e40 0x5599fc492b20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1046000700 0 client.25957 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 --2- 172.21.15.7:0/633418909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 >> 172.21.15.7:0/633418909 conn(0x5599fc3b5170 msgr2=0x5599fc483740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.233+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 shutdown_connections 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.234+0000 7f1069c71b80 1 -- 172.21.15.7:0/633418909 wait complete. 2024-06-13T07:00:23.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.234+0000 7f1069c71b80 20 client.25957 trim_cache size 0 max 16384 2024-06-13T07:00:23.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 Processor -- start 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 -- start start 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc492660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b7190 con 0x5599fc404e40 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b49c0 con 0x5599fc4885b0 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.270+0000 7f1069c71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1058113ab0 con 0x5599fc493300 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40584/0 (socket says 172.21.15.7:40584) 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 -- 172.21.15.7:0/2310599607 learned_addr learned my addr 172.21.15.7:0/2310599607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 -- 172.21.15.7:0/2310599607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 msgr2=0x5599fc48a6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 -- 172.21.15.7:0/2310599607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 msgr2=0x5599fc492660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc492660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.271+0000 7f105d200700 1 -- 172.21.15.7:0/2310599607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fc4a5520 con 0x5599fc493300 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.272+0000 7f105d200700 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f105000a770 tx=0x7f105000aa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ba78f7dbd048eb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.272+0000 7f104cc00700 1 -- 172.21.15.7:0/2310599607 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050015070 con 0x5599fc493300 2024-06-13T07:00:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.272+0000 7f104cc00700 1 -- 172.21.15.7:0/2310599607 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1050011ce0 con 0x5599fc493300 2024-06-13T07:00:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.272+0000 7f104cc00700 1 -- 172.21.15.7:0/2310599607 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050018db0 con 0x5599fc493300 2024-06-13T07:00:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 -- 172.21.15.7:0/2310599607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 msgr2=0x5599fc48ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f105000a770 tx=0x7f105000aa80 comp rx=0 tx=0).stop 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 -- 172.21.15.7:0/2310599607 shutdown_connections 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc48ac10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc404e40 0x5599fc492660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 --2- 172.21.15.7:0/2310599607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc4885b0 0x5599fc48a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 -- 172.21.15.7:0/2310599607 >> 172.21.15.7:0/2310599607 conn(0x5599fc3b5170 msgr2=0x5599fc489fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 -- 172.21.15.7:0/2310599607 shutdown_connections 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.273+0000 7f1069c71b80 1 -- 172.21.15.7:0/2310599607 wait complete. 2024-06-13T07:00:23.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.274+0000 7f1069c71b80 1 -- start start 2024-06-13T07:00:23.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.274+0000 7f1069c71b80 1 Processor -- start 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.275+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.275+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc4885b0 0x5599fc507200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1069c71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc49afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1069c71b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b2e80 con 0x5599fc4885b0 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1069c71b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b42f0 con 0x5599fc404e40 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1069c71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b3550 con 0x5599fc493300 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1069c71b80 10 client.0 _ll_register_callbacks cb 0x5599fc374720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36106/0 (socket says 172.21.15.7:36106) 2024-06-13T07:00:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 -- 172.21.15.7:0/524278480 learned_addr learned my addr 172.21.15.7:0/524278480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f105d200700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc49afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 -- 172.21.15.7:0/524278480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 msgr2=0x5599fc49afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc49afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 -- 172.21.15.7:0/524278480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc4885b0 msgr2=0x5599fc507200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc4885b0 0x5599fc507200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.276+0000 7f1057e00700 1 -- 172.21.15.7:0/524278480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fc49b500 con 0x5599fc404e40 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1057e00700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10480093c0 tx=0x7f1048015d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90c9d2eb217ebb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1046000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104802ea70 con 0x5599fc404e40 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f104802ec10 con 0x5599fc404e40 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1050011ce0 con 0x5599fc404e40 2024-06-13T07:00:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.277+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f104802ee10 con 0x5599fc404e40 2024-06-13T07:00:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.278+0000 7f1069c71b80 10 client.25982 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.278+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599fc51f060 con 0x5599fc404e40 2024-06-13T07:00:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.278+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1050011ce0 con 0x5599fc404e40 2024-06-13T07:00:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1048026ce0 con 0x5599fc404e40 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 20 client.25982 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc51f760 0x5599fc521b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 4 client.25982 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599fc3e2580 con 0x5599fc51f760 2024-06-13T07:00:23.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.279+0000 7f104f400700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc51f760 0x5599fc521b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:23.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.280+0000 7f104f400700 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc51f760 0x5599fc521b80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb843f140084d907 server_cookie=af0af756b5bf833e in_seq=0 out_seq=0 2024-06-13T07:00:23.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.280+0000 7f1046000700 10 client.25982 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:23.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.280+0000 7f1046000700 1 -- 172.21.15.7:0/524278480 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5599fc3e2580 con 0x5599fc51f760 2024-06-13T07:00:23.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.280+0000 7f1046000700 10 client.25982 handle_command_reply: tid=0 2024-06-13T07:00:23.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.280+0000 7f1069c71b80 1 client.25982 shutdown 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 msgr2=0x5599fc506cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10480093c0 tx=0x7f1048015d00 comp rx=0 tx=0).stop 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 shutdown_connections 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5599fc51f760 0x5599fc521b80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599fc493300 0x5599fc49afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5599fc404e40 0x5599fc506cc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 --2- 172.21.15.7:0/524278480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5599fc4885b0 0x5599fc507200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 >> 172.21.15.7:0/524278480 conn(0x5599fc3b5170 msgr2=0x5599fc48a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.281+0000 7f1046000700 0 client.25982 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:23.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.282+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 shutdown_connections 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.282+0000 7f1069c71b80 1 -- 172.21.15.7:0/524278480 wait complete. 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.282+0000 7f1069c71b80 20 client.25982 trim_cache size 0 max 16384 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:00:23.282 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 msgr2=0x7f10380a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 0x7f10380a52b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f10480094f0 tx=0x7f1048009040 comp rx=0 tx=0).stop 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 msgr2=0x7f1058074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040009f90 comp rx=0 tx=0).stop 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 shutdown_connections 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f10380a2df0 0x7f10380a52b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1058107840 0x7f105806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1058102740 0x7f1058073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 --2- 172.21.15.7:0/1545237124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10581030f0 0x7f1058074120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:23.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 >> 172.21.15.7:0/1545237124 conn(0x7f10580795f0 msgr2=0x7f1058108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:23.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 shutdown_connections 2024-06-13T07:00:23.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:23.283+0000 7f105f200700 1 -- 172.21.15.7:0/1545237124 wait complete. 2024-06-13T07:00:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:23 smithi189 ceph-mon[36616]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 578 op/s 2024-06-13T07:00:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:23 smithi175 ceph-mon[36886]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 578 op/s 2024-06-13T07:00:23.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:23 smithi007 ceph-mon[32077]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 578 op/s 2024-06-13T07:00:24.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:00:24.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:00:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:25.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:25 smithi007 ceph-mon[32077]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-06-13T07:00:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:25 smithi189 ceph-mon[36616]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-06-13T07:00:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:25 smithi175 ceph-mon[36886]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 576 op/s 2024-06-13T07:00:27.518 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:27 smithi007 ceph-mon[32077]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 551 op/s 2024-06-13T07:00:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:27 smithi189 ceph-mon[36616]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 551 op/s 2024-06-13T07:00:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:27 smithi175 ceph-mon[36886]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 551 op/s 2024-06-13T07:00:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 265.0 s, 1094.0 tps, lat 87.853 ms stddev 96.202 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 -- 172.21.15.7:0/3228306988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157410cbb0 msgr2=0x7f1574109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 --2- 172.21.15.7:0/3228306988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157410cbb0 0x7f1574109490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b0a0 tx=0x7f156c00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 -- 172.21.15.7:0/3228306988 shutdown_connections 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 --2- 172.21.15.7:0/3228306988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117520 0x7f1574119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 --2- 172.21.15.7:0/3228306988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157410cbb0 0x7f1574109490 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b0a0 tx=0x7f156c00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 --2- 172.21.15.7:0/3228306988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 0x7f1574109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.922+0000 7f157b200700 1 -- 172.21.15.7:0/3228306988 >> 172.21.15.7:0/3228306988 conn(0x7f157406d600 msgr2=0x7f157406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:27.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.923+0000 7f157b200700 1 -- 172.21.15.7:0/3228306988 shutdown_connections 2024-06-13T07:00:27.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.923+0000 7f157b200700 1 -- 172.21.15.7:0/3228306988 wait complete. 2024-06-13T07:00:27.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.924+0000 7f157b200700 1 Processor -- start 2024-06-13T07:00:27.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.924+0000 7f157b200700 1 -- start start 2024-06-13T07:00:27.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 0x7f15741140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117070 0x7f1574114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f157411d2c0 con 0x7f1574117070 2024-06-13T07:00:27.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f157411d140 con 0x7f15741099d0 2024-06-13T07:00:27.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.925+0000 7f157b200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f157411d440 con 0x7f1574117520 2024-06-13T07:00:27.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.926+0000 7f1573400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.926+0000 7f1573400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40610/0 (socket says 172.21.15.7:40610) 2024-06-13T07:00:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.926+0000 7f1573400700 1 -- 172.21.15.7:0/1647924605 learned_addr learned my addr 172.21.15.7:0/1647924605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.926+0000 7f1579200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117070 0x7f1574114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.926+0000 7f1573e00700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 0x7f15741140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.927+0000 7f1573400700 1 -- 172.21.15.7:0/1647924605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 msgr2=0x7f15741140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.927+0000 7f1573400700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 0x7f15741140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.927+0000 7f1573400700 1 -- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117070 msgr2=0x7f1574114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.927+0000 7f1573400700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117070 0x7f1574114b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.927+0000 7f1573400700 1 -- 172.21.15.7:0/1647924605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15741b43c0 con 0x7f1574117520 2024-06-13T07:00:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.930+0000 7f1573400700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1564009fc0 tx=0x7f156400b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.931+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564026020 con 0x7f1574117520 2024-06-13T07:00:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.931+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15741a27f0 con 0x7f1574117520 2024-06-13T07:00:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.932+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741a2d80 con 0x7f1574117520 2024-06-13T07:00:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.932+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1564022b20 con 0x7f1574117520 2024-06-13T07:00:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.932+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564021c10 con 0x7f1574117520 2024-06-13T07:00:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.933+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1554005330 con 0x7f1574117520 2024-06-13T07:00:27.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.936+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1564029020 con 0x7f1574117520 2024-06-13T07:00:27.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.939+0000 7f1570c00700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 0x7f15580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:27.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.940+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f15640df790 con 0x7f1574117520 2024-06-13T07:00:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.940+0000 7f1573e00700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 0x7f15580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.942+0000 7f1573e00700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 0x7f15580a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f156c009040 tx=0x7f156c0090d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:27.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:27.944+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1564030020 con 0x7f1574117520 2024-06-13T07:00:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.423+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1554005620 con 0x7f1574117520 2024-06-13T07:00:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.424+0000 7f1570c00700 1 -- 172.21.15.7:0/1647924605 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11871 (secure 0 0 0) 0x7f15640ac350 con 0x7f1574117520 2024-06-13T07:00:28.424 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:00:28.424 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:00:15.083820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T07:00:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.427+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 msgr2=0x7f15580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.427+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 0x7f15580a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f156c009040 tx=0x7f156c0090d0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.427+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 msgr2=0x7f15741145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.427+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1564009fc0 tx=0x7f156400b830 comp rx=0 tx=0).stop 2024-06-13T07:00:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 shutdown_connections 2024-06-13T07:00:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1574117070 0x7f1574114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f15580a2cf0 0x7f15580a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15741099d0 0x7f15741140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 --2- 172.21.15.7:0/1647924605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1574117520 0x7f15741145f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.428+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 >> 172.21.15.7:0/1647924605 conn(0x7f157406d600 msgr2=0x7f1574117900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.430+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 shutdown_connections 2024-06-13T07:00:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:28.430+0000 7f157b200700 1 -- 172.21.15.7:0/1647924605 wait complete. 2024-06-13T07:00:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-06-13T07:00:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 -- 172.21.15.7:0/3209144414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec0743a0 msgr2=0x7f40ec074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 --2- 172.21.15.7:0/3209144414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec0743a0 0x7f40ec074800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009900 tx=0x7f40d802a4b0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 -- 172.21.15.7:0/3209144414 shutdown_connections 2024-06-13T07:00:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 --2- 172.21.15.7:0/3209144414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec109890 0x7f40ec10bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 --2- 172.21.15.7:0/3209144414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec0743a0 0x7f40ec074800 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 --2- 172.21.15.7:0/3209144414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec075980 0x7f40ec073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 -- 172.21.15.7:0/3209144414 >> 172.21.15.7:0/3209144414 conn(0x7f40ec0fcf70 msgr2=0x7f40ec0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 -- 172.21.15.7:0/3209144414 shutdown_connections 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 -- 172.21.15.7:0/3209144414 wait complete. 2024-06-13T07:00:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.983+0000 7f40f2c00640 1 Processor -- start 2024-06-13T07:00:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 -- start start 2024-06-13T07:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec075980 0x7f40ec073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec109890 0x7f40ec071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec06be10 con 0x7f40ec109890 2024-06-13T07:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec06bc90 con 0x7f40ec0743a0 2024-06-13T07:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.989+0000 7f40f2c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec06bf90 con 0x7f40ec075980 2024-06-13T07:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40eb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec075980 0x7f40ec073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36150/0 (socket says 172.21.15.7:36150) 2024-06-13T07:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 -- 172.21.15.7:0/858242823 learned_addr learned my addr 172.21.15.7:0/858242823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 -- 172.21.15.7:0/858242823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec075980 msgr2=0x7f40ec073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec075980 0x7f40ec073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 -- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec109890 msgr2=0x7f40ec071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec109890 0x7f40ec071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 -- 172.21.15.7:0/858242823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec071bc0 con 0x7f40ec0743a0 2024-06-13T07:00:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.994+0000 7f40ebe00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b450 tx=0x7f40dc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.996+0000 7f40e8c00640 1 -- 172.21.15.7:0/858242823 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc015020 con 0x7f40ec0743a0 2024-06-13T07:00:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.996+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec071eb0 con 0x7f40ec0743a0 2024-06-13T07:00:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.996+0000 7f40e8c00640 1 -- 172.21.15.7:0/858242823 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc011d70 con 0x7f40ec0743a0 2024-06-13T07:00:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.996+0000 7f40e8c00640 1 -- 172.21.15.7:0/858242823 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00f340 con 0x7f40ec0743a0 2024-06-13T07:00:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:28.996+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1aac60 con 0x7f40ec0743a0 2024-06-13T07:00:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.003+0000 7f40e8c00640 1 -- 172.21.15.7:0/858242823 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f40dc018020 con 0x7f40ec0743a0 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.009+0000 7f40f59eb740 1 Processor -- start 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.010+0000 7f40f59eb740 1 -- start start 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.010+0000 7f40e8c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 0x7f40d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.011+0000 7f40f0c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 0x7f40d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558913112420 con 0x5589131b4760 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589130b9c50 con 0x5589131af5f0 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f59eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55891310e3e0 con 0x5589131aec00 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40e8c00640 1 -- 172.21.15.7:0/858242823 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f40dc0ceeb0 con 0x7f40ec0743a0 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36156/0 (socket says 172.21.15.7:36156) 2024-06-13T07:00:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40eb400640 1 -- 172.21.15.7:0/4119158485 learned_addr learned my addr 172.21.15.7:0/4119158485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.012+0000 7f40f0c00640 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.013+0000 7f40ebe00640 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 -- 172.21.15.7:0/4119158485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 msgr2=0x5589131b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 -- 172.21.15.7:0/4119158485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 msgr2=0x5589131b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 -- 172.21.15.7:0/4119158485 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558912e26210 con 0x5589131aec00 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.014+0000 7f40ebe00640 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131af020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00be10 tx=0x7f40dc09bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12365e8496157422 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.015+0000 7f40e1200640 1 -- 172.21.15.7:0/4119158485 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc09f780 con 0x5589131aec00 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.015+0000 7f40e1200640 1 -- 172.21.15.7:0/4119158485 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc0a4040 con 0x5589131aec00 2024-06-13T07:00:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.015+0000 7f40e1200640 1 -- 172.21.15.7:0/4119158485 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00eb50 con 0x5589131aec00 2024-06-13T07:00:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.015+0000 7f40f0c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 0x7f40d00a51c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005c40 tx=0x7f40e40127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 -- 172.21.15.7:0/4119158485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 msgr2=0x5589131af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131af020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00be10 tx=0x7f40dc09bad0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 -- 172.21.15.7:0/4119158485 shutdown_connections 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4119158485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131af020 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 -- 172.21.15.7:0/4119158485 >> 172.21.15.7:0/4119158485 conn(0x558913176fb0 msgr2=0x5589131aa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 -- 172.21.15.7:0/4119158485 shutdown_connections 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.016+0000 7f40f59eb740 1 -- 172.21.15.7:0/4119158485 wait complete. 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.017+0000 7f40f59eb740 1 -- start start 2024-06-13T07:00:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.017+0000 7f40f59eb740 1 Processor -- start 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131aec00 0x5589131b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131af5f0 0x5589131b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5589131b7710 con 0x5589131af5f0 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5589130cd0b0 con 0x5589131aec00 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.019+0000 7f40f59eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589131b7890 con 0x5589131b4760 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f59eb740 10 client.0 _ll_register_callbacks cb 0x5589130b8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40652/0 (socket says 172.21.15.7:40652) 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 -- 172.21.15.7:0/1925756198 learned_addr learned my addr 172.21.15.7:0/1925756198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40ebe00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131aec00 0x5589131b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 -- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131aec00 msgr2=0x5589131b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131aec00 0x5589131b8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 -- 172.21.15.7:0/1925756198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131af5f0 msgr2=0x5589131b94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131af5f0 0x5589131b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.020+0000 7f40f0c00640 1 -- 172.21.15.7:0/1925756198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558912ef2620 con 0x5589131b4760 2024-06-13T07:00:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.022+0000 7f40f0c00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40e40109c0 tx=0x7f40e4014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0edc406b00cafa4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.022+0000 7f40d5600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.023+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4023070 con 0x5589131b4760 2024-06-13T07:00:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.023+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f40e400fb40 con 0x5589131b4760 2024-06-13T07:00:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.023+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc0a4040 con 0x5589131b4760 2024-06-13T07:00:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.023+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40e402d050 con 0x5589131b4760 2024-06-13T07:00:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.024+0000 7f40f59eb740 10 client.25978 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.024+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5589131c38c0 con 0x5589131b4760 2024-06-13T07:00:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc0a4040 con 0x5589131b4760 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f40e401d410 con 0x5589131b4760 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 10 client.25978 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 20 client.25978 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5589131c5000 0x5589131c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 4 client.25978 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.025+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891311fb50 con 0x5589131c5000 2024-06-13T07:00:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.027+0000 7f40ebe00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5589131c5000 0x5589131c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.027+0000 7f40ebe00640 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5589131c5000 0x5589131c74a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366d63f235c4b6ba server_cookie=47161484d8e3d05 in_seq=0 out_seq=0 2024-06-13T07:00:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.027+0000 7f40d5600640 10 client.25978 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.028+0000 7f40d5600640 1 -- 172.21.15.7:0/1925756198 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55891311fb50 con 0x5589131c5000 2024-06-13T07:00:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.028+0000 7f40d5600640 10 client.25978 handle_command_reply: tid=0 2024-06-13T07:00:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.029+0000 7f40f59eb740 1 client.25978 shutdown 2024-06-13T07:00:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.029+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 msgr2=0x5589131c04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.029+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f40e40109c0 tx=0x7f40e4014040 comp rx=0 tx=0).stop 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 shutdown_connections 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5589131c5000 0x5589131c74a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131b4760 0x5589131c04e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131af5f0 0x5589131b94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 --2- 172.21.15.7:0/1925756198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131aec00 0x5589131b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40d5600640 0 client.25978 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 >> 172.21.15.7:0/1925756198 conn(0x558913176fb0 msgr2=0x5589131a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 shutdown_connections 2024-06-13T07:00:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 1 -- 172.21.15.7:0/1925756198 wait complete. 2024-06-13T07:00:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.031+0000 7f40f59eb740 20 client.25978 trim_cache size 0 max 16384 2024-06-13T07:00:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.071+0000 7f40f59eb740 1 Processor -- start 2024-06-13T07:00:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 -- start start 2024-06-13T07:00:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b7840 con 0x5589131b4760 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b50b0 con 0x5589131af5f0 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.073+0000 7f40f59eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b2f00 con 0x5589131aec00 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36176/0 (socket says 172.21.15.7:36176) 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 -- 172.21.15.7:0/4182571353 learned_addr learned my addr 172.21.15.7:0/4182571353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40f0c00640 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 -- 172.21.15.7:0/4182571353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 msgr2=0x5589131b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 -- 172.21.15.7:0/4182571353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 msgr2=0x5589131c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.074+0000 7f40eb400640 1 -- 172.21.15.7:0/4182571353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589131afe80 con 0x5589131af5f0 2024-06-13T07:00:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.075+0000 7f40eb400640 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40d8031610 tx=0x7f40d80040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea11aa82b3a9af server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.076+0000 7f40d7e00640 1 -- 172.21.15.7:0/4182571353 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8035040 con 0x5589131af5f0 2024-06-13T07:00:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.076+0000 7f40d7e00640 1 -- 172.21.15.7:0/4182571353 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d8005520 con 0x5589131af5f0 2024-06-13T07:00:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.076+0000 7f40d7e00640 1 -- 172.21.15.7:0/4182571353 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8031910 con 0x5589131af5f0 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 -- 172.21.15.7:0/4182571353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 msgr2=0x5589131b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40d8031610 tx=0x7f40d80040f0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 -- 172.21.15.7:0/4182571353 shutdown_connections 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131b0e50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 --2- 172.21.15.7:0/4182571353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 -- 172.21.15.7:0/4182571353 >> 172.21.15.7:0/4182571353 conn(0x558913176fb0 msgr2=0x5589131aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 -- 172.21.15.7:0/4182571353 shutdown_connections 2024-06-13T07:00:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.077+0000 7f40f59eb740 1 -- 172.21.15.7:0/4182571353 wait complete. 2024-06-13T07:00:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.078+0000 7f40f59eb740 1 -- start start 2024-06-13T07:00:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.078+0000 7f40f59eb740 1 Processor -- start 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b3570 con 0x5589131b4760 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b49e0 con 0x5589131af5f0 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1b3c40 con 0x5589131aec00 2024-06-13T07:00:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40f59eb740 10 client.0 _ll_register_callbacks cb 0x7f40e40120b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40ebe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:40666/0 (socket says 172.21.15.7:40666) 2024-06-13T07:00:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40eb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36188/0 (socket says 172.21.15.7:36188) 2024-06-13T07:00:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.079+0000 7f40ebe00640 1 -- 172.21.15.7:0/3741858144 learned_addr learned my addr 172.21.15.7:0/3741858144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 -- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 msgr2=0x5589131bdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 -- 172.21.15.7:0/3741858144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 msgr2=0x5589131bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 -- 172.21.15.7:0/3741858144 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589131bf580 con 0x5589131aec00 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40eb400640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40ebe00640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0109d0 tx=0x7f40dc0092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489724d2538b790 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.080+0000 7f40d5600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0a4040 con 0x5589131aec00 2024-06-13T07:00:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f40dc097ac0 con 0x5589131aec00 2024-06-13T07:00:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d8005520 con 0x5589131aec00 2024-06-13T07:00:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40dc0b4210 con 0x5589131aec00 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40f59eb740 10 client.25984 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.081+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5589131c0890 con 0x5589131aec00 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d8005520 con 0x5589131aec00 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f40dc09a820 con 0x5589131aec00 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 10 client.25984 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 20 client.25984 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558913238820 0x55891323ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 4 client.25984 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.082+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558913119ea0 con 0x558913238820 2024-06-13T07:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.083+0000 7f40eb400640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558913238820 0x55891323ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.083+0000 7f40d5600640 10 client.25984 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.083+0000 7f40eb400640 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558913238820 0x55891323ac10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5397ea4be6cffa50 server_cookie=315cafc432bcd096 in_seq=0 out_seq=0 2024-06-13T07:00:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.107+0000 7f40d5600640 1 -- 172.21.15.7:0/3741858144 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558913119ea0 con 0x558913238820 2024-06-13T07:00:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.107+0000 7f40d5600640 10 client.25984 handle_command_reply: tid=0 2024-06-13T07:00:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.107+0000 7f40f59eb740 1 client.25984 shutdown 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 msgr2=0x5589131bda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0109d0 tx=0x7f40dc0092a0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 shutdown_connections 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558913238820 0x55891323ac10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589131b4760 0x5589131bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5589131af5f0 0x5589131bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 --2- 172.21.15.7:0/3741858144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589131aec00 0x5589131bda40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.109+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 >> 172.21.15.7:0/3741858144 conn(0x558913176fb0 msgr2=0x5589131b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.110+0000 7f40d5600640 0 client.25984 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.110+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 shutdown_connections 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.110+0000 7f40f59eb740 1 -- 172.21.15.7:0/3741858144 wait complete. 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.110+0000 7f40f59eb740 20 client.25984 trim_cache size 0 max 16384 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:00:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:00:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.111+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 msgr2=0x7f40d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.111+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 0x7f40d00a51c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005c40 tx=0x7f40e40127c0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.111+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 msgr2=0x7f40ec072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.111+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b450 tx=0x7f40dc00b910 comp rx=0 tx=0).stop 2024-06-13T07:00:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 shutdown_connections 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f40d00a2d00 0x7f40d00a51c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec109890 0x7f40ec071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40ec075980 0x7f40ec073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 --2- 172.21.15.7:0/858242823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40ec0743a0 0x7f40ec072ee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 >> 172.21.15.7:0/858242823 conn(0x7f40ec0fcf70 msgr2=0x7f40ec105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 shutdown_connections 2024-06-13T07:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:29.112+0000 7f40f2c00640 1 -- 172.21.15.7:0/858242823 wait complete. 2024-06-13T07:00:29.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:00:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:29 smithi007 ceph-mon[32077]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 612 op/s 2024-06-13T07:00:29.573 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:29 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1647924605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:29 smithi189 ceph-mon[36616]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 612 op/s 2024-06-13T07:00:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:29 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1647924605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:29 smithi175 ceph-mon[36886]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.2 MiB/s wr, 612 op/s 2024-06-13T07:00:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:29 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1647924605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:00:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:31 smithi189 ceph-mon[36616]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 MiB/s wr, 507 op/s 2024-06-13T07:00:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:31 smithi175 ceph-mon[36886]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 MiB/s wr, 507 op/s 2024-06-13T07:00:31.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:31 smithi007 ceph-mon[32077]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 MiB/s wr, 507 op/s 2024-06-13T07:00:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 270.0 s, 561.6 tps, lat 180.861 ms stddev 198.550 2024-06-13T07:00:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:33 smithi189 ceph-mon[36616]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-06-13T07:00:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:33 smithi175 ceph-mon[36886]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-06-13T07:00:33.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:33 smithi007 ceph-mon[32077]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 611 op/s 2024-06-13T07:00:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:35 smithi189 ceph-mon[36616]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.5 MiB/s wr, 630 op/s 2024-06-13T07:00:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:35 smithi175 ceph-mon[36886]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.5 MiB/s wr, 630 op/s 2024-06-13T07:00:35.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:35 smithi007 ceph-mon[32077]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.5 MiB/s wr, 630 op/s 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 -- 172.21.15.7:0/269683942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4069d50 msgr2=0x7f53f4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 --2- 172.21.15.7:0/269683942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4069d50 0x7f53f4107000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009990 tx=0x7f53e8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 -- 172.21.15.7:0/269683942 shutdown_connections 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 --2- 172.21.15.7:0/269683942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4100ac0 0x7f53f4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 --2- 172.21.15.7:0/269683942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f4106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 --2- 172.21.15.7:0/269683942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4069d50 0x7f53f4107000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 -- 172.21.15.7:0/269683942 >> 172.21.15.7:0/269683942 conn(0x7f53f4077a00 msgr2=0x7f53f40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.256+0000 7f53fc000700 1 -- 172.21.15.7:0/269683942 shutdown_connections 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.257+0000 7f53fc000700 1 -- 172.21.15.7:0/269683942 wait complete. 2024-06-13T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.258+0000 7f53fc000700 1 Processor -- start 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.258+0000 7f53fc000700 1 -- start start 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4100ac0 0x7f53f419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e5d0 con 0x7f53f4100ac0 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e450 con 0x7f53f40698e0 2024-06-13T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.259+0000 7f53fc000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410e750 con 0x7f53f4069d50 2024-06-13T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.262+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.262+0000 7f53f9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.262+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50926/0 (socket says 172.21.15.7:50926) 2024-06-13T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.262+0000 7f53f8c00700 1 -- 172.21.15.7:0/2855473437 learned_addr learned my addr 172.21.15.7:0/2855473437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 -- 172.21.15.7:0/2855473437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 msgr2=0x7f53f419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 -- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4100ac0 msgr2=0x7f53f419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4100ac0 0x7f53f419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 -- 172.21.15.7:0/2855473437 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f419fea0 con 0x7f53f4069d50 2024-06-13T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f9600700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.263+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.267+0000 7f53f2000700 1 -- 172.21.15.7:0/2855473437 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8027070 con 0x7f53f4069d50 2024-06-13T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.267+0000 7f53f2000700 1 -- 172.21.15.7:0/2855473437 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e80056d0 con 0x7f53f4069d50 2024-06-13T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.267+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a0130 con 0x7f53f4069d50 2024-06-13T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.267+0000 7f53f2000700 1 -- 172.21.15.7:0/2855473437 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8022db0 con 0x7f53f4069d50 2024-06-13T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.267+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f4110cb0 con 0x7f53f4069d50 2024-06-13T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.272+0000 7f53f2000700 1 -- 172.21.15.7:0/2855473437 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f53e8009d10 con 0x7f53f4069d50 2024-06-13T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.273+0000 7f54069c5b80 1 Processor -- start 2024-06-13T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.274+0000 7f54069c5b80 1 -- start start 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.274+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c894c0 0x559623c82d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c83690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f54069c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559623b80a10 con 0x559623c83250 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f54069c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559623bc3b00 con 0x559623c894c0 2024-06-13T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f54069c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559623b7e8f0 con 0x559623c88ba0 2024-06-13T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f53fa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f53fa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50940/0 (socket says 172.21.15.7:50940) 2024-06-13T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f53fa000700 1 -- 172.21.15.7:0/3053895750 learned_addr learned my addr 172.21.15.7:0/3053895750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.275+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c83690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.276+0000 7f53fa000700 1 -- 172.21.15.7:0/3053895750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c894c0 msgr2=0x559623c82d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.276+0000 7f53fa000700 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c894c0 0x559623c82d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.276+0000 7f53fa000700 1 -- 172.21.15.7:0/3053895750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 msgr2=0x559623c83690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.277+0000 7f53f2000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 0x7f53dc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.277+0000 7f53fa000700 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c83690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.277+0000 7f53fa000700 1 -- 172.21.15.7:0/3053895750 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559623b6bf90 con 0x559623c88ba0 2024-06-13T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.277+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c83690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.278+0000 7f53f2000700 1 -- 172.21.15.7:0/2855473437 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f53e80d0070 con 0x7f53f4069d50 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.278+0000 7f53fa000700 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00e380 tx=0x7f53ec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3751cc1e5919585c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.278+0000 7f53f9600700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 0x7f53dc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.278+0000 7f53e2a00700 1 -- 172.21.15.7:0/3053895750 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec004bb0 con 0x559623c88ba0 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.278+0000 7f53e2a00700 1 -- 172.21.15.7:0/3053895750 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53ec004d50 con 0x559623c88ba0 2024-06-13T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.279+0000 7f53e2a00700 1 -- 172.21.15.7:0/3053895750 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec004480 con 0x559623c88ba0 2024-06-13T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.280+0000 7f54069c5b80 1 -- 172.21.15.7:0/3053895750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 msgr2=0x559623c88f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.280+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00e380 tx=0x7f53ec00b040 comp rx=0 tx=0).stop 2024-06-13T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.281+0000 7f54069c5b80 1 -- 172.21.15.7:0/3053895750 shutdown_connections 2024-06-13T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.281+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c83690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.281+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c88f80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.281+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3053895750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c894c0 0x559623c82d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.281+0000 7f54069c5b80 1 -- 172.21.15.7:0/3053895750 >> 172.21.15.7:0/3053895750 conn(0x559623b9e170 msgr2=0x559623c6cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.284+0000 7f54069c5b80 1 -- 172.21.15.7:0/3053895750 shutdown_connections 2024-06-13T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.284+0000 7f54069c5b80 1 -- 172.21.15.7:0/3053895750 wait complete. 2024-06-13T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.286+0000 7f54069c5b80 1 -- start start 2024-06-13T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.287+0000 7f54069c5b80 1 Processor -- start 2024-06-13T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.288+0000 7f53f9600700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 0x7f53dc0a52b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53e4011490 tx=0x7f53e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.288+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c8f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c894c0 0x559623c8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50944/0 (socket says 172.21.15.7:50944) 2024-06-13T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f53f9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c8f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.289+0000 7f53f9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c8f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54216/0 (socket says 172.21.15.7:54216) 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 -- 172.21.15.7:0/1706632775 learned_addr learned my addr 172.21.15.7:0/1706632775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559623b5ff60 con 0x559623c894c0 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559623bbfac0 con 0x559623c83250 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559623c8fe40 con 0x559623c88ba0 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f54069c5b80 10 client.0 _ll_register_callbacks cb 0x559623b59380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 -- 172.21.15.7:0/1706632775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 msgr2=0x559623c8f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c8f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 -- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c894c0 msgr2=0x559623c8c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c894c0 0x559623c8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.290+0000 7f53f8c00700 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559623b128d0 con 0x559623c88ba0 2024-06-13T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.291+0000 7f53f8c00700 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53e80098e0 tx=0x7f53e8022750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e6808f146a9bfd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.291+0000 7f53d7400700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.291+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e80bc040 con 0x559623c88ba0 2024-06-13T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.291+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f53e80a4d10 con 0x559623c88ba0 2024-06-13T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.291+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53ec004d50 con 0x559623c88ba0 2024-06-13T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.293+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f53e809c100 con 0x559623c88ba0 2024-06-13T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.293+0000 7f54069c5b80 10 client.25996 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.294+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559623c9a520 con 0x559623c88ba0 2024-06-13T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.295+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53ec004d50 con 0x559623c88ba0 2024-06-13T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.295+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f53e8085b50 con 0x559623c88ba0 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 10 client.25996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 20 client.25996 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623c9cc40 0x559623c9f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 4 client.25996 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.297+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559623bd1000 con 0x559623c9cc40 2024-06-13T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.298+0000 7f53f9600700 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623c9cc40 0x559623c9f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.303+0000 7f53f9600700 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623c9cc40 0x559623c9f060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4d33409d8b0ef90 server_cookie=6d05c576f7b8b179 in_seq=0 out_seq=0 2024-06-13T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.303+0000 7f53d7400700 10 client.25996 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.307+0000 7f53d7400700 1 -- 172.21.15.7:0/1706632775 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559623bd1000 con 0x559623c9cc40 2024-06-13T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.307+0000 7f53d7400700 10 client.25996 handle_command_reply: tid=0 2024-06-13T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.307+0000 7f54069c5b80 1 client.25996 shutdown 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 msgr2=0x559623c8c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53e80098e0 tx=0x7f53e8022750 comp rx=0 tx=0).stop 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 shutdown_connections 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c894c0 0x559623c8c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623c9cc40 0x559623c9f060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c8f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 --2- 172.21.15.7:0/1706632775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c88ba0 0x559623c8c4b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 >> 172.21.15.7:0/1706632775 conn(0x559623b9e170 msgr2=0x559623c66790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.308+0000 7f53d7400700 0 client.25996 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.309+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 shutdown_connections 2024-06-13T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.309+0000 7f54069c5b80 1 -- 172.21.15.7:0/1706632775 wait complete. 2024-06-13T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.309+0000 7f54069c5b80 20 client.25996 trim_cache size 0 max 16384 2024-06-13T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 Processor -- start 2024-06-13T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 -- start start 2024-06-13T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623c8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8011270 con 0x559623c83250 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800eaa0 con 0x559623c88ba0 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.342+0000 7f54069c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800c8f0 con 0x559623c894c0 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.343+0000 7f53f9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.343+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.343+0000 7f53f8c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54228/0 (socket says 172.21.15.7:54228) 2024-06-13T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.343+0000 7f53f8c00700 1 -- 172.21.15.7:0/2719627907 learned_addr learned my addr 172.21.15.7:0/2719627907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 -- 172.21.15.7:0/2719627907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 msgr2=0x559623c8b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53fa000700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623c8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623c8b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 -- 172.21.15.7:0/2719627907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 msgr2=0x559623c67130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c67130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 -- 172.21.15.7:0/2719627907 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559623c8aa20 con 0x559623c88ba0 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53fa000700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623c8b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f9600700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.344+0000 7f53f8c00700 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53e8085ef0 tx=0x7f53e809aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=313bf2efe8a5bf2d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.347+0000 7f53e2000700 1 -- 172.21.15.7:0/2719627907 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e80bc070 con 0x559623c88ba0 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.347+0000 7f53e2000700 1 -- 172.21.15.7:0/2719627907 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e809c5c0 con 0x559623c88ba0 2024-06-13T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.347+0000 7f53e2000700 1 -- 172.21.15.7:0/2719627907 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8098860 con 0x559623c88ba0 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 -- 172.21.15.7:0/2719627907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 msgr2=0x559623c89fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53e8085ef0 tx=0x7f53e809aad0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 -- 172.21.15.7:0/2719627907 shutdown_connections 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623c8b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c83250 0x559623c67130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 --2- 172.21.15.7:0/2719627907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c88ba0 0x559623c89fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 -- 172.21.15.7:0/2719627907 >> 172.21.15.7:0/2719627907 conn(0x559623b9e170 msgr2=0x559623c70cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 -- 172.21.15.7:0/2719627907 shutdown_connections 2024-06-13T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.348+0000 7f54069c5b80 1 -- 172.21.15.7:0/2719627907 wait complete. 2024-06-13T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.349+0000 7f54069c5b80 1 -- start start 2024-06-13T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.349+0000 7f54069c5b80 1 Processor -- start 2024-06-13T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 0x559623ca4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623ca4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800cf60 con 0x559623c88ba0 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800e3d0 con 0x559623c83250 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f54069c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800d630 con 0x559623c894c0 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f53f9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f53f9600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54230/0 (socket says 172.21.15.7:54230) 2024-06-13T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f53f9600700 1 -- 172.21.15.7:0/3404317856 learned_addr learned my addr 172.21.15.7:0/3404317856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.350+0000 7f53fa000700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623ca4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.351+0000 7f53f9600700 1 -- 172.21.15.7:0/3404317856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 msgr2=0x559623ca4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.351+0000 7f53f9600700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623ca4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.351+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 0x559623ca4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.351+0000 7f53f9600700 1 -- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 msgr2=0x559623ca4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.351+0000 7f54069c5b80 10 client.0 _ll_register_callbacks cb 0x559623b59380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.352+0000 7f53f9600700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 0x559623ca4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.352+0000 7f53f9600700 1 -- 172.21.15.7:0/3404317856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559623ca4f10 con 0x559623c83250 2024-06-13T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.352+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e809c5c0 con 0x559623c83250 2024-06-13T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.352+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 0x559623ca4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.354+0000 7f53f9600700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559623c9f8a0 tx=0x7f53e4013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8740794796b4fb72 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.354+0000 7f53d7400700 10 client.26012 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.354+0000 7f53d7400700 1 -- 172.21.15.7:0/3404317856 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4036060 con 0x559623c83250 2024-06-13T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.355+0000 7f53d7400700 1 -- 172.21.15.7:0/3404317856 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f53e4029600 con 0x559623c83250 2024-06-13T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.355+0000 7f53d7400700 1 -- 172.21.15.7:0/3404317856 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f53e4029800 con 0x559623c83250 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.355+0000 7f54069c5b80 10 client.26012 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.355+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=34}) -- 0x559623d07780 con 0x559623c83250 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f53d7400700 1 -- 172.21.15.7:0/3404317856 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f53e40299a0 con 0x559623c83250 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 10 client.26012 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 20 client.26012 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623d07d80 0x559623d0a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 4 client.26012 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.356+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559623bcb580 con 0x559623d07d80 2024-06-13T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.357+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623d07d80 0x559623d0a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.361+0000 7f53f8c00700 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623d07d80 0x559623d0a1c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b28576a9e6500f73 server_cookie=d6cd4b8b2503a5a6 in_seq=0 out_seq=0 2024-06-13T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.361+0000 7f53d7400700 10 client.26012 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.362+0000 7f53d7400700 1 -- 172.21.15.7:0/3404317856 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559623bcb580 con 0x559623d07d80 2024-06-13T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.362+0000 7f53d7400700 10 client.26012 handle_command_reply: tid=0 2024-06-13T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.362+0000 7f54069c5b80 1 client.26012 shutdown 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.362+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 msgr2=0x559623c95810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.362+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559623c9f8a0 tx=0x7f53e4013040 comp rx=0 tx=0).stop 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 shutdown_connections 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559623c894c0 0x559623ca4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559623d07d80 0x559623d0a1c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559623c83250 0x559623c95810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 --2- 172.21.15.7:0/3404317856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559623c88ba0 0x559623ca4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f53d7400700 0 client.26012 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 >> 172.21.15.7:0/3404317856 conn(0x559623b9e170 msgr2=0x559623c70cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 shutdown_connections 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.363+0000 7f54069c5b80 1 -- 172.21.15.7:0/3404317856 wait complete. 2024-06-13T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.364+0000 7f54069c5b80 20 client.26012 trim_cache size 0 max 16384 2024-06-13T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 msgr2=0x7f53dc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 0x7f53dc0a52b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53e4011490 tx=0x7f53e4009260 comp rx=0 tx=0).stop 2024-06-13T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 msgr2=0x7f53f419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f53e8000c00 tx=0x7f53e800bab0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 shutdown_connections 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53f4100ac0 0x7f53f419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53dc0a2df0 0x7f53dc0a52b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53f40698e0 0x7f53f419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 --2- 172.21.15.7:0/2855473437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f4069d50 0x7f53f419b420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.365+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 >> 172.21.15.7:0/2855473437 conn(0x7f53f4077a00 msgr2=0x7f53f410fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.367+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 shutdown_connections 2024-06-13T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:36.367+0000 7f53fc000700 1 -- 172.21.15.7:0/2855473437 wait complete. 2024-06-13T07:00:37.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:00:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:37 smithi189 ceph-mon[36616]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-06-13T07:00:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:37 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:37 smithi175 ceph-mon[36886]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-06-13T07:00:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:37 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:37.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:37 smithi007 ceph-mon[32077]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-06-13T07:00:37.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:37 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:00:37.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 275.0 s, 459.2 tps, lat 210.011 ms stddev 221.469 2024-06-13T07:00:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:39 smithi189 ceph-mon[36616]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 764 op/s 2024-06-13T07:00:39.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:39.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:39.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:00:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:39 smithi175 ceph-mon[36886]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 764 op/s 2024-06-13T07:00:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:00:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:39 smithi007 ceph-mon[32077]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 MiB/s wr, 764 op/s 2024-06-13T07:00:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:00:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:00:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.650+0000 7f6142800700 1 -- 172.21.15.7:0/2945955223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c06a6e0 msgr2=0x7f613c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 --2- 172.21.15.7:0/2945955223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c06a6e0 0x7f613c06ab40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f612c0098f0 tx=0x7f612c009c00 comp rx=0 tx=0).stop 2024-06-13T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 -- 172.21.15.7:0/2945955223 shutdown_connections 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 --2- 172.21.15.7:0/2945955223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c104ed0 0x7f613c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 --2- 172.21.15.7:0/2945955223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c1073c0 0x7f613c06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 --2- 172.21.15.7:0/2945955223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c06a6e0 0x7f613c06ab40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 -- 172.21.15.7:0/2945955223 >> 172.21.15.7:0/2945955223 conn(0x7f613c078340 msgr2=0x7f613c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 -- 172.21.15.7:0/2945955223 shutdown_connections 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.651+0000 7f6142800700 1 -- 172.21.15.7:0/2945955223 wait complete. 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.653+0000 7f6142800700 1 Processor -- start 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.653+0000 7f6142800700 1 -- start start 2024-06-13T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.656+0000 7f6142800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f6142800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c104ed0 0x7f613c06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f6142800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c1073c0 0x7f613c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54238/0 (socket says 172.21.15.7:54238) 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f613b400700 1 -- 172.21.15.7:0/2390098035 learned_addr learned my addr 172.21.15.7:0/2390098035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f613c118db0 con 0x7f613c1073c0 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f613c118c30 con 0x7f613c06a6e0 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.657+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f613c118f30 con 0x7f613c104ed0 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.658+0000 7f613b400700 1 -- 172.21.15.7:0/2390098035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c104ed0 msgr2=0x7f613c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.658+0000 7f613b400700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c104ed0 0x7f613c06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.658+0000 7f613b400700 1 -- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c1073c0 msgr2=0x7f613c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.658+0000 7f613b400700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c1073c0 0x7f613c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.658+0000 7f613b400700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c06d910 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.659+0000 7f613b400700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f612800d980 tx=0x7f612800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.660+0000 7f6133e00700 1 -- 172.21.15.7:0/2390098035 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128016070 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.660+0000 7f6133e00700 1 -- 172.21.15.7:0/2390098035 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612800bdc0 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.660+0000 7f6133e00700 1 -- 172.21.15.7:0/2390098035 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128004cf0 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.660+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c071800 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.661+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c071cf0 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.673+0000 7f614d175b80 1 Processor -- start 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.675+0000 7f614d175b80 1 -- start start 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.675+0000 7f614d175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.677+0000 7f613be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.677+0000 7f613be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43256/0 (socket says 172.21.15.7:43256) 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.677+0000 7f6133e00700 1 -- 172.21.15.7:0/2390098035 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f612800b8e0 con 0x7f613c06a6e0 2024-06-13T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f614d175b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f614d175b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 0x564e672b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f614d175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564e671c2a10 con 0x564e672b9b90 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f614d175b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564e67205b00 con 0x564e672ba4b0 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f614d175b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564e671c08f0 con 0x564e672b4240 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.681+0000 7f613be00700 1 -- 172.21.15.7:0/1030805780 learned_addr learned my addr 172.21.15.7:0/1030805780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.682+0000 7f613b400700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.682+0000 7f613aa00700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 0x564e672b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.683+0000 7f6133e00700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 0x7f61200a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.682+0000 7f613be00700 1 -- 172.21.15.7:0/1030805780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 msgr2=0x564e672b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.683+0000 7f613be00700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 0x564e672b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.683+0000 7f613be00700 1 -- 172.21.15.7:0/1030805780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 msgr2=0x564e672b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.683+0000 7f613be00700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.683+0000 7f613be00700 1 -- 172.21.15.7:0/1030805780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e671adf90 con 0x564e672b9b90 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.684+0000 7f6133e00700 1 -- 172.21.15.7:0/2390098035 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f61280cf060 con 0x7f613c06a6e0 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.684+0000 7f613be00700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f613400e4a0 tx=0x7f613400e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f34294316328e35 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.685+0000 7f613be00700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 0x7f61200a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.685+0000 7f613b400700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.686+0000 7f6130c00700 1 -- 172.21.15.7:0/1030805780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134009360 con 0x564e672b9b90 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.686+0000 7f6130c00700 1 -- 172.21.15.7:0/1030805780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340075c0 con 0x564e672b9b90 2024-06-13T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.686+0000 7f6130c00700 1 -- 172.21.15.7:0/1030805780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134004360 con 0x564e672b9b90 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.688+0000 7f614d175b80 1 -- 172.21.15.7:0/1030805780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 msgr2=0x564e672b9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.688+0000 7f614d175b80 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f613400e4a0 tx=0x7f613400e7b0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.688+0000 7f613be00700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 0x7f61200a52b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6134018000 tx=0x7f61340045e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.686+0000 7f613aa00700 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 0x564e672b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 -- 172.21.15.7:0/1030805780 shutdown_connections 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b4240 0x564e672b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672b9f70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 --2- 172.21.15.7:0/1030805780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 -- 172.21.15.7:0/1030805780 >> 172.21.15.7:0/1030805780 conn(0x564e671e0170 msgr2=0x564e672aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.689+0000 7f614d175b80 1 -- 172.21.15.7:0/1030805780 shutdown_connections 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.690+0000 7f614d175b80 1 -- 172.21.15.7:0/1030805780 wait complete. 2024-06-13T07:00:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.692+0000 7f614d175b80 1 -- start start 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.693+0000 7f614d175b80 1 Processor -- start 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.694+0000 7f614d175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 0x564e672c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.694+0000 7f614d175b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.694+0000 7f614d175b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f614d175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564e671a1f60 con 0x564e672b4240 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f614d175b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564e67201ac0 con 0x564e672ba4b0 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f614d175b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564e672c0e90 con 0x564e672b9b90 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f613aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f613be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.695+0000 7f613be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54252/0 (socket says 172.21.15.7:54252) 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f613be00700 1 -- 172.21.15.7:0/3638982673 learned_addr learned my addr 172.21.15.7:0/3638982673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f613b400700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 0x564e672c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f614d175b80 10 client.0 _ll_register_callbacks cb 0x564e6719b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f613be00700 1 -- 172.21.15.7:0/3638982673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 msgr2=0x564e672bd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f613be00700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672bd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.696+0000 7f613be00700 1 -- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 msgr2=0x564e672c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.697+0000 7f613be00700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 0x564e672c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.697+0000 7f613be00700 1 -- 172.21.15.7:0/3638982673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e671548d0 con 0x564e672ba4b0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.697+0000 7f613b400700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 0x564e672c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.697+0000 7f613be00700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6134009820 tx=0x7f6134019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b224a9d6456967 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.697+0000 7f6125600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.698+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134030020 con 0x564e672ba4b0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.698+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6134004870 con 0x564e672ba4b0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.698+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61340075c0 con 0x564e672ba4b0 2024-06-13T07:00:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.699+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f613403b050 con 0x564e672ba4b0 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.699+0000 7f614d175b80 10 client.26024 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.699+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564e672cb520 con 0x564e672ba4b0 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.700+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340075c0 con 0x564e672ba4b0 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6134007770 con 0x564e672ba4b0 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 10 client.26024 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 10 client.26024 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 20 client.26024 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e672cdb00 0x564e672cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 4 client.26024 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e67213000 con 0x564e672cdb00 2024-06-13T07:00:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.703+0000 7f613b400700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e672cdb00 0x564e672cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.704+0000 7f613b400700 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e672cdb00 0x564e672cfef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3eaa668f4ff9146 server_cookie=135301c28fe55f32 in_seq=0 out_seq=0 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.704+0000 7f6125600700 10 client.26024 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.706+0000 7f6125600700 1 -- 172.21.15.7:0/3638982673 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564e67213000 con 0x564e672cdb00 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.706+0000 7f6125600700 10 client.26024 handle_command_reply: tid=0 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.706+0000 7f614d175b80 1 client.26024 shutdown 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 msgr2=0x564e672bda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6134009820 tx=0x7f6134019000 comp rx=0 tx=0).stop 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 shutdown_connections 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672ba4b0 0x564e672bda70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b4240 0x564e672c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e672cdb00 0x564e672cfef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 --2- 172.21.15.7:0/3638982673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672bd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.710+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 >> 172.21.15.7:0/3638982673 conn(0x564e671e0170 msgr2=0x564e672a8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.711+0000 7f6125600700 0 client.26024 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.711+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 shutdown_connections 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.712+0000 7f614d175b80 1 -- 172.21.15.7:0/3638982673 wait complete. 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.712+0000 7f614d175b80 20 client.26024 trim_cache size 0 max 16384 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.744+0000 7f614d175b80 1 Processor -- start 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.744+0000 7f614d175b80 1 -- start start 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672ba4b0 0x564e672a9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bfa30 con 0x564e672ba4b0 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bd260 con 0x564e672b4240 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f614d175b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bb0b0 con 0x564e672b9b90 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54258/0 (socket says 172.21.15.7:54258) 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 -- 172.21.15.7:0/2492552820 learned_addr learned my addr 172.21.15.7:0/2492552820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613aa00700 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 -- 172.21.15.7:0/2492552820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 msgr2=0x564e672af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 -- 172.21.15.7:0/2492552820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672ba4b0 msgr2=0x564e672a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672ba4b0 0x564e672a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.745+0000 7f613b400700 1 -- 172.21.15.7:0/2492552820 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e672cfb10 con 0x564e672b4240 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.747+0000 7f613b400700 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f612800d950 tx=0x7f61280d1430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8aa845723399f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.748+0000 7f6127e00700 1 -- 172.21.15.7:0/2492552820 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128014030 con 0x564e672b4240 2024-06-13T07:00:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.748+0000 7f6127e00700 1 -- 172.21.15.7:0/2492552820 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612809bc20 con 0x564e672b4240 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.748+0000 7f6127e00700 1 -- 172.21.15.7:0/2492552820 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612800bbd0 con 0x564e672b4240 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 -- 172.21.15.7:0/2492552820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 msgr2=0x564e672b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f612800d950 tx=0x7f61280d1430 comp rx=0 tx=0).stop 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 -- 172.21.15.7:0/2492552820 shutdown_connections 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672ba4b0 0x564e672a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672b3d00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 --2- 172.21.15.7:0/2492552820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672b9b90 0x564e672af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 -- 172.21.15.7:0/2492552820 >> 172.21.15.7:0/2492552820 conn(0x564e671e0170 msgr2=0x564e672aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 -- 172.21.15.7:0/2492552820 shutdown_connections 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.749+0000 7f614d175b80 1 -- 172.21.15.7:0/2492552820 wait complete. 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.750+0000 7f614d175b80 1 -- start start 2024-06-13T07:00:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.750+0000 7f614d175b80 1 Processor -- start 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.751+0000 7f614d175b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f613b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54264/0 (socket says 172.21.15.7:54264) 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bb720 con 0x564e672b9b90 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bcb90 con 0x564e672b4240 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1bbdf0 con 0x564e672ba4b0 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.752+0000 7f614d175b80 10 client.0 _ll_register_callbacks cb 0x564e6719b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51002/0 (socket says 172.21.15.7:51002) 2024-06-13T07:00:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 -- 172.21.15.7:0/285675491 learned_addr learned my addr 172.21.15.7:0/285675491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613aa00700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 -- 172.21.15.7:0/285675491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 msgr2=0x564e672d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672d59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 -- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 msgr2=0x564e672c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.753+0000 7f613be00700 1 -- 172.21.15.7:0/285675491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e672c6c50 con 0x564e672ba4b0 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.754+0000 7f613aa00700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.754+0000 7f6125600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.754+0000 7f613be00700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6134007c10 tx=0x7f61340094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b33734390dc2fbf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.755+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134030070 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.755+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f613401a040 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.755+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612809bc20 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.757+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f613400ee60 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.757+0000 7f614d175b80 10 client.26011 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.757+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564e67337520 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.757+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612809bc20 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.758+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f613402b590 con 0x564e672ba4b0 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 10 client.26011 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 10 client.26011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:00:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 20 client.26011 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e67337ab0 0x564e67339ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 4 client.26011 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e6720d580 con 0x564e67337ab0 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.759+0000 7f613b400700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e67337ab0 0x564e67339ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.760+0000 7f613b400700 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e67337ab0 0x564e67339ed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318b7d5bb378c511 server_cookie=d78a7e5b2cb4c688 in_seq=0 out_seq=0 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.760+0000 7f6125600700 10 client.26011 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.761+0000 7f6125600700 1 -- 172.21.15.7:0/285675491 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e6720d580 con 0x564e67337ab0 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.761+0000 7f6125600700 10 client.26011 handle_command_reply: tid=0 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.761+0000 7f614d175b80 1 client.26011 shutdown 2024-06-13T07:00:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 msgr2=0x564e672c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6134007c10 tx=0x7f61340094f0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.063 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:00:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 shutdown_connections 2024-06-13T07:00:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x564e67337ab0 0x564e67339ed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e672ba4b0 0x564e672c66c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564e672b4240 0x564e672d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 --2- 172.21.15.7:0/285675491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e672b9b90 0x564e672c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.762+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 >> 172.21.15.7:0/285675491 conn(0x564e671e0170 msgr2=0x564e672ba890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.763+0000 7f6125600700 0 client.26011 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.763+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 shutdown_connections 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.763+0000 7f614d175b80 1 -- 172.21.15.7:0/285675491 wait complete. 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.763+0000 7f614d175b80 20 client.26011 trim_cache size 0 max 16384 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 msgr2=0x7f61200a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 0x7f61200a52b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6134018000 tx=0x7f61340045e0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 msgr2=0x7f613c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f612800d980 tx=0x7f612800dd40 comp rx=0 tx=0).stop 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 shutdown_connections 2024-06-13T07:00:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c1073c0 0x7f613c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f61200a2df0 0x7f61200a52b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f613c06a6e0 0x7f613c073cd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 --2- 172.21.15.7:0/2390098035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613c104ed0 0x7f613c06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.766+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 >> 172.21.15.7:0/2390098035 conn(0x7f613c078340 msgr2=0x7f613c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.767+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 shutdown_connections 2024-06-13T07:00:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:40.767+0000 7f6142800700 1 -- 172.21.15.7:0/2390098035 wait complete. 2024-06-13T07:00:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:41 smithi189 ceph-mon[36616]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-06-13T07:00:41.608 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:41 smithi007 ceph-mon[32077]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-06-13T07:00:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:41 smithi175 ceph-mon[36886]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.2 MiB/s wr, 627 op/s 2024-06-13T07:00:42.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:00:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.348+0000 7f90b8a00640 1 -- 172.21.15.7:0/1488108148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4104c30 msgr2=0x7f90b4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.348+0000 7f90b8a00640 1 --2- 172.21.15.7:0/1488108148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4104c30 0x7f90b4107020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009900 tx=0x7f90a002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.353+0000 7f90b8a00640 1 -- 172.21.15.7:0/1488108148 shutdown_connections 2024-06-13T07:00:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.353+0000 7f90b8a00640 1 --2- 172.21.15.7:0/1488108148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b4107560 0x7f90b4109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.353+0000 7f90b8a00640 1 --2- 172.21.15.7:0/1488108148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4104c30 0x7f90b4107020 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009900 tx=0x7f90a002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.353+0000 7f90b8a00640 1 --2- 172.21.15.7:0/1488108148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b4102300 0x7f90b41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.353+0000 7f90b8a00640 1 -- 172.21.15.7:0/1488108148 >> 172.21.15.7:0/1488108148 conn(0x7f90b40fbc90 msgr2=0x7f90b40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.356+0000 7f90b8a00640 1 -- 172.21.15.7:0/1488108148 shutdown_connections 2024-06-13T07:00:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.357+0000 7f90b8a00640 1 -- 172.21.15.7:0/1488108148 wait complete. 2024-06-13T07:00:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.358+0000 7f90b8a00640 1 Processor -- start 2024-06-13T07:00:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 -- start start 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4102300 0x7f90b4195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b4107560 0x7f90b41960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410d4f0 con 0x7f90b419b130 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410d370 con 0x7f90b4107560 2024-06-13T07:00:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.359+0000 7f90b8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410d670 con 0x7f90b4102300 2024-06-13T07:00:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.361+0000 7f90b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.361+0000 7f90b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54632/0 (socket says 172.21.15.7:54632) 2024-06-13T07:00:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.361+0000 7f90b3200640 1 -- 172.21.15.7:0/4112355984 learned_addr learned my addr 172.21.15.7:0/4112355984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.362+0000 7f90b3200640 1 -- 172.21.15.7:0/4112355984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4102300 msgr2=0x7f90b4195b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:00:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.362+0000 7f90b3200640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4102300 0x7f90b4195b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.362+0000 7f90b3200640 1 -- 172.21.15.7:0/4112355984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b4107560 msgr2=0x7f90b41960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.362+0000 7f90b3200640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b4107560 0x7f90b41960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.362+0000 7f90b3200640 1 -- 172.21.15.7:0/4112355984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b419dbb0 con 0x7f90b419b130 2024-06-13T07:00:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.363+0000 7f90b3200640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009fd0 tx=0x7f90a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.363+0000 7f90ab400640 1 -- 172.21.15.7:0/4112355984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4011830 con 0x7f90b419b130 2024-06-13T07:00:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.364+0000 7f90ab400640 1 -- 172.21.15.7:0/4112355984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4016070 con 0x7f90b419b130 2024-06-13T07:00:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.364+0000 7f90ab400640 1 -- 172.21.15.7:0/4112355984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f2e0 con 0x7f90b419b130 2024-06-13T07:00:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.364+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b419dea0 con 0x7f90b419b130 2024-06-13T07:00:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.364+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b419e420 con 0x7f90b419b130 2024-06-13T07:00:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.373+0000 7f90bbdfd740 1 Processor -- start 2024-06-13T07:00:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.373+0000 7f90ab400640 1 -- 172.21.15.7:0/4112355984 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f90a401b460 con 0x7f90b419b130 2024-06-13T07:00:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.375+0000 7f90ab400640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 0x7f90980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.376+0000 7f90ab400640 1 -- 172.21.15.7:0/4112355984 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a40ceac0 con 0x7f90b419b130 2024-06-13T07:00:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.376+0000 7f90b1e00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 0x7f90980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.376+0000 7f90bbdfd740 1 -- start start 2024-06-13T07:00:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.377+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721912de0 0x55c721913200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.377+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:59998/0 (socket says 172.21.15.7:59998) 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90b2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721912de0 0x55c721913200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90b1e00640 1 -- 172.21.15.7:0/280222960 learned_addr learned my addr 172.21.15.7:0/280222960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c72191add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c721876420 con 0x55c721918940 2024-06-13T07:00:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.378+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c72181dc50 con 0x55c7219137d0 2024-06-13T07:00:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.379+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7218723e0 con 0x55c721912de0 2024-06-13T07:00:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.379+0000 7f90b3200640 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c72191add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.381+0000 7f90b3200640 1 -- 172.21.15.7:0/280222960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721912de0 msgr2=0x55c721913200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.381+0000 7f90b3200640 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721912de0 0x55c721913200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.382+0000 7f90b3200640 1 -- 172.21.15.7:0/280222960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 msgr2=0x55c7219182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.382+0000 7f90b3200640 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.382+0000 7f90b3200640 1 -- 172.21.15.7:0/280222960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72158a210 con 0x55c721918940 2024-06-13T07:00:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.382+0000 7f90b1e00640 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.383+0000 7f90b3200640 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c72191add0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f90a40cc220 tx=0x7f90a40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b3b16bf9bf5b62 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.383+0000 7f90b1e00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 0x7f90980a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55c721913bd0 tx=0x7f90a0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.387+0000 7f909f400640 1 -- 172.21.15.7:0/280222960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a409af80 con 0x55c721918940 2024-06-13T07:00:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.387+0000 7f909f400640 1 -- 172.21.15.7:0/280222960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a409f330 con 0x55c721918940 2024-06-13T07:00:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.387+0000 7f909f400640 1 -- 172.21.15.7:0/280222960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40b9bc0 con 0x55c721918940 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.388+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 msgr2=0x55c72191add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.388+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c72191add0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f90a40cc220 tx=0x7f90a40047d0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.389+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 shutdown_connections 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.389+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c72191add0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.389+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c7219137d0 0x55c7219182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.390+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/280222960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721912de0 0x55c721913200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.390+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 >> 172.21.15.7:0/280222960 conn(0x55c7218dafb0 msgr2=0x55c72190e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.392+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 shutdown_connections 2024-06-13T07:00:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.392+0000 7f90bbdfd740 1 -- 172.21.15.7:0/280222960 wait complete. 2024-06-13T07:00:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.393+0000 7f90bbdfd740 1 -- start start 2024-06-13T07:00:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.393+0000 7f90bbdfd740 1 Processor -- start 2024-06-13T07:00:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.395+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.396+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c72191d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.396+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c72191b830 con 0x55c721918940 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c7218310b0 con 0x55c721912de0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c72191b9b0 con 0x55c7219137d0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90bbdfd740 10 client.0 _ll_register_callbacks cb 0x55c72181cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54664/0 (socket says 172.21.15.7:54664) 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.397+0000 7f90b3200640 1 -- 172.21.15.7:0/995907711 learned_addr learned my addr 172.21.15.7:0/995907711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b1e00640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c72191d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 -- 172.21.15.7:0/995907711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 msgr2=0x55c72191d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c72191d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 -- 172.21.15.7:0/995907711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 msgr2=0x55c721924660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b2800640 1 -- 172.21.15.7:0/995907711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c721656620 con 0x55c721912de0 2024-06-13T07:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.398+0000 7f90b3200640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.399+0000 7f90b2800640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191d0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c721914030 tx=0x7f90ac010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd07e4e7ee6e5646 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.399+0000 7f9093e00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.399+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac017070 con 0x55c721912de0 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.399+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a409f330 con 0x55c721912de0 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.400+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90ac003d80 con 0x55c721912de0 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.400+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90ac00e880 con 0x55c721912de0 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.401+0000 7f90bbdfd740 10 client.26039 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.401+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c721927a90 con 0x55c721912de0 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a409f330 con 0x55c721912de0 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f90ac011440 con 0x55c721912de0 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 10 client.26039 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 20 client.26039 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c72192b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 4 client.26039 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.403+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c721883b50 con 0x55c7219291d0 2024-06-13T07:00:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.404+0000 7f90b1e00640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c72192b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.405+0000 7f90b1e00640 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c72192b640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33db32e2a8675576 server_cookie=e5b87ea3d0260388 in_seq=0 out_seq=0 2024-06-13T07:00:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.405+0000 7f9093e00640 10 client.26039 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.406+0000 7f9093e00640 1 -- 172.21.15.7:0/995907711 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c721883b50 con 0x55c7219291d0 2024-06-13T07:00:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.406+0000 7f9093e00640 10 client.26039 handle_command_reply: tid=0 2024-06-13T07:00:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.406+0000 7f90bbdfd740 1 client.26039 shutdown 2024-06-13T07:00:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.408+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 msgr2=0x55c72191d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.408+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191d0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55c721914030 tx=0x7f90ac010cb0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.410+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 shutdown_connections 2024-06-13T07:00:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.410+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c72192b640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.410+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721918940 0x55c721924660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f9093e00640 0 client.26039 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c72191d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/995907711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191d0e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 >> 172.21.15.7:0/995907711 conn(0x55c7218dafb0 msgr2=0x55c72190d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 shutdown_connections 2024-06-13T07:00:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 1 -- 172.21.15.7:0/995907711 wait complete. 2024-06-13T07:00:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.411+0000 7f90bbdfd740 20 client.26039 trim_cache size 0 max 16384 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.444+0000 7f90bbdfd740 1 Processor -- start 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 -- start start 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 0x55c7219245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 0x55c721911cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41bc0e0 con 0x55c7219137d0 2024-06-13T07:00:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41b9950 con 0x55c721912de0 2024-06-13T07:00:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.445+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41b77a0 con 0x55c721918940 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.446+0000 7f90b3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 0x55c721911cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.446+0000 7f90b2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.446+0000 7f90b3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 0x55c721911cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58842/0 (socket says 172.21.15.7:58842) 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.446+0000 7f90b3200640 1 -- 172.21.15.7:0/1243982814 learned_addr learned my addr 172.21.15.7:0/1243982814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.446+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 0x55c7219245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.447+0000 7f90b1e00640 1 -- 172.21.15.7:0/1243982814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 msgr2=0x55c721911cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.447+0000 7f90b1e00640 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 0x55c721911cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.447+0000 7f90b1e00640 1 -- 172.21.15.7:0/1243982814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 msgr2=0x55c72191bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.447+0000 7f90b1e00640 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.447+0000 7f90b1e00640 1 -- 172.21.15.7:0/1243982814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c721914060 con 0x55c7219137d0 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.448+0000 7f90b1e00640 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 0x55c7219245f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c721914240 tx=0x7f90a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4536c42d097476 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.449+0000 7f909ea00640 1 -- 172.21.15.7:0/1243982814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0055070 con 0x55c7219137d0 2024-06-13T07:00:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.450+0000 7f909ea00640 1 -- 172.21.15.7:0/1243982814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0041040 con 0x55c7219137d0 2024-06-13T07:00:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.450+0000 7f909ea00640 1 -- 172.21.15.7:0/1243982814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0055070 con 0x55c7219137d0 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.450+0000 7f90bbdfd740 1 -- 172.21.15.7:0/1243982814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 msgr2=0x55c7219245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.450+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 0x55c7219245f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c721914240 tx=0x7f90a0005fd0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.451+0000 7f90bbdfd740 1 -- 172.21.15.7:0/1243982814 shutdown_connections 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.451+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c721918940 0x55c721911cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.451+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7219137d0 0x55c7219245f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.451+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/1243982814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721912de0 0x55c72191bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.451+0000 7f90bbdfd740 1 -- 172.21.15.7:0/1243982814 >> 172.21.15.7:0/1243982814 conn(0x55c7218dafb0 msgr2=0x55c7219115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.454+0000 7f90bbdfd740 1 -- 172.21.15.7:0/1243982814 shutdown_connections 2024-06-13T07:00:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.454+0000 7f90bbdfd740 1 -- 172.21.15.7:0/1243982814 wait complete. 2024-06-13T07:00:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.455+0000 7f90bbdfd740 1 -- start start 2024-06-13T07:00:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.456+0000 7f90bbdfd740 1 Processor -- start 2024-06-13T07:00:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.456+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721912de0 0x55c7219215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.456+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c721921af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.456+0000 7f90bbdfd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 0x55c721991f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41b7e10 con 0x55c721912de0 2024-06-13T07:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41b9280 con 0x55c721918940 2024-06-13T07:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90bbdfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41b84e0 con 0x55c7219137d0 2024-06-13T07:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c721921af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90b3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 0x55c721991f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c721921af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58844/0 (socket says 172.21.15.7:58844) 2024-06-13T07:00:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.457+0000 7f90b1e00640 1 -- 172.21.15.7:0/3148948588 learned_addr learned my addr 172.21.15.7:0/3148948588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90bbdfd740 10 client.0 _ll_register_callbacks cb 0x7f90ac009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b2800640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721912de0 0x55c7219215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b3200640 1 -- 172.21.15.7:0/3148948588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 msgr2=0x55c721921af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b3200640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c721921af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b3200640 1 -- 172.21.15.7:0/3148948588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721912de0 msgr2=0x55c7219215b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b3200640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721912de0 0x55c7219215b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.458+0000 7f90b3200640 1 -- 172.21.15.7:0/3148948588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c721992450 con 0x55c721918940 2024-06-13T07:00:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.460+0000 7f90b3200640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 0x55c721991f10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90a4097950 tx=0x7f90a4010ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d9a60ae4e56023 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.460+0000 7f9093e00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.460+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a409f330 con 0x55c721918940 2024-06-13T07:00:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.460+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a40b9bc0 con 0x55c721918940 2024-06-13T07:00:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.460+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a0041040 con 0x55c721918940 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.461+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90a407d070 con 0x55c721918940 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.461+0000 7f90bbdfd740 10 client.26042 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.461+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c721993710 con 0x55c721918940 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0041040 con 0x55c721918940 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f90a4096cd0 con 0x55c721918940 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f90bbdfd740 10 client.26042 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f90bbdfd740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f90bbdfd740 20 client.26042 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.462+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c721995ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.463+0000 7f90bbdfd740 4 client.26042 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.463+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c72187dea0 con 0x55c7219291d0 2024-06-13T07:00:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.464+0000 7f90b2800640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c721995ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.466+0000 7f90b2800640 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c721995ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ee6053d2364036 server_cookie=86dfec37129cc2ca in_seq=0 out_seq=0 2024-06-13T07:00:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.467+0000 7f9093e00640 10 client.26042 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.511+0000 7f9093e00640 1 -- 172.21.15.7:0/3148948588 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c72187dea0 con 0x55c7219291d0 2024-06-13T07:00:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.511+0000 7f9093e00640 10 client.26042 handle_command_reply: tid=0 2024-06-13T07:00:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.511+0000 7f90bbdfd740 1 client.26042 shutdown 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 msgr2=0x55c721991f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 0x55c721991f10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90a4097950 tx=0x7f90a4010ec0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 shutdown_connections 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c7219291d0 0x55c721995ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c721918940 0x55c721991f10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7219137d0 0x55c721921af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 --2- 172.21.15.7:0/3148948588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c721912de0 0x55c7219215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 >> 172.21.15.7:0/3148948588 conn(0x55c7218dafb0 msgr2=0x55c721911e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.516+0000 7f9093e00640 0 client.26042 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.517+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 shutdown_connections 2024-06-13T07:00:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.517+0000 7f90bbdfd740 1 -- 172.21.15.7:0/3148948588 wait complete. 2024-06-13T07:00:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.517+0000 7f90bbdfd740 20 client.26042 trim_cache size 0 max 16384 2024-06-13T07:00:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:00:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:00:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 msgr2=0x7f90980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 0x7f90980a50c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55c721913bd0 tx=0x7f90a0000bc0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 msgr2=0x7f90b419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009fd0 tx=0x7f90a400b8e0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 shutdown_connections 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f90980a2c00 0x7f90980a50c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b419b130 0x7f90b419d5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b4107560 0x7f90b41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 --2- 172.21.15.7:0/4112355984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b4102300 0x7f90b4195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.518+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 >> 172.21.15.7:0/4112355984 conn(0x7f90b40fbc90 msgr2=0x7f90b4101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.519+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 shutdown_connections 2024-06-13T07:00:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:42.519+0000 7f90b8a00640 1 -- 172.21.15.7:0/4112355984 wait complete. 2024-06-13T07:00:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 280.0 s, 520.8 tps, lat 197.451 ms stddev 241.246 2024-06-13T07:00:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:43 smithi189 ceph-mon[36616]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-06-13T07:00:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:43 smithi175 ceph-mon[36886]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-06-13T07:00:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:43 smithi007 ceph-mon[32077]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.234+0000 7fb437200700 1 -- 172.21.15.7:0/3422872134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb430110c70 msgr2=0x7fb430109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.234+0000 7fb437200700 1 --2- 172.21.15.7:0/3422872134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb430110c70 0x7fb430109490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b0a0 tx=0x7fb42800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 -- 172.21.15.7:0/3422872134 shutdown_connections 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 --2- 172.21.15.7:0/3422872134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb43011bc70 0x7fb43011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 --2- 172.21.15.7:0/3422872134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb430110c70 0x7fb430109490 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 --2- 172.21.15.7:0/3422872134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4301099d0 0x7fb430109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 -- 172.21.15.7:0/3422872134 >> 172.21.15.7:0/3422872134 conn(0x7fb43006d600 msgr2=0x7fb43006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 -- 172.21.15.7:0/3422872134 shutdown_connections 2024-06-13T07:00:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.235+0000 7fb437200700 1 -- 172.21.15.7:0/3422872134 wait complete. 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.236+0000 7fb437200700 1 Processor -- start 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 -- start start 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 0x7fb43011a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430110c70 0x7fb430113300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 0x7fb430113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301216f0 con 0x7fb430110c70 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb430121570 con 0x7fb43011bc70 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.237+0000 7fb437200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb430121870 con 0x7fb4301099d0 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.238+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 0x7fb43011a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.238+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 0x7fb43011a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58852/0 (socket says 172.21.15.7:58852) 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.238+0000 7fb42fe00700 1 -- 172.21.15.7:0/2117068173 learned_addr learned my addr 172.21.15.7:0/2117068173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.238+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 0x7fb430113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.239+0000 7fb435200700 1 -- 172.21.15.7:0/2117068173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 msgr2=0x7fb43011a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.239+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 0x7fb43011a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.239+0000 7fb435200700 1 -- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430110c70 msgr2=0x7fb430113300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.239+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430110c70 0x7fb430113300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.239+0000 7fb435200700 1 -- 172.21.15.7:0/2117068173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb430113e30 con 0x7fb43011bc70 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.240+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 0x7fb430113840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b3c0 tx=0x7fb41c00b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.240+0000 7fb42cc00700 1 -- 172.21.15.7:0/2117068173 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015020 con 0x7fb43011bc70 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.240+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb430114120 con 0x7fb43011bc70 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.240+0000 7fb42cc00700 1 -- 172.21.15.7:0/2117068173 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41c011d10 con 0x7fb43011bc70 2024-06-13T07:00:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.240+0000 7fb42cc00700 1 -- 172.21.15.7:0/2117068173 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015020 con 0x7fb43011bc70 2024-06-13T07:00:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.241+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb430118110 con 0x7fb43011bc70 2024-06-13T07:00:45.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.253+0000 7fb42cc00700 1 -- 172.21.15.7:0/2117068173 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb41c018020 con 0x7fb43011bc70 2024-06-13T07:00:45.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.253+0000 7fb441b18b80 1 Processor -- start 2024-06-13T07:00:45.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 -- start start 2024-06-13T07:00:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04687be40 0x55c0468ff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c0468ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c04680ea10 con 0x55c0468ff5b0 2024-06-13T07:00:45.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c046851b00 con 0x55c04690a300 2024-06-13T07:00:45.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.254+0000 7fb441b18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c04680c8f0 con 0x55c04687be40 2024-06-13T07:00:45.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.255+0000 7fb42f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.255+0000 7fb42f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60050/0 (socket says 172.21.15.7:60050) 2024-06-13T07:00:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.255+0000 7fb42f400700 1 -- 172.21.15.7:0/2126454433 learned_addr learned my addr 172.21.15.7:0/2126454433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.255+0000 7fb42f400700 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04687be40 0x55c0468ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.256+0000 7fb42fe00700 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c0468ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.258+0000 7fb42cc00700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 0x7fb4140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.258+0000 7fb42cc00700 1 -- 172.21.15.7:0/2117068173 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb41c0ce620 con 0x7fb43011bc70 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.258+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 0x7fb4140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.259+0000 7fb435200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 0x7fb4140a53e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb41c09aa70 tx=0x7fb41c0106e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.260+0000 7fb42f400700 1 -- 172.21.15.7:0/2126454433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04687be40 msgr2=0x55c0468ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.260+0000 7fb42f400700 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04687be40 0x55c0468ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.260+0000 7fb42f400700 1 -- 172.21.15.7:0/2126454433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 msgr2=0x55c0468ffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.260+0000 7fb42f400700 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c0468ffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.260+0000 7fb42f400700 1 -- 172.21.15.7:0/2126454433 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0467f9f90 con 0x55c04690a300 2024-06-13T07:00:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.261+0000 7fb42f400700 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb420005d90 tx=0x7fb42000af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5863a5040a40206b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.262+0000 7fb425600700 1 -- 172.21.15.7:0/2126454433 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420012230 con 0x55c04690a300 2024-06-13T07:00:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.262+0000 7fb425600700 1 -- 172.21.15.7:0/2126454433 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb420011910 con 0x55c04690a300 2024-06-13T07:00:45.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.262+0000 7fb425600700 1 -- 172.21.15.7:0/2126454433 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42000d6d0 con 0x55c04690a300 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.263+0000 7fb441b18b80 1 -- 172.21.15.7:0/2126454433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 msgr2=0x55c04690c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.263+0000 7fb441b18b80 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb420005d90 tx=0x7fb42000af90 comp rx=0 tx=0).stop 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 -- 172.21.15.7:0/2126454433 shutdown_connections 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690c6f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04687be40 0x55c0468ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 --2- 172.21.15.7:0/2126454433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c0468ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 -- 172.21.15.7:0/2126454433 >> 172.21.15.7:0/2126454433 conn(0x55c04682c170 msgr2=0x55c0468fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 -- 172.21.15.7:0/2126454433 shutdown_connections 2024-06-13T07:00:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.264+0000 7fb441b18b80 1 -- 172.21.15.7:0/2126454433 wait complete. 2024-06-13T07:00:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.265+0000 7fb441b18b80 1 -- start start 2024-06-13T07:00:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.266+0000 7fb441b18b80 1 Processor -- start 2024-06-13T07:00:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.266+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046909a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.266+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c046906d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb441b18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c0467edf60 con 0x55c0468ff5b0 2024-06-13T07:00:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb441b18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c04684dac0 con 0x55c04687be40 2024-06-13T07:00:45.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb441b18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c04690d010 con 0x55c04690a300 2024-06-13T07:00:45.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb441b18b80 10 client.0 _ll_register_callbacks cb 0x55c0467eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.267+0000 7fb42f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.268+0000 7fb42f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58864/0 (socket says 172.21.15.7:58864) 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.268+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c046906d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.268+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c046906d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54744/0 (socket says 172.21.15.7:54744) 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.268+0000 7fb42fe00700 1 -- 172.21.15.7:0/910465368 learned_addr learned my addr 172.21.15.7:0/910465368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.268+0000 7fb42f400700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046909a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.269+0000 7fb42f400700 1 -- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 msgr2=0x55c046909a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.269+0000 7fb42f400700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046909a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.269+0000 7fb42f400700 1 -- 172.21.15.7:0/910465368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 msgr2=0x55c046906d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.269+0000 7fb42f400700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c046906d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.269+0000 7fb42f400700 1 -- 172.21.15.7:0/910465368 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0467a08d0 con 0x55c04690a300 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb42f400700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb420005f80 tx=0x7fb420012ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b97521bc8e602d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb41a000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420011430 con 0x55c04690a300 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb4200115d0 con 0x55c04690a300 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb420011910 con 0x55c04690a300 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb420030050 con 0x55c04690a300 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb441b18b80 10 client.26035 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.270+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c046917790 con 0x55c04690a300 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb420011910 con 0x55c04690a300 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb42001d040 con 0x55c04690a300 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 10 client.26035 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 20 client.26035 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c046919c60 0x55c04691c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 4 client.26035 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.271+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c04685f000 con 0x55c046919c60 2024-06-13T07:00:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.272+0000 7fb42fe00700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c046919c60 0x55c04691c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.272+0000 7fb42fe00700 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c046919c60 0x55c04691c050 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a168d6da4e425bc5 server_cookie=fa418700df0a5c20 in_seq=0 out_seq=0 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.272+0000 7fb41a000700 10 client.26035 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.273+0000 7fb41a000700 1 -- 172.21.15.7:0/910465368 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c04685f000 con 0x55c046919c60 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.273+0000 7fb41a000700 10 client.26035 handle_command_reply: tid=0 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.273+0000 7fb441b18b80 1 client.26035 shutdown 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.273+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 msgr2=0x55c0469072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.273+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb420005f80 tx=0x7fb420012ea0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 shutdown_connections 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c046919c60 0x55c04691c050 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c04690a300 0x55c0469072b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046909a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 --2- 172.21.15.7:0/910465368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c0468ff5b0 0x55c046906d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 >> 172.21.15.7:0/910465368 conn(0x55c04682c170 msgr2=0x55c0468fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb41a000700 0 client.26035 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 shutdown_connections 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 1 -- 172.21.15.7:0/910465368 wait complete. 2024-06-13T07:00:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.274+0000 7fb441b18b80 20 client.26035 trim_cache size 0 max 16384 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.310+0000 7fb441b18b80 1 Processor -- start 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.310+0000 7fb441b18b80 1 -- start start 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.310+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04687be40 0x55c0468ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb441b18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b71c0 con 0x55c04687be40 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb441b18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b49f0 con 0x55c04690a300 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb441b18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b2840 con 0x55c0468ff5b0 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58878/0 (socket says 172.21.15.7:58878) 2024-06-13T07:00:45.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.311+0000 7fb42fe00700 1 -- 172.21.15.7:0/480711755 learned_addr learned my addr 172.21.15.7:0/480711755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.312+0000 7fb42fe00700 1 -- 172.21.15.7:0/480711755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 msgr2=0x55c04690b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.312+0000 7fb42fe00700 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.312+0000 7fb42fe00700 1 -- 172.21.15.7:0/480711755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04687be40 msgr2=0x55c0468ffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.312+0000 7fb42fe00700 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04687be40 0x55c0468ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.312+0000 7fb42fe00700 1 -- 172.21.15.7:0/480711755 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0469029b0 con 0x55c0468ff5b0 2024-06-13T07:00:45.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.313+0000 7fb42fe00700 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d010 tx=0x7fb42800b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=556c7d3ab572c247 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.313+0000 7fb424c00700 1 -- 172.21.15.7:0/480711755 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42804b070 con 0x55c0468ff5b0 2024-06-13T07:00:45.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.313+0000 7fb424c00700 1 -- 172.21.15.7:0/480711755 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb428024400 con 0x55c0468ff5b0 2024-06-13T07:00:45.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.315+0000 7fb424c00700 1 -- 172.21.15.7:0/480711755 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428007740 con 0x55c0468ff5b0 2024-06-13T07:00:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.316+0000 7fb441b18b80 1 -- 172.21.15.7:0/480711755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 msgr2=0x55c046908cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.316+0000 7fb441b18b80 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d010 tx=0x7fb42800b7d0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.317+0000 7fb441b18b80 1 -- 172.21.15.7:0/480711755 shutdown_connections 2024-06-13T07:00:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.317+0000 7fb441b18b80 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04690a300 0x55c04690b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.317+0000 7fb441b18b80 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04687be40 0x55c0468ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.317+0000 7fb441b18b80 1 --2- 172.21.15.7:0/480711755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046908cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.317+0000 7fb441b18b80 1 -- 172.21.15.7:0/480711755 >> 172.21.15.7:0/480711755 conn(0x55c04682c170 msgr2=0x55c0469085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.318+0000 7fb441b18b80 1 -- 172.21.15.7:0/480711755 shutdown_connections 2024-06-13T07:00:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.318+0000 7fb441b18b80 1 -- 172.21.15.7:0/480711755 wait complete. 2024-06-13T07:00:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.319+0000 7fb441b18b80 1 -- start start 2024-06-13T07:00:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.320+0000 7fb441b18b80 1 Processor -- start 2024-06-13T07:00:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.321+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046912540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 0x55c0469040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b2eb0 con 0x55c04690a300 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b4320 con 0x55c04687be40 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4301b3580 con 0x55c0468ff5b0 2024-06-13T07:00:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.323+0000 7fb441b18b80 10 client.0 _ll_register_callbacks cb 0x55c0467eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.324+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.324+0000 7fb42fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58894/0 (socket says 172.21.15.7:58894) 2024-06-13T07:00:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.324+0000 7fb42fe00700 1 -- 172.21.15.7:0/1949915987 learned_addr learned my addr 172.21.15.7:0/1949915987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:45.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42f400700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046912540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42fe00700 1 -- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 msgr2=0x55c046912540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42fe00700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046912540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42fe00700 1 -- 172.21.15.7:0/1949915987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 msgr2=0x55c0469040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:45.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42f400700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 0x55c0469040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42fe00700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 0x55c0469040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.325+0000 7fb42fe00700 1 -- 172.21.15.7:0/1949915987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c046904910 con 0x55c0468ff5b0 2024-06-13T07:00:45.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.326+0000 7fb42f400700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 0x55c0469040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.329+0000 7fb42fe00700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b3f0 tx=0x7fb428003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a18251446393c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:45.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.329+0000 7fb41a000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.329+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42804b070 con 0x55c0468ff5b0 2024-06-13T07:00:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.330+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb428024400 con 0x55c0468ff5b0 2024-06-13T07:00:45.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.330+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb4280245b0 con 0x55c0468ff5b0 2024-06-13T07:00:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.330+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4280247b0 con 0x55c0468ff5b0 2024-06-13T07:00:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.331+0000 7fb441b18b80 10 client.26041 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:45.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.331+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c0469218c0 con 0x55c0468ff5b0 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.331+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb428024400 con 0x55c0468ff5b0 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb428011070 con 0x55c0468ff5b0 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 10 client.26041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 20 client.26041 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c0469848b0 0x55c046986ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:45.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 4 client.26041 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:00:45.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c046859580 con 0x55c0469848b0 2024-06-13T07:00:45.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.332+0000 7fb42f400700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c0469848b0 0x55c046986ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:45.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.333+0000 7fb42f400700 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c0469848b0 0x55c046986ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b6eccb040daf1f server_cookie=5b0193651ce2efca in_seq=0 out_seq=0 2024-06-13T07:00:45.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.333+0000 7fb41a000700 10 client.26041 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:45.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.333+0000 7fb41a000700 1 -- 172.21.15.7:0/1949915987 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c046859580 con 0x55c0469848b0 2024-06-13T07:00:45.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.333+0000 7fb41a000700 10 client.26041 handle_command_reply: tid=0 2024-06-13T07:00:45.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.334+0000 7fb441b18b80 1 client.26041 shutdown 2024-06-13T07:00:45.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.334+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 msgr2=0x55c046912a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.334+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b3f0 tx=0x7fb428003a00 comp rx=0 tx=0).stop 2024-06-13T07:00:45.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 shutdown_connections 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c0469848b0 0x55c046986ca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c04690a300 0x55c0469040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c04687be40 0x55c046912540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 --2- 172.21.15.7:0/1949915987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0468ff5b0 0x55c046912a80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 >> 172.21.15.7:0/1949915987 conn(0x55c04682c170 msgr2=0x55c046908e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb41a000700 0 client.26041 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 shutdown_connections 2024-06-13T07:00:45.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.335+0000 7fb441b18b80 1 -- 172.21.15.7:0/1949915987 wait complete. 2024-06-13T07:00:45.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.336+0000 7fb441b18b80 20 client.26041 trim_cache size 0 max 16384 2024-06-13T07:00:45.337 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:00:45.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 msgr2=0x7fb4140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 0x7fb4140a53e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb41c09aa70 tx=0x7fb41c0106e0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 msgr2=0x7fb430113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 0x7fb430113840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b3c0 tx=0x7fb41c00b6d0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 shutdown_connections 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb4140a2f20 0x7fb4140a53e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb43011bc70 0x7fb430113840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4301099d0 0x7fb43011a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 --2- 172.21.15.7:0/2117068173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430110c70 0x7fb430113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:45.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 >> 172.21.15.7:0/2117068173 conn(0x7fb43006d600 msgr2=0x7fb430071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:45.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 shutdown_connections 2024-06-13T07:00:45.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:45.337+0000 7fb437200700 1 -- 172.21.15.7:0/2117068173 wait complete. 2024-06-13T07:00:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:45 smithi175 ceph-mon[36886]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 MiB/s wr, 715 op/s 2024-06-13T07:00:45.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:45 smithi007 ceph-mon[32077]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 MiB/s wr, 715 op/s 2024-06-13T07:00:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:45 smithi189 ceph-mon[36616]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 MiB/s wr, 715 op/s 2024-06-13T07:00:47.164 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:00:47.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:00:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:47 smithi175 ceph-mon[36886]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-06-13T07:00:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:47 smithi007 ceph-mon[32077]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-06-13T07:00:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:47 smithi189 ceph-mon[36616]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 598 op/s 2024-06-13T07:00:47.837 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 285.0 s, 575.4 tps, lat 173.160 ms stddev 215.633 2024-06-13T07:00:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:49 smithi175 ceph-mon[36886]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 717 op/s 2024-06-13T07:00:49.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:49 smithi007 ceph-mon[32077]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 717 op/s 2024-06-13T07:00:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:49 smithi189 ceph-mon[36616]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 717 op/s 2024-06-13T07:00:51.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.322+0000 7f0d77600700 1 -- 172.21.15.7:0/1077837407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d70105070 msgr2=0x7f0d70111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.322+0000 7f0d77600700 1 --2- 172.21.15.7:0/1077837407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d70105070 0x7f0d70111580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d68014a30 tx=0x7f0d68014d40 comp rx=0 tx=0).stop 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 -- 172.21.15.7:0/1077837407 shutdown_connections 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 --2- 172.21.15.7:0/1077837407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d70105070 0x7f0d70111580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 --2- 172.21.15.7:0/1077837407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7010a6b0 0x7f0d7010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 --2- 172.21.15.7:0/1077837407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d701046b0 0x7f0d70104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 -- 172.21.15.7:0/1077837407 >> 172.21.15.7:0/1077837407 conn(0x7f0d701001c0 msgr2=0x7f0d701025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.323+0000 7f0d77600700 1 -- 172.21.15.7:0/1077837407 shutdown_connections 2024-06-13T07:00:51.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.324+0000 7f0d77600700 1 -- 172.21.15.7:0/1077837407 wait complete. 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 Processor -- start 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 -- start start 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 0x7f0d70073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700791f0 con 0x7f0d70105070 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70079070 con 0x7f0d701046b0 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.325+0000 7f0d77600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70079370 con 0x7f0d7010a6b0 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d75600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34748/0 (socket says 172.21.15.7:34748) 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d75600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37608/0 (socket says 172.21.15.7:37608) 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d74c00700 1 -- 172.21.15.7:0/3509076212 learned_addr learned my addr 172.21.15.7:0/3509076212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d75600700 1 -- 172.21.15.7:0/3509076212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 msgr2=0x7f0d70073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d75600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.326+0000 7f0d75600700 1 -- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 msgr2=0x7f0d70073f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 0x7f0d70073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d75600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 0x7f0d70073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d75600700 1 -- 172.21.15.7:0/3509076212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7006c1f0 con 0x7f0d7010a6b0 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 0x7f0d70073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d75600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d68014950 tx=0x7f0d6801cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d6d600700 1 -- 172.21.15.7:0/3509076212 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68021020 con 0x7f0d7010a6b0 2024-06-13T07:00:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d6d600700 1 -- 172.21.15.7:0/3509076212 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6801c860 con 0x7f0d7010a6b0 2024-06-13T07:00:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.327+0000 7f0d6d600700 1 -- 172.21.15.7:0/3509076212 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800d020 con 0x7f0d7010a6b0 2024-06-13T07:00:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.328+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7006c480 con 0x7f0d7010a6b0 2024-06-13T07:00:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.328+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701bcec0 con 0x7f0d7010a6b0 2024-06-13T07:00:51.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.334+0000 7f0d6d600700 1 -- 172.21.15.7:0/3509076212 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0d6800f450 con 0x7f0d7010a6b0 2024-06-13T07:00:51.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 Processor -- start 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 -- start start 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d6d600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 0x7f0d580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2b5380 0x55d78f2b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d78f1b9a90 con 0x55d78f2b5380 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d78f1fcb80 con 0x55d78f226ec0 2024-06-13T07:00:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d78f1b7970 con 0x55d78f2aa630 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.336+0000 7f0d6d600700 1 -- 172.21.15.7:0/3509076212 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0d680cf540 con 0x7f0d7010a6b0 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34756/0 (socket says 172.21.15.7:34756) 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 -- 172.21.15.7:0/3605272648 learned_addr learned my addr 172.21.15.7:0/3605272648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 0x7f0d580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 -- 172.21.15.7:0/3605272648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 msgr2=0x55d78f2aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 -- 172.21.15.7:0/3605272648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2b5380 msgr2=0x55d78f2b7770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2b5380 0x55d78f2b7770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 -- 172.21.15.7:0/3605272648 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78f1a5010 con 0x55d78f226ec0 2024-06-13T07:00:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400ad80 tx=0x7f0d6400f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc876e5dccbd7b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d62000700 1 -- 172.21.15.7:0/3605272648 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004a30 con 0x55d78f226ec0 2024-06-13T07:00:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d62000700 1 -- 172.21.15.7:0/3605272648 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6400fe60 con 0x55d78f226ec0 2024-06-13T07:00:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.338+0000 7f0d62000700 1 -- 172.21.15.7:0/3605272648 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64011710 con 0x55d78f226ec0 2024-06-13T07:00:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.339+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3605272648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 msgr2=0x55d78f2aa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.339+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400ad80 tx=0x7f0d6400f640 comp rx=0 tx=0).stop 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 0x7f0d580a5160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c006fd0 tx=0x7f0d5c006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3605272648 shutdown_connections 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2b5380 0x55d78f2b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2aa0f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3605272648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.340+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3605272648 >> 172.21.15.7:0/3605272648 conn(0x55d78f1d71f0 msgr2=0x55d78f2a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.341+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3605272648 shutdown_connections 2024-06-13T07:00:51.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.341+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3605272648 wait complete. 2024-06-13T07:00:51.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.342+0000 7f0d81ef9b80 1 -- start start 2024-06-13T07:00:51.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.342+0000 7f0d81ef9b80 1 Processor -- start 2024-06-13T07:00:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.344+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 0x55d78f2b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.344+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.344+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d78f198fe0 con 0x55d78f226ec0 2024-06-13T07:00:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d78f1f8b40 con 0x55d78f2b5380 2024-06-13T07:00:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d78f2b8040 con 0x55d78f2aa630 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d81ef9b80 10 client.0 _ll_register_callbacks cb 0x55d78f1967a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 0x55d78f2b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 0x55d78f2b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60836/0 (socket says 172.21.15.7:60836) 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d74c00700 1 -- 172.21.15.7:0/3229264287 learned_addr learned my addr 172.21.15.7:0/3229264287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.345+0000 7f0d75600700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 -- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 msgr2=0x55d78f2b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 -- 172.21.15.7:0/3229264287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 msgr2=0x55d78f2b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 0x55d78f2b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 -- 172.21.15.7:0/3229264287 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78f14b950 con 0x55d78f2aa630 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2b1e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d7006d1a0 tx=0x7f0d5c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5958b46a37f80888 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d56a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.346+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c028060 con 0x55d78f2aa630 2024-06-13T07:00:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.347+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0d5c018d80 con 0x55d78f2aa630 2024-06-13T07:00:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.347+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6400fe60 con 0x55d78f2aa630 2024-06-13T07:00:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.347+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d5c00f040 con 0x55d78f2aa630 2024-06-13T07:00:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.347+0000 7f0d81ef9b80 10 client.26050 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.347+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d78f2c2630 con 0x55d78f2aa630 2024-06-13T07:00:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.351+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6400fe60 con 0x55d78f2aa630 2024-06-13T07:00:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.351+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0d5c016550 con 0x55d78f2aa630 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 20 client.26050 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f2c4bc0 0x55d78f2c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 4 client.26050 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.352+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d78f20a080 con 0x55d78f2c4bc0 2024-06-13T07:00:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.357+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f2c4bc0 0x55d78f2c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.359+0000 7f0d74c00700 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f2c4bc0 0x55d78f2c6fb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6c5d97eb3baaff server_cookie=271fc26d8face795 in_seq=0 out_seq=0 2024-06-13T07:00:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.359+0000 7f0d56a00700 10 client.26050 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.361+0000 7f0d56a00700 1 -- 172.21.15.7:0/3229264287 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d78f20a080 con 0x55d78f2c4bc0 2024-06-13T07:00:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.361+0000 7f0d56a00700 10 client.26050 handle_command_reply: tid=0 2024-06-13T07:00:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.361+0000 7f0d81ef9b80 1 client.26050 shutdown 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.361+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 msgr2=0x55d78f2b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.361+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2b1e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d7006d1a0 tx=0x7f0d5c004770 comp rx=0 tx=0).stop 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 shutdown_connections 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f2c4bc0 0x55d78f2c6fb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f226ec0 0x55d78f2b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/3229264287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2aa630 0x55d78f2b1e20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 >> 172.21.15.7:0/3229264287 conn(0x55d78f1d71f0 msgr2=0x55d78f2a57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.362+0000 7f0d56a00700 0 client.26050 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.363+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 shutdown_connections 2024-06-13T07:00:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.364+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/3229264287 wait complete. 2024-06-13T07:00:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.365+0000 7f0d81ef9b80 20 client.26050 trim_cache size 0 max 16384 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 Processor -- start 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 -- start start 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c8070 con 0x55d78f2a5b80 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c58a0 con 0x55d78f226ec0 2024-06-13T07:00:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.408+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c36f0 con 0x55d78f2b5380 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.409+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.409+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34766/0 (socket says 172.21.15.7:34766) 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.409+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.409+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37634/0 (socket says 172.21.15.7:37634) 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.409+0000 7f0d6fe00700 1 -- 172.21.15.7:0/4218270741 learned_addr learned my addr 172.21.15.7:0/4218270741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 -- 172.21.15.7:0/4218270741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 msgr2=0x55d78f2a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 -- 172.21.15.7:0/4218270741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 msgr2=0x55d78f2b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 -- 172.21.15.7:0/4218270741 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78f29de90 con 0x55d78f2b5380 2024-06-13T07:00:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.410+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c01f040 tx=0x7f0d5c00cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430c8a18cc4bbff4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.411+0000 7f0d61600700 1 -- 172.21.15.7:0/4218270741 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c032080 con 0x55d78f2b5380 2024-06-13T07:00:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.411+0000 7f0d61600700 1 -- 172.21.15.7:0/4218270741 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5c016b90 con 0x55d78f2b5380 2024-06-13T07:00:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.411+0000 7f0d61600700 1 -- 172.21.15.7:0/4218270741 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c02dad0 con 0x55d78f2b5380 2024-06-13T07:00:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/4218270741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 msgr2=0x55d78f2b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c01f040 tx=0x7f0d5c00cae0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/4218270741 shutdown_connections 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f226ec0 0x55d78f2a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/4218270741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f2b5380 0x55d78f2b6770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.412+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/4218270741 >> 172.21.15.7:0/4218270741 conn(0x55d78f1d71f0 msgr2=0x55d78f2a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.413+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/4218270741 shutdown_connections 2024-06-13T07:00:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.413+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/4218270741 wait complete. 2024-06-13T07:00:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.413+0000 7f0d81ef9b80 1 -- start start 2024-06-13T07:00:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.414+0000 7f0d81ef9b80 1 Processor -- start 2024-06-13T07:00:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.415+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.415+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.415+0000 7f0d81ef9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.415+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.415+0000 7f0d74c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37638/0 (socket says 172.21.15.7:37638) 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c3d60 con 0x55d78f2a5b80 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c51d0 con 0x55d78f2b5380 2024-06-13T07:00:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d81ef9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701c4430 con 0x55d78f226ec0 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d81ef9b80 10 client.0 _ll_register_callbacks cb 0x55d78f1967a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d75600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d75600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34782/0 (socket says 172.21.15.7:34782) 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d75600700 1 -- 172.21.15.7:0/2345847995 learned_addr learned my addr 172.21.15.7:0/2345847995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d74c00700 1 -- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 msgr2=0x55d78f2ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.416+0000 7f0d74c00700 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.417+0000 7f0d74c00700 1 -- 172.21.15.7:0/2345847995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 msgr2=0x55d78f2ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.417+0000 7f0d74c00700 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.417+0000 7f0d74c00700 1 -- 172.21.15.7:0/2345847995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78f2aeb30 con 0x55d78f226ec0 2024-06-13T07:00:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.417+0000 7f0d74c00700 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d64004d90 tx=0x7f0d6400fb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df577b075341dc40 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.417+0000 7f0d56a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.418+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64032020 con 0x55d78f226ec0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.418+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d5c016b90 con 0x55d78f226ec0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.418+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0d64004020 con 0x55d78f226ec0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.418+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d64004220 con 0x55d78f226ec0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.418+0000 7f0d81ef9b80 10 client.26056 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.419+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d78f2afe50 con 0x55d78f226ec0 2024-06-13T07:00:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.419+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5c016b90 con 0x55d78f226ec0 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.419+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0d64003810 con 0x55d78f226ec0 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 10 client.26056 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 20 client.26056 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f330a40 0x55d78f332e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 4 client.26056 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d78f204600 con 0x55d78f330a40 2024-06-13T07:00:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.420+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f330a40 0x55d78f332e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.421+0000 7f0d6fe00700 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f330a40 0x55d78f332e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e05fc7fbdf1e40b server_cookie=5a5083f94ed284c in_seq=0 out_seq=0 2024-06-13T07:00:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.421+0000 7f0d56a00700 10 client.26056 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d56a00700 1 -- 172.21.15.7:0/2345847995 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d78f204600 con 0x55d78f330a40 2024-06-13T07:00:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d56a00700 10 client.26056 handle_command_reply: tid=0 2024-06-13T07:00:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 client.26056 shutdown 2024-06-13T07:00:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 msgr2=0x55d78f2bd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d64004d90 tx=0x7f0d6400fb70 comp rx=0 tx=0).stop 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 shutdown_connections 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d78f330a40 0x55d78f332e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d78f2b5380 0x55d78f2ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d78f226ec0 0x55d78f2bd6e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 --2- 172.21.15.7:0/2345847995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d78f2a5b80 0x55d78f2ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 >> 172.21.15.7:0/2345847995 conn(0x55d78f1d71f0 msgr2=0x55d78f2c0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.423+0000 7f0d56a00700 0 client.26056 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.424+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 shutdown_connections 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.424+0000 7f0d81ef9b80 1 -- 172.21.15.7:0/2345847995 wait complete. 2024-06-13T07:00:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.424+0000 7f0d81ef9b80 20 client.26056 trim_cache size 0 max 16384 2024-06-13T07:00:51.426 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:00:51.426 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:00:51.426 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f20c6339-5202-4124-8c6b-a2182b721b61", 2024-06-13T07:00:51.426 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:00:51.426 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.425+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 msgr2=0x7f0d580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.425+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 0x7f0d580a5160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c006fd0 tx=0x7f0d5c006c40 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.425+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 msgr2=0x7f0d7006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.425+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d68014950 tx=0x7f0d6801cdf0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 shutdown_connections 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d7010a6b0 0x7f0d7006b9e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0d580a2ca0 0x7f0d580a5160 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d701046b0 0x7f0d70073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 --2- 172.21.15.7:0/3509076212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d70105070 0x7f0d70073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 >> 172.21.15.7:0/3509076212 conn(0x7f0d701001c0 msgr2=0x7f0d701005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 shutdown_connections 2024-06-13T07:00:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:51.426+0000 7f0d77600700 1 -- 172.21.15.7:0/3509076212 wait complete. 2024-06-13T07:00:51.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:51 smithi007 ceph-mon[32077]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:00:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:51 smithi189 ceph-mon[36616]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:00:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:51 smithi175 ceph-mon[36886]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:00:52.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:00:52.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:52 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:00:52.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:52 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:52 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:52.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:52 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:52.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:52 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:52 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:00:52.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:52 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:52 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:52.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:52 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:52.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:52 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:52 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:00:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:52 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:52 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:52 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:52 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 290.0 s, 503.5 tps, lat 198.779 ms stddev 211.880 2024-06-13T07:00:53.149 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:00:53.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:53.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 558 op/s 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 558 op/s 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 558 op/s 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:53.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:00:55.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:55 smithi007 ceph-mon[32077]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-06-13T07:00:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.734+0000 7f4dcce00640 1 -- 172.21.15.7:0/3109755591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0097930 msgr2=0x7f4dc009e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.735+0000 7f4dcce00640 1 --2- 172.21.15.7:0/3109755591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0097930 0x7f4dc009e1c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4dc804fa30 tx=0x7f4dc80963d0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.735+0000 7f4dcce00640 1 -- 172.21.15.7:0/3109755591 shutdown_connections 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.735+0000 7f4dcce00640 1 --2- 172.21.15.7:0/3109755591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0097930 0x7f4dc009e1c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.735+0000 7f4dcce00640 1 --2- 172.21.15.7:0/3109755591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0096f70 0x7f4dc00973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.736+0000 7f4dcce00640 1 --2- 172.21.15.7:0/3109755591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc00961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.736+0000 7f4dcce00640 1 -- 172.21.15.7:0/3109755591 >> 172.21.15.7:0/3109755591 conn(0x7f4dc00912d0 msgr2=0x7f4dc0093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.737+0000 7f4dcce00640 1 -- 172.21.15.7:0/3109755591 shutdown_connections 2024-06-13T07:00:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.737+0000 7f4dcce00640 1 -- 172.21.15.7:0/3109755591 wait complete. 2024-06-13T07:00:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.739+0000 7f4dcce00640 1 Processor -- start 2024-06-13T07:00:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 -- start start 2024-06-13T07:00:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc012df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 0x7f4dc012e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc00a1960 con 0x7f4dc0095d80 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc00a17e0 con 0x7f4dc0097930 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.740+0000 7f4dcce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc00a1ae0 con 0x7f4dc0096f70 2024-06-13T07:00:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34794/0 (socket says 172.21.15.7:34794) 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc7e00640 1 -- 172.21.15.7:0/468359979 learned_addr learned my addr 172.21.15.7:0/468359979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 0x7f4dc012e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc7400640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc012df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 -- 172.21.15.7:0/468359979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 msgr2=0x7f4dc0135510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 -- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 msgr2=0x7f4dc012df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc012df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc6a00640 1 -- 172.21.15.7:0/468359979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc0135b80 con 0x7f4dc0096f70 2024-06-13T07:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.741+0000 7f4dc7400640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc012df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:00:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.742+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 0x7f4dc012e4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e790 tx=0x7f4db400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.742+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.743+0000 7f4dbbe00640 1 -- 172.21.15.7:0/468359979 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db400eea0 con 0x7f4dc0096f70 2024-06-13T07:00:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.743+0000 7f4dbbe00640 1 -- 172.21.15.7:0/468359979 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db400ccc0 con 0x7f4dc0096f70 2024-06-13T07:00:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.743+0000 7f4dbbe00640 1 -- 172.21.15.7:0/468359979 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4018780 con 0x7f4dc0096f70 2024-06-13T07:00:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.743+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc0135e70 con 0x7f4dc0096f70 2024-06-13T07:00:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.744+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01362e0 con 0x7f4dc0096f70 2024-06-13T07:00:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.750+0000 7f4dbbe00640 1 -- 172.21.15.7:0/468359979 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4db4010430 con 0x7f4dc0096f70 2024-06-13T07:00:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.752+0000 7f4dd1211740 1 Processor -- start 2024-06-13T07:00:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 -- start start 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d997a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d99cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d99f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d8f2420 con 0x55ee5d997fe0 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d899c50 con 0x55ee5d99d150 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dd1211740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d8ee3e0 con 0x55ee5d9975f0 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d99f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d99f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34810/0 (socket says 172.21.15.7:34810) 2024-06-13T07:00:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.756+0000 7f4dc6a00640 1 -- 172.21.15.7:0/1093290139 learned_addr learned my addr 172.21.15.7:0/1093290139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d997a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dbbe00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 0x7f4db00a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dbbe00640 1 -- 172.21.15.7:0/468359979 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4db40cf4d0 con 0x7f4dc0096f70 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 -- 172.21.15.7:0/1093290139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 msgr2=0x55ee5d99f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d99f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 -- 172.21.15.7:0/1093290139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 msgr2=0x55ee5d99cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d99cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.757+0000 7f4dc7e00640 1 -- 172.21.15.7:0/1093290139 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5d606210 con 0x55ee5d9975f0 2024-06-13T07:00:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.758+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 0x7f4db00a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.758+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d997a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8067650 tx=0x7f4dc809d3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8306e8b869420f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.759+0000 7f4db8c00640 1 -- 172.21.15.7:0/1093290139 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80ae070 con 0x55ee5d9975f0 2024-06-13T07:00:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.759+0000 7f4db8c00640 1 -- 172.21.15.7:0/1093290139 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8073880 con 0x55ee5d9975f0 2024-06-13T07:00:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.759+0000 7f4db8c00640 1 -- 172.21.15.7:0/1093290139 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc809dd30 con 0x55ee5d9975f0 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 0x7f4db00a51a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4dc809dff0 tx=0x7f4dc806a470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 -- 172.21.15.7:0/1093290139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 msgr2=0x55ee5d997a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d997a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8067650 tx=0x7f4dc809d3a0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 -- 172.21.15.7:0/1093290139 shutdown_connections 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d99f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d99cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 --2- 172.21.15.7:0/1093290139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d997a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 -- 172.21.15.7:0/1093290139 >> 172.21.15.7:0/1093290139 conn(0x55ee5d956fb0 msgr2=0x55ee5d993070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 -- 172.21.15.7:0/1093290139 shutdown_connections 2024-06-13T07:00:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.760+0000 7f4dd1211740 1 -- 172.21.15.7:0/1093290139 wait complete. 2024-06-13T07:00:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.761+0000 7f4dd1211740 1 -- start start 2024-06-13T07:00:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.761+0000 7f4dd1211740 1 Processor -- start 2024-06-13T07:00:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d99d150 0x55ee5d9b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d9b1110 con 0x55ee5d997fe0 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d8ad0b0 con 0x55ee5d9975f0 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d99d150 0x55ee5d9b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34816/0 (socket says 172.21.15.7:34816) 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dc7400640 1 -- 172.21.15.7:0/2476635236 learned_addr learned my addr 172.21.15.7:0/2476635236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.762+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d9b1290 con 0x55ee5d99d150 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dd1211740 10 client.0 _ll_register_callbacks cb 0x55ee5d898f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 -- 172.21.15.7:0/2476635236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d99d150 msgr2=0x55ee5d9b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d99d150 0x55ee5d9b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 -- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 msgr2=0x55ee5d9b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.763+0000 7f4dc6a00640 1 -- 172.21.15.7:0/2476635236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5d6d2620 con 0x55ee5d997fe0 2024-06-13T07:00:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.764+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b2f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4db4005fd0 tx=0x7f4db40d2ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecca3ce86fa459bb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.764+0000 7f4dad600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:00:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.765+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db409b350 con 0x55ee5d997fe0 2024-06-13T07:00:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.765+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4db4010a30 con 0x55ee5d997fe0 2024-06-13T07:00:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.766+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc8073880 con 0x55ee5d997fe0 2024-06-13T07:00:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.767+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4db409c0f0 con 0x55ee5d997fe0 2024-06-13T07:00:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.768+0000 7f4dd1211740 10 client.15690 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.768+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee5d9bd1f0 con 0x55ee5d997fe0 2024-06-13T07:00:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.769+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8073880 con 0x55ee5d997fe0 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4db409ae40 con 0x55ee5d997fe0 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 10 client.15690 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 20 client.15690 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5d9c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 4 client.15690 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.770+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee5d8ffb50 con 0x55ee5d9be820 2024-06-13T07:00:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.771+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5d9c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.773+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5d9c0c70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba02cae332e0a2f server_cookie=9472ddfc30cec44e in_seq=0 out_seq=0 2024-06-13T07:00:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.773+0000 7f4dad600640 10 client.15690 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.774+0000 7f4dad600640 1 -- 172.21.15.7:0/2476635236 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee5d8ffb50 con 0x55ee5d9be820 2024-06-13T07:00:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.774+0000 7f4dad600640 10 client.15690 handle_command_reply: tid=0 2024-06-13T07:00:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.774+0000 7f4dd1211740 1 client.15690 shutdown 2024-06-13T07:00:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.776+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 msgr2=0x55ee5d9b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.776+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b2f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4db4005fd0 tx=0x7f4db40d2ba0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 shutdown_connections 2024-06-13T07:00:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5d9c0c70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d99d150 0x55ee5d9b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dad600640 0 client.15690 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b2f30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2476635236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.777+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 >> 172.21.15.7:0/2476635236 conn(0x55ee5d956fb0 msgr2=0x55ee5d9899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.779+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 shutdown_connections 2024-06-13T07:00:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.779+0000 7f4dd1211740 1 -- 172.21.15.7:0/2476635236 wait complete. 2024-06-13T07:00:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.780+0000 7f4dd1211740 20 client.15690 trim_cache size 0 max 16384 2024-06-13T07:00:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.830+0000 7f4dd1211740 1 Processor -- start 2024-06-13T07:00:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 -- start start 2024-06-13T07:00:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013ec10 con 0x55ee5d997fe0 2024-06-13T07:00:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013c480 con 0x55ee5d99d150 2024-06-13T07:00:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dd1211740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013a2d0 con 0x55ee5d9975f0 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.832+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37674/0 (socket says 172.21.15.7:37674) 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 -- 172.21.15.7:0/350155875 learned_addr learned my addr 172.21.15.7:0/350155875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7e00640 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 -- 172.21.15.7:0/350155875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 msgr2=0x55ee5d9c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 -- 172.21.15.7:0/350155875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 msgr2=0x55ee5d9b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 -- 172.21.15.7:0/350155875 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5d9bd010 con 0x55ee5d9975f0 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.833+0000 7f4dc7400640 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005740 tx=0x7f4dbc014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee84b7f85c851e21 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.834+0000 7f4dafe00640 1 -- 172.21.15.7:0/350155875 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0142d0 con 0x55ee5d9975f0 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.834+0000 7f4dafe00640 1 -- 172.21.15.7:0/350155875 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc01b810 con 0x55ee5d9975f0 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.834+0000 7f4dafe00640 1 -- 172.21.15.7:0/350155875 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc012a80 con 0x55ee5d9975f0 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.835+0000 7f4dd1211740 1 -- 172.21.15.7:0/350155875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 msgr2=0x55ee5d989820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.835+0000 7f4dd1211740 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005740 tx=0x7f4dbc014840 comp rx=0 tx=0).stop 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 -- 172.21.15.7:0/350155875 shutdown_connections 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 --2- 172.21.15.7:0/350155875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d9975f0 0x55ee5d989820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 -- 172.21.15.7:0/350155875 >> 172.21.15.7:0/350155875 conn(0x55ee5d956fb0 msgr2=0x55ee5d989110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 -- 172.21.15.7:0/350155875 shutdown_connections 2024-06-13T07:00:55.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:55 smithi189 ceph-mon[36616]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-06-13T07:00:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.837+0000 7f4dd1211740 1 -- 172.21.15.7:0/350155875 wait complete. 2024-06-13T07:00:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.838+0000 7f4dd1211740 1 -- start start 2024-06-13T07:00:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.838+0000 7f4dd1211740 1 Processor -- start 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.840+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.840+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.840+0000 7f4dd1211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.840+0000 7f4dd1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013a940 con 0x55ee5d9975f0 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.840+0000 7f4dd1211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013bdb0 con 0x55ee5d99d150 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9b8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34844/0 (socket says 172.21.15.7:34844) 2024-06-13T07:00:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dc7e00640 1 -- 172.21.15.7:0/2655681664 learned_addr learned my addr 172.21.15.7:0/2655681664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc013b010 con 0x55ee5d997fe0 2024-06-13T07:00:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.841+0000 7f4dd1211740 10 client.0 _ll_register_callbacks cb 0x7f4db4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 -- 172.21.15.7:0/2655681664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d997fe0 msgr2=0x55ee5d9b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 -- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 msgr2=0x55ee5d9b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.842+0000 7f4dc7400640 1 -- 172.21.15.7:0/2655681664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5d9b9060 con 0x55ee5d9975f0 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.843+0000 7f4dc7400640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b7570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0098d0 tx=0x7f4dbc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88bd89048420a4b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.843+0000 7f4dad600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:00:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.844+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc031060 con 0x55ee5d9975f0 2024-06-13T07:00:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.844+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4dbc01b530 con 0x55ee5d9975f0 2024-06-13T07:00:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.844+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc01b810 con 0x55ee5d9975f0 2024-06-13T07:00:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.847+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc038050 con 0x55ee5d9975f0 2024-06-13T07:00:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.848+0000 7f4dd1211740 10 client.15693 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.848+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee5da29550 con 0x55ee5d9975f0 2024-06-13T07:00:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.849+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc01b810 con 0x55ee5d9975f0 2024-06-13T07:00:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f4dbc00f040 con 0x55ee5d9975f0 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 10 client.15693 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 20 client.15693 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5da2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 4 client.15693 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:00:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.850+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee5d8f9ea0 con 0x55ee5d9be820 2024-06-13T07:00:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.852+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5da2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.852+0000 7f4dc6a00640 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5da2bdf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96b044f1c275cfd server_cookie=e4b159295be77d91 in_seq=0 out_seq=0 2024-06-13T07:00:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:55.852+0000 7f4dad600640 10 client.15693 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:55 smithi175 ceph-mon[36886]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-06-13T07:00:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.015+0000 7f4dad600640 1 -- 172.21.15.7:0/2655681664 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ee5d8f9ea0 con 0x55ee5d9be820 2024-06-13T07:00:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.016+0000 7f4dad600640 10 client.15693 handle_command_reply: tid=0 2024-06-13T07:00:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.016+0000 7f4dd1211740 1 client.15693 shutdown 2024-06-13T07:00:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.017+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 msgr2=0x55ee5d9b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.017+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b7570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0098d0 tx=0x7f4dbc009b30 comp rx=0 tx=0).stop 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.017+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 shutdown_connections 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.017+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ee5d9be820 0x55ee5da2bdf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d99d150 0x55ee5d9b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d997fe0 0x55ee5d9b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dd1211740 1 --2- 172.21.15.7:0/2655681664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d9975f0 0x55ee5d9b7570 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 >> 172.21.15.7:0/2655681664 conn(0x55ee5d956fb0 msgr2=0x55ee5d9965c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dad600640 0 client.15693 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.018+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 shutdown_connections 2024-06-13T07:00:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.019+0000 7f4dd1211740 1 -- 172.21.15.7:0/2655681664 wait complete. 2024-06-13T07:00:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.019+0000 7f4dd1211740 20 client.15693 trim_cache size 0 max 16384 2024-06-13T07:00:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:00:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:00:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:00:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 msgr2=0x7f4db00a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 0x7f4db00a51a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4dc809dff0 tx=0x7f4dc806a470 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 msgr2=0x7f4dc012e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 0x7f4dc012e4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e790 tx=0x7f4db400ec50 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 shutdown_connections 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f4db00a2ce0 0x7f4db00a51a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc0097930 0x7f4dc0135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc0096f70 0x7f4dc012e4a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 --2- 172.21.15.7:0/468359979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0095d80 0x7f4dc012df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 >> 172.21.15.7:0/468359979 conn(0x7f4dc00912d0 msgr2=0x7f4dc0006b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 shutdown_connections 2024-06-13T07:00:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:00:56.021+0000 7f4dcce00640 1 -- 172.21.15.7:0/468359979 wait complete. 2024-06-13T07:00:57.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.621+0000 7fca7a400700 1 -- 172.21.15.7:0/108500924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0a1670 msgr2=0x7fca6c0a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.621+0000 7fca7a400700 1 --2- 172.21.15.7:0/108500924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0a1670 0x7fca6c0a9240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fca74067f10 tx=0x7fca74068aa0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.622+0000 7fca7a400700 1 -- 172.21.15.7:0/108500924 shutdown_connections 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.622+0000 7fca7a400700 1 --2- 172.21.15.7:0/108500924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c0a1670 0x7fca6c0a9240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.622+0000 7fca7a400700 1 --2- 172.21.15.7:0/108500924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c09c5e0 0x7fca6c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.622+0000 7fca7a400700 1 --2- 172.21.15.7:0/108500924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09cf90 0x7fca6c0a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.622+0000 7fca7a400700 1 -- 172.21.15.7:0/108500924 >> 172.21.15.7:0/108500924 conn(0x7fca6c00c880 msgr2=0x7fca6c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.623+0000 7fca7a400700 1 -- 172.21.15.7:0/108500924 shutdown_connections 2024-06-13T07:00:57.624 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:57 smithi007 ceph-mon[32077]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2024-06-13T07:00:57.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.623+0000 7fca7a400700 1 -- 172.21.15.7:0/108500924 wait complete. 2024-06-13T07:00:57.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.626+0000 7fca7a400700 1 Processor -- start 2024-06-13T07:00:57.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.627+0000 7fca7a400700 1 -- start start 2024-06-13T07:00:57.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.628+0000 7fca7a400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.628+0000 7fca7a400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c09cf90 0x7fca6c0162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca79000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca7a400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 0x7fca6c00f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca7a400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0ac5e0 con 0x7fca6c0a1670 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca7a400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0ac460 con 0x7fca6c09cf90 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca7a400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0ac760 con 0x7fca6c09c5e0 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca79000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37700/0 (socket says 172.21.15.7:37700) 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.629+0000 7fca79000700 1 -- 172.21.15.7:0/4020758423 learned_addr learned my addr 172.21.15.7:0/4020758423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:57.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.630+0000 7fca73e00700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c09cf90 0x7fca6c0162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.630+0000 7fca79a00700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 0x7fca6c00f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.630+0000 7fca79000700 1 -- 172.21.15.7:0/4020758423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c09cf90 msgr2=0x7fca6c0162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.630+0000 7fca79000700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c09cf90 0x7fca6c0162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.630+0000 7fca79000700 1 -- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 msgr2=0x7fca6c00f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.631+0000 7fca79000700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 0x7fca6c00f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.631+0000 7fca79000700 1 -- 172.21.15.7:0/4020758423 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c00f9a0 con 0x7fca6c09c5e0 2024-06-13T07:00:57.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.631+0000 7fca79a00700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 0x7fca6c00f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.632+0000 7fca79000700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d980 tx=0x7fca6800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.632+0000 7fca71600700 1 -- 172.21.15.7:0/4020758423 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68016070 con 0x7fca6c09c5e0 2024-06-13T07:00:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.632+0000 7fca71600700 1 -- 172.21.15.7:0/4020758423 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca68004500 con 0x7fca6c09c5e0 2024-06-13T07:00:57.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.632+0000 7fca71600700 1 -- 172.21.15.7:0/4020758423 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004d00 con 0x7fca6c09c5e0 2024-06-13T07:00:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.633+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c00fc90 con 0x7fca6c09c5e0 2024-06-13T07:00:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.633+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c013b70 con 0x7fca6c09c5e0 2024-06-13T07:00:57.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.639+0000 7fca71600700 1 -- 172.21.15.7:0/4020758423 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fca68005020 con 0x7fca6c09c5e0 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 Processor -- start 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 -- start start 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x564321741420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x564321741de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432174c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56432164ea10 con 0x564321741960 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564321691b00 con 0x56432174a570 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.640+0000 7fca8646cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56432164c8f0 con 0x5643216bbe40 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.641+0000 7fca79a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x564321741420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.641+0000 7fca79a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x564321741420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37710/0 (socket says 172.21.15.7:37710) 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.641+0000 7fca79a00700 1 -- 172.21.15.7:0/2442964199 learned_addr learned my addr 172.21.15.7:0/2442964199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.641+0000 7fca73e00700 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432174c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.641+0000 7fca79000700 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x564321741de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca73e00700 1 -- 172.21.15.7:0/2442964199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 msgr2=0x564321741420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca73e00700 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x564321741420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca73e00700 1 -- 172.21.15.7:0/2442964199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 msgr2=0x564321741de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca73e00700 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x564321741de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca73e00700 1 -- 172.21.15.7:0/2442964199 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564321639f90 con 0x56432174a570 2024-06-13T07:00:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.642+0000 7fca71600700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 0x7fca5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.643+0000 7fca71600700 1 -- 172.21.15.7:0/4020758423 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca680cf1d0 con 0x7fca6c09c5e0 2024-06-13T07:00:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.643+0000 7fca73e00700 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432174c960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca6c010690 tx=0x7fca6000dec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f51761f9d00a228 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.644+0000 7fca66000700 1 -- 172.21.15.7:0/2442964199 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6000fa80 con 0x56432174a570 2024-06-13T07:00:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.644+0000 7fca66000700 1 -- 172.21.15.7:0/2442964199 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6000fc20 con 0x56432174a570 2024-06-13T07:00:57.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.644+0000 7fca66000700 1 -- 172.21.15.7:0/2442964199 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60010660 con 0x56432174a570 2024-06-13T07:00:57.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.644+0000 7fca79a00700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 0x7fca5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.645+0000 7fca8646cb80 1 -- 172.21.15.7:0/2442964199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 msgr2=0x56432174c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.645+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432174c960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca6c010690 tx=0x7fca6000dec0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 -- 172.21.15.7:0/2442964199 shutdown_connections 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432174c960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x564321741420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2442964199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x564321741de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 -- 172.21.15.7:0/2442964199 >> 172.21.15.7:0/2442964199 conn(0x56432166c170 msgr2=0x56432173b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 -- 172.21.15.7:0/2442964199 shutdown_connections 2024-06-13T07:00:57.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.646+0000 7fca8646cb80 1 -- 172.21.15.7:0/2442964199 wait complete. 2024-06-13T07:00:57.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.647+0000 7fca8646cb80 1 -- start start 2024-06-13T07:00:57.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.647+0000 7fca8646cb80 1 Processor -- start 2024-06-13T07:00:57.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x564321749c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56432162df60 con 0x5643216bbe40 2024-06-13T07:00:57.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56432168dac0 con 0x564321741960 2024-06-13T07:00:57.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56432174d1e0 con 0x56432174a570 2024-06-13T07:00:57.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.648+0000 7fca8646cb80 10 client.0 _ll_register_callbacks cb 0x56432162b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:57.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34874/0 (socket says 172.21.15.7:34874) 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 -- 172.21.15.7:0/2448717734 learned_addr learned my addr 172.21.15.7:0/2448717734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca79000700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x564321749c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 -- 172.21.15.7:0/2448717734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 msgr2=0x5643217474c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217474c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 -- 172.21.15.7:0/2448717734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 msgr2=0x564321749c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x564321749c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.649+0000 7fca73e00700 1 -- 172.21.15.7:0/2448717734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643215e08d0 con 0x564321741960 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.650+0000 7fca73e00700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca60011f10 tx=0x7fca60011d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2094a006f8da3a18 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.650+0000 7fca5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.650+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6002b060 con 0x564321741960 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.650+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca60011650 con 0x564321741960 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.650+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6000fc20 con 0x564321741960 2024-06-13T07:00:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.651+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca6000fa70 con 0x564321741960 2024-06-13T07:00:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.651+0000 7fca79a00700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 0x7fca5c0a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fca74051a30 tx=0x7fca74081040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.651+0000 7fca8646cb80 10 client.26105 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.651+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5643217578e0 con 0x564321741960 2024-06-13T07:00:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.651+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca6000fc20 con 0x564321741960 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fca6001b410 con 0x564321741960 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 10 client.26105 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 20 client.26105 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56432175a190 0x56432175c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 4 client.26105 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:00:57.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.652+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56432169f000 con 0x56432175a190 2024-06-13T07:00:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.654+0000 7fca79000700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56432175a190 0x56432175c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.655+0000 7fca79000700 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56432175a190 0x56432175c650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81231dd84c6e7871 server_cookie=1f4a83ad2cec3a67 in_seq=0 out_seq=0 2024-06-13T07:00:57.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.655+0000 7fca5aa00700 10 client.26105 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:57.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.655+0000 7fca5aa00700 1 -- 172.21.15.7:0/2448717734 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56432169f000 con 0x56432175a190 2024-06-13T07:00:57.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.655+0000 7fca5aa00700 10 client.26105 handle_command_reply: tid=0 2024-06-13T07:00:57.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.656+0000 7fca8646cb80 1 client.26105 shutdown 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 msgr2=0x564321746f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca60011f10 tx=0x7fca60011d40 comp rx=0 tx=0).stop 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 shutdown_connections 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56432175a190 0x56432175c650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x564321749c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 --2- 172.21.15.7:0/2448717734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x564321746f80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 >> 172.21.15.7:0/2448717734 conn(0x56432166c170 msgr2=0x56432173b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca5aa00700 0 client.26105 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:57.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 shutdown_connections 2024-06-13T07:00:57.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 1 -- 172.21.15.7:0/2448717734 wait complete. 2024-06-13T07:00:57.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.657+0000 7fca8646cb80 20 client.26105 trim_cache size 0 max 16384 2024-06-13T07:00:57.669 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:57 smithi175 ceph-mon[36886]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2024-06-13T07:00:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.690+0000 7fca8646cb80 1 Processor -- start 2024-06-13T07:00:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.690+0000 7fca8646cb80 1 -- start start 2024-06-13T07:00:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x56432173b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c153310 con 0x564321741960 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c150b40 con 0x56432174a570 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca8646cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c14e990 con 0x5643216bbe40 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca79a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca79000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca79a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34880/0 (socket says 172.21.15.7:34880) 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca79000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37716/0 (socket says 172.21.15.7:37716) 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca79a00700 1 -- 172.21.15.7:0/111890617 learned_addr learned my addr 172.21.15.7:0/111890617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.691+0000 7fca73e00700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x56432173b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 -- 172.21.15.7:0/111890617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 msgr2=0x5643217474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 -- 172.21.15.7:0/111890617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 msgr2=0x56432173b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x56432173b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 -- 172.21.15.7:0/111890617 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432174d1e0 con 0x56432174a570 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca73e00700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x56432173b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79000700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.693+0000 7fca79a00700 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca74067f10 tx=0x7fca7408e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98e148b9798c278 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.694+0000 7fca65600700 1 -- 172.21.15.7:0/111890617 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74084040 con 0x56432174a570 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.694+0000 7fca65600700 1 -- 172.21.15.7:0/111890617 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7409f070 con 0x56432174a570 2024-06-13T07:00:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.694+0000 7fca65600700 1 -- 172.21.15.7:0/111890617 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7407ec50 con 0x56432174a570 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 -- 172.21.15.7:0/111890617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 msgr2=0x56432173b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca74067f10 tx=0x7fca7408e000 comp rx=0 tx=0).stop 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 -- 172.21.15.7:0/111890617 shutdown_connections 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56432174a570 0x56432173b830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643216bbe40 0x5643217474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 --2- 172.21.15.7:0/111890617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564321741960 0x56432173b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.695+0000 7fca8646cb80 1 -- 172.21.15.7:0/111890617 >> 172.21.15.7:0/111890617 conn(0x56432166c170 msgr2=0x564321745870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.696+0000 7fca8646cb80 1 -- 172.21.15.7:0/111890617 shutdown_connections 2024-06-13T07:00:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.696+0000 7fca8646cb80 1 -- 172.21.15.7:0/111890617 wait complete. 2024-06-13T07:00:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.696+0000 7fca8646cb80 1 -- start start 2024-06-13T07:00:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.696+0000 7fca8646cb80 1 Processor -- start 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x5643217529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c14f000 con 0x5643216bbe40 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c150470 con 0x564321741960 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c14f6d0 con 0x56432174a570 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca8646cb80 10 client.0 _ll_register_callbacks cb 0x56432162b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca79000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.697+0000 7fca79000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60970/0 (socket says 172.21.15.7:60970) 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 -- 172.21.15.7:0/376355378 learned_addr learned my addr 172.21.15.7:0/376355378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:00:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca73e00700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x5643217529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 -- 172.21.15.7:0/376355378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 msgr2=0x5643217435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 -- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 msgr2=0x5643217529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x5643217529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 -- 172.21.15.7:0/376355378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564321743b20 con 0x5643216bbe40 2024-06-13T07:00:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca73e00700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x5643217529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:00:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca79000700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca6809b400 tx=0x7fca6800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e8823a39dd11c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:00:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.698+0000 7fca5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.699+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca680b4060 con 0x5643216bbe40 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.699+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fca680997a0 con 0x5643216bbe40 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.699+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7409f070 con 0x5643216bbe40 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fca6809b060 con 0x5643216bbe40 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 10 client.15705 fetch_fsmap learned FSMap version 34 2024-06-13T07:00:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5643217c4780 con 0x5643216bbe40 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7409f070 con 0x5643216bbe40 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fca680a3e70 con 0x5643216bbe40 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 10 client.15705 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 20 client.15705 populate_metadata read hostname 'smithi007' 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5643217c4eb0 0x5643217c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 4 client.15705 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.700+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564321742be0 con 0x5643217c4eb0 2024-06-13T07:00:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.701+0000 7fca73e00700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5643217c4eb0 0x5643217c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:00:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.703+0000 7fca5aa00700 10 client.15705 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.703+0000 7fca73e00700 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5643217c4eb0 0x5643217c72d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef62c8a2bf58a03 server_cookie=4fa95007aae650fa in_seq=0 out_seq=0 2024-06-13T07:00:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca5aa00700 1 -- 172.21.15.7:0/376355378 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564321742be0 con 0x5643217c4eb0 2024-06-13T07:00:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca5aa00700 10 client.15705 handle_command_reply: tid=0 2024-06-13T07:00:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 client.15705 shutdown 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 msgr2=0x5643217524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca6809b400 tx=0x7fca6800be90 comp rx=0 tx=0).stop 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 shutdown_connections 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5643217c4eb0 0x5643217c72d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56432174a570 0x5643217435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643216bbe40 0x5643217524a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 --2- 172.21.15.7:0/376355378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564321741960 0x5643217529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.704+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 >> 172.21.15.7:0/376355378 conn(0x56432166c170 msgr2=0x56432173b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.705+0000 7fca5aa00700 0 client.15705 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:00:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.705+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 shutdown_connections 2024-06-13T07:00:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.705+0000 7fca8646cb80 1 -- 172.21.15.7:0/376355378 wait complete. 2024-06-13T07:00:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.705+0000 7fca8646cb80 20 client.15705 trim_cache size 0 max 16384 2024-06-13T07:00:57.706 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:00:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:00:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:00:57.707 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:00:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 msgr2=0x7fca5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 0x7fca5c0a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fca74051a30 tx=0x7fca74081040 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 msgr2=0x7fca6c015db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca6800d980 tx=0x7fca6800dd40 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 shutdown_connections 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca6c0a1670 0x7fca6c00f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca6c09c5e0 0x7fca6c015db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fca5c0a2d70 0x7fca5c0a5230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 --2- 172.21.15.7:0/4020758423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca6c09cf90 0x7fca6c0162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.707+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 >> 172.21.15.7:0/4020758423 conn(0x7fca6c00c880 msgr2=0x7fca6c00a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:00:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.708+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 shutdown_connections 2024-06-13T07:00:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:00:57.708+0000 7fca7a400700 1 -- 172.21.15.7:0/4020758423 wait complete. 2024-06-13T07:00:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:57 smithi189 ceph-mon[36616]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2024-06-13T07:00:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 295.0 s, 589.3 tps, lat 167.649 ms stddev 178.983 2024-06-13T07:00:58.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:58 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:00:58.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:58 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:00:58.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:58 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:00:58.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:58 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:00:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:58 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:00:58.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:58 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:00:59.222 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:00:59.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:00:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:59 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:00:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:00:59 smithi189 ceph-mon[36616]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-06-13T07:00:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:59 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:00:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:00:59 smithi175 ceph-mon[36886]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-06-13T07:00:59.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:59 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:00:59.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:00:59 smithi007 ceph-mon[32077]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 10 MiB/s wr, 744 op/s 2024-06-13T07:00:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:01.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:01 smithi007 ceph-mon[32077]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 580 op/s 2024-06-13T07:01:01.794 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:01 smithi189 ceph-mon[36616]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 580 op/s 2024-06-13T07:01:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:01 smithi175 ceph-mon[36886]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 580 op/s 2024-06-13T07:01:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 300.0 s, 599.4 tps, lat 167.913 ms stddev 247.905 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.329+0000 7fab29e00700 1 -- 172.21.15.7:0/331243728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab2410eac0 msgr2=0x7fab2410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.329+0000 7fab29e00700 1 --2- 172.21.15.7:0/331243728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab2410eac0 0x7fab2410eea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab14009990 tx=0x7fab14009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 -- 172.21.15.7:0/331243728 shutdown_connections 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 --2- 172.21.15.7:0/331243728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab24109480 0x7fab24115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 --2- 172.21.15.7:0/331243728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab2410eac0 0x7fab2410eea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 --2- 172.21.15.7:0/331243728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab24108ac0 0x7fab24108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 -- 172.21.15.7:0/331243728 >> 172.21.15.7:0/331243728 conn(0x7fab24075c60 msgr2=0x7fab241072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 -- 172.21.15.7:0/331243728 shutdown_connections 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.330+0000 7fab29e00700 1 -- 172.21.15.7:0/331243728 wait complete. 2024-06-13T07:01:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.331+0000 7fab29e00700 1 Processor -- start 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.332+0000 7fab29e00700 1 -- start start 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.332+0000 7fab29e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab24108ac0 0x7fab24073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.332+0000 7fab29e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.332+0000 7fab29e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2410eac0 0x7fab2406c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36654/0 (socket says 172.21.15.7:36654) 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 -- 172.21.15.7:0/616095918 learned_addr learned my addr 172.21.15.7:0/616095918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab24118e60 con 0x7fab2410eac0 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab24118ce0 con 0x7fab24109480 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab24118fe0 con 0x7fab24108ac0 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab23400700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2410eac0 0x7fab2406c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 -- 172.21.15.7:0/616095918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab24108ac0 msgr2=0x7fab24073fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab24108ac0 0x7fab24073fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 -- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2410eac0 msgr2=0x7fab2406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2410eac0 0x7fab2406c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.333+0000 7fab22000700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2406cd30 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.334+0000 7fab22000700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d980 tx=0x7fab1000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.334+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10017070 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.334+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10004d10 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.334+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241b4100 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.334+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab100055c0 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.335+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241b4610 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.337+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2410a200 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.338+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fab10004750 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.340+0000 7fab28a00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 0x7fab080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.340+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fab100cf310 con 0x7fab24109480 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.340+0000 7fab22a00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 0x7fab080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.342+0000 7fab22a00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 0x7fab080a5160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fab140098e0 tx=0x7fab14005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.345+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab10020020 con 0x7fab24109480 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.847+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab2402dca0 con 0x7fab24109480 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.851+0000 7fab28a00700 1 -- 172.21.15.7:0/616095918 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11871 (secure 0 0 0) 0x7fab1009bed0 con 0x7fab24109480 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:00:15.083820+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.859+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 msgr2=0x7fab080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.859+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 0x7fab080a5160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fab140098e0 tx=0x7fab14005e80 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 msgr2=0x7fab2406bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d980 tx=0x7fab1000dd40 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 shutdown_connections 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2410eac0 0x7fab2406c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab24108ac0 0x7fab24073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab080a2ca0 0x7fab080a5160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 --2- 172.21.15.7:0/616095918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fab24109480 0x7fab2406bf80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 >> 172.21.15.7:0/616095918 conn(0x7fab24075c60 msgr2=0x7fab24076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.860+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 shutdown_connections 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:02.861+0000 7fab29e00700 1 -- 172.21.15.7:0/616095918 wait complete. 2024-06-13T07:01:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-06-13T07:01:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:03 smithi007 ceph-mon[32077]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-06-13T07:01:03.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:03 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/616095918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:03 smithi189 ceph-mon[36616]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-06-13T07:01:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:03 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/616095918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:03 smithi175 ceph-mon[36886]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 695 op/s 2024-06-13T07:01:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:03 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/616095918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:01:03.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:01:04.687 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:05 smithi007 ceph-mon[32077]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 672 op/s 2024-06-13T07:01:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:05 smithi189 ceph-mon[36616]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 672 op/s 2024-06-13T07:01:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:05 smithi175 ceph-mon[36886]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 12 MiB/s wr, 672 op/s 2024-06-13T07:01:07.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.156+0000 7f051a600700 1 -- 172.21.15.7:0/2108572954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051406a170 msgr2=0x7f051406a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.156+0000 7f051a600700 1 --2- 172.21.15.7:0/2108572954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051406a170 0x7f051406a550 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0504009990 tx=0x7f0504009ca0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.157+0000 7f051a600700 1 -- 172.21.15.7:0/2108572954 shutdown_connections 2024-06-13T07:01:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.157+0000 7f051a600700 1 --2- 172.21.15.7:0/2108572954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051410ef40 0x7f0514111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.157+0000 7f051a600700 1 --2- 172.21.15.7:0/2108572954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051406a170 0x7f051406a550 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.157+0000 7f051a600700 1 --2- 172.21.15.7:0/2108572954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406aa90 0x7f051410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.157+0000 7f051a600700 1 -- 172.21.15.7:0/2108572954 >> 172.21.15.7:0/2108572954 conn(0x7f05140782c0 msgr2=0x7f05140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.158+0000 7f051a600700 1 -- 172.21.15.7:0/2108572954 shutdown_connections 2024-06-13T07:01:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.158+0000 7f051a600700 1 -- 172.21.15.7:0/2108572954 wait complete. 2024-06-13T07:01:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.160+0000 7f051a600700 1 Processor -- start 2024-06-13T07:01:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.160+0000 7f051a600700 1 -- start start 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 0x7f051419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 0x7f051419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 0x7f051419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f0513400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 0x7f051419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f0513400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 0x7f051419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36670/0 (socket says 172.21.15.7:36670) 2024-06-13T07:01:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f0513e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 0x7f051419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f0513400700 1 -- 172.21.15.7:0/2028992252 learned_addr learned my addr 172.21.15.7:0/2028992252 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f0512a00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 0x7f051419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05141149c0 con 0x7f051410ef40 2024-06-13T07:01:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0514114840 con 0x7f051406a170 2024-06-13T07:01:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.161+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0514114b40 con 0x7f051406aa90 2024-06-13T07:01:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0512a00700 1 -- 172.21.15.7:0/2028992252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 msgr2=0x7f051419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0512a00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 0x7f051419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0512a00700 1 -- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 msgr2=0x7f051419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0512a00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 0x7f051419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0512a00700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051419fe60 con 0x7f051406aa90 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.162+0000 7f0513e00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 0x7f051419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.163+0000 7f0512a00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 0x7f051419b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f050000e9f0 tx=0x7f050000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.163+0000 7f050be00700 1 -- 172.21.15.7:0/2028992252 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050000cbd0 con 0x7f051406aa90 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.163+0000 7f050be00700 1 -- 172.21.15.7:0/2028992252 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050000cd70 con 0x7f051406aa90 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.163+0000 7f050be00700 1 -- 172.21.15.7:0/2028992252 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500018840 con 0x7f051406aa90 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.163+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05141a0150 con 0x7f051406aa90 2024-06-13T07:01:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.164+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05141a79d0 con 0x7f051406aa90 2024-06-13T07:01:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.168+0000 7f050be00700 1 -- 172.21.15.7:0/2028992252 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f05000189e0 con 0x7f051406aa90 2024-06-13T07:01:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.171+0000 7f050be00700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 0x7f04f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.171+0000 7f050be00700 1 -- 172.21.15.7:0/2028992252 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05000cf740 con 0x7f051406aa90 2024-06-13T07:01:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.172+0000 7f0513400700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 0x7f04f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.173+0000 7f0513400700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 0x7f04f80a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0504006010 tx=0x7f050401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 Processor -- start 2024-06-13T07:01:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 -- start start 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9712f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9712f5b0 0x559a9712fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a9703ea10 con 0x559a9712f5b0 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a97081b00 con 0x559a970abe40 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.174+0000 7f0524f39b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a9703c8f0 con 0x559a9713a300 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9712f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38614/0 (socket says 172.21.15.7:38614) 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513e00700 1 -- 172.21.15.7:0/3458929502 learned_addr learned my addr 172.21.15.7:0/3458929502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0512a00700 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9712f5b0 0x559a9712fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 -- 172.21.15.7:0/3458929502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 msgr2=0x559a9713c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 -- 172.21.15.7:0/3458929502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9712f5b0 msgr2=0x559a9712fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9712f5b0 0x559a9712fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513400700 1 -- 172.21.15.7:0/3458929502 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a97029f90 con 0x559a970abe40 2024-06-13T07:01:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.175+0000 7f0513e00700 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.176+0000 7f0513400700 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9712f070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f050401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23903b50960bfe8c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.177+0000 7f0508c00700 1 -- 172.21.15.7:0/3458929502 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400f9c0 con 0x559a970abe40 2024-06-13T07:01:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.177+0000 7f0508c00700 1 -- 172.21.15.7:0/3458929502 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0504032070 con 0x559a970abe40 2024-06-13T07:01:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.177+0000 7f0508c00700 1 -- 172.21.15.7:0/3458929502 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400f9c0 con 0x559a970abe40 2024-06-13T07:01:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 -- 172.21.15.7:0/3458929502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 msgr2=0x559a9712f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9712f070 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f050401e000 comp rx=0 tx=0).stop 2024-06-13T07:01:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 -- 172.21.15.7:0/3458929502 shutdown_connections 2024-06-13T07:01:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9713a300 0x559a9713c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9712f070 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.179+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3458929502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9712f5b0 0x559a9712fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.180+0000 7f0524f39b80 1 -- 172.21.15.7:0/3458929502 >> 172.21.15.7:0/3458929502 conn(0x559a9705c170 msgr2=0x559a9712b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.180+0000 7f0524f39b80 1 -- 172.21.15.7:0/3458929502 shutdown_connections 2024-06-13T07:01:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.180+0000 7f0524f39b80 1 -- 172.21.15.7:0/3458929502 wait complete. 2024-06-13T07:01:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.181+0000 7f0524f39b80 1 -- start start 2024-06-13T07:01:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.181+0000 7f0524f39b80 1 Processor -- start 2024-06-13T07:01:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a97139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a97137280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559a9701df60 con 0x559a9713a300 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559a9707dac0 con 0x559a970abe40 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a9713d000 con 0x559a9712f5b0 2024-06-13T07:01:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0524f39b80 10 client.0 _ll_register_callbacks cb 0x559a9701b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0512a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0512a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38618/0 (socket says 172.21.15.7:38618) 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.182+0000 7f0512a00700 1 -- 172.21.15.7:0/3310989999 learned_addr learned my addr 172.21.15.7:0/3310989999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 -- 172.21.15.7:0/3310989999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 msgr2=0x559a97139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a97139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 -- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 msgr2=0x559a97137280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a97137280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 -- 172.21.15.7:0/3310989999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a96fd08d0 con 0x559a9712f5b0 2024-06-13T07:01:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.183+0000 7f0512a00700 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05000978e0 tx=0x7f050009c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95416e41a057c813 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.184+0000 7f04fd600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:07.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.184+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050009ffb0 con 0x559a9712f5b0 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.184+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0500099f00 con 0x559a9712f5b0 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.184+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0504032070 con 0x559a9712f5b0 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.185+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f05000a4870 con 0x559a9712f5b0 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.185+0000 7f0524f39b80 10 client.26101 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.185+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a97160f10 con 0x559a9712f5b0 2024-06-13T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.185+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0504032070 con 0x559a9712f5b0 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.185+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0500019ce0 con 0x559a9712f5b0 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 10 client.26101 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 10 client.26101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 20 client.26101 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971635f0 0x559a97165a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 4 client.26101 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a9708f000 con 0x559a971635f0 2024-06-13T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.186+0000 7f0513400700 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971635f0 0x559a97165a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.187+0000 7f0513400700 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971635f0 0x559a97165a20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f3ebec8a3c66ffe server_cookie=1963d7bb6e41aa65 in_seq=0 out_seq=0 2024-06-13T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.187+0000 7f04fd600700 10 client.26101 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.189+0000 7f04fd600700 1 -- 172.21.15.7:0/3310989999 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a9708f000 con 0x559a971635f0 2024-06-13T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.189+0000 7f04fd600700 10 client.26101 handle_command_reply: tid=0 2024-06-13T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.189+0000 7f0524f39b80 1 client.26101 shutdown 2024-06-13T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 msgr2=0x559a97136d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f05000978e0 tx=0x7f050009c140 comp rx=0 tx=0).stop 2024-06-13T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 shutdown_connections 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a97137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971635f0 0x559a97165a20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a97139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 --2- 172.21.15.7:0/3310989999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a97136d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f04fd600700 0 client.26101 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 >> 172.21.15.7:0/3310989999 conn(0x559a9705c170 msgr2=0x559a9712a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.190+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 shutdown_connections 2024-06-13T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.191+0000 7f0524f39b80 1 -- 172.21.15.7:0/3310989999 wait complete. 2024-06-13T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.191+0000 7f0524f39b80 20 client.26101 trim_cache size 0 max 16384 2024-06-13T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.220+0000 7f0524f39b80 1 Processor -- start 2024-06-13T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.220+0000 7f0524f39b80 1 -- start start 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.220+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a970abe40 0x559a9712b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9712a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0524f39b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0514072ca0 con 0x559a970abe40 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0524f39b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05140704d0 con 0x559a9713a300 2024-06-13T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0524f39b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f051406e2e0 con 0x559a9712f5b0 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0513e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0513e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36694/0 (socket says 172.21.15.7:36694) 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0513e00700 1 -- 172.21.15.7:0/13123962 learned_addr learned my addr 172.21.15.7:0/13123962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0513400700 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a970abe40 0x559a9712b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.221+0000 7f0513e00700 1 -- 172.21.15.7:0/13123962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 msgr2=0x559a9712a5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f0513e00700 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9712a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f0513e00700 1 -- 172.21.15.7:0/13123962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a970abe40 msgr2=0x559a9712b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f0513e00700 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a970abe40 0x559a9712b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f0513e00700 1 -- 172.21.15.7:0/13123962 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a97130ba0 con 0x559a9713a300 2024-06-13T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f0513e00700 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f050c009dd0 tx=0x7f050c0076e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50e21fbe179cc07c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f04ffe00700 1 -- 172.21.15.7:0/13123962 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c01a020 con 0x559a9713a300 2024-06-13T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f04ffe00700 1 -- 172.21.15.7:0/13123962 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c016940 con 0x559a9713a300 2024-06-13T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.222+0000 7f04ffe00700 1 -- 172.21.15.7:0/13123962 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c015a20 con 0x559a9713a300 2024-06-13T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 -- 172.21.15.7:0/13123962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 msgr2=0x559a97139580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f050c009dd0 tx=0x7f050c0076e0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 -- 172.21.15.7:0/13123962 shutdown_connections 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a9713a300 0x559a97139580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a970abe40 0x559a9712b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 --2- 172.21.15.7:0/13123962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9712a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.223+0000 7f0524f39b80 1 -- 172.21.15.7:0/13123962 >> 172.21.15.7:0/13123962 conn(0x559a9705c170 msgr2=0x559a97129eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.224+0000 7f0524f39b80 1 -- 172.21.15.7:0/13123962 shutdown_connections 2024-06-13T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.225+0000 7f0524f39b80 1 -- 172.21.15.7:0/13123962 wait complete. 2024-06-13T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.225+0000 7f0524f39b80 1 -- start start 2024-06-13T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.225+0000 7f0524f39b80 1 Processor -- start 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9714b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a9716a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f051406e990 con 0x559a9713a300 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f051406fe00 con 0x559a970abe40 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f051406f060 con 0x559a9712f5b0 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0524f39b80 10 client.0 _ll_register_callbacks cb 0x559a9701b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0513400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0513400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36708/0 (socket says 172.21.15.7:36708) 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0513400700 1 -- 172.21.15.7:0/1626630311 learned_addr learned my addr 172.21.15.7:0/1626630311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.227+0000 7f0513e00700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a9716a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0512a00700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9714b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 -- 172.21.15.7:0/1626630311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 msgr2=0x559a9714b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9714b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 -- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 msgr2=0x559a9716a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a9716a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 -- 172.21.15.7:0/1626630311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9716b1b0 con 0x559a970abe40 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513e00700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a9716a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f0513400700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f0504023a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158f981eb65a6f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f04fd600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.228+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050403f070 con 0x559a970abe40 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0504023d90 con 0x559a970abe40 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c016940 con 0x559a970abe40 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0504048050 con 0x559a970abe40 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f0524f39b80 10 client.26132 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a9714a6e0 con 0x559a970abe40 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.229+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c016940 con 0x559a970abe40 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0504042020 con 0x559a970abe40 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 10 client.26132 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 10 client.26132 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 20 client.26132 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971ce2c0 0x559a971d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 4 client.26132 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.230+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a97089580 con 0x559a971ce2c0 2024-06-13T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.231+0000 7f0512a00700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971ce2c0 0x559a971d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.232+0000 7f0512a00700 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971ce2c0 0x559a971d06b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dea714435baddae3 server_cookie=a5277f2b7380bbac in_seq=0 out_seq=0 2024-06-13T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.232+0000 7f04fd600700 10 client.26132 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.233+0000 7f04fd600700 1 -- 172.21.15.7:0/1626630311 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a97089580 con 0x559a971ce2c0 2024-06-13T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.233+0000 7f04fd600700 10 client.26132 handle_command_reply: tid=0 2024-06-13T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.233+0000 7f0524f39b80 1 client.26132 shutdown 2024-06-13T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 msgr2=0x559a9714ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f0504023a00 comp rx=0 tx=0).stop 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 shutdown_connections 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559a9713a300 0x559a9716a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x559a971ce2c0 0x559a971d06b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559a970abe40 0x559a9714ae30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 --2- 172.21.15.7:0/1626630311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a9712f5b0 0x559a9714b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 >> 172.21.15.7:0/1626630311 conn(0x559a9705c170 msgr2=0x559a9714e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.234+0000 7f04fd600700 0 client.26132 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.235+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 shutdown_connections 2024-06-13T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.236+0000 7f0524f39b80 1 -- 172.21.15.7:0/1626630311 wait complete. 2024-06-13T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.236+0000 7f0524f39b80 20 client.26132 trim_cache size 0 max 16384 2024-06-13T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 msgr2=0x7f04f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 0x7f04f80a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0504006010 tx=0x7f050401a040 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 msgr2=0x7f051419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 0x7f051419b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f050000e9f0 tx=0x7f050000edb0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 shutdown_connections 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f051410ef40 0x7f051419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04f80a2ca0 0x7f04f80a5160 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0504006010 tx=0x7f050401a040 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f051406a170 0x7f051419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 --2- 172.21.15.7:0/2028992252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f051406aa90 0x7f051419b390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.238+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 >> 172.21.15.7:0/2028992252 conn(0x7f05140782c0 msgr2=0x7f05141105e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.239+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 shutdown_connections 2024-06-13T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:07.239+0000 7f051a600700 1 -- 172.21.15.7:0/2028992252 wait complete. 2024-06-13T07:01:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:07 smithi189 ceph-mon[36616]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 546 op/s 2024-06-13T07:01:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:07 smithi175 ceph-mon[36886]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 546 op/s 2024-06-13T07:01:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 305.0 s, 645.2 tps, lat 158.614 ms stddev 191.843 2024-06-13T07:01:07.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:07 smithi007 ceph-mon[32077]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 12 MiB/s wr, 546 op/s 2024-06-13T07:01:08.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:01:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 -- 172.21.15.7:0/2657005614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb890103c20 msgr2=0x7fb890104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 --2- 172.21.15.7:0/2657005614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb890103c20 0x7fb890104040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb880009900 tx=0x7fb88002a410 comp rx=0 tx=0).stop 2024-06-13T07:01:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 -- 172.21.15.7:0/2657005614 shutdown_connections 2024-06-13T07:01:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 --2- 172.21.15.7:0/2657005614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8901057d0 0x7fb89010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 --2- 172.21.15.7:0/2657005614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890104e10 0x7fb890105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 --2- 172.21.15.7:0/2657005614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb890103c20 0x7fb890104040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 -- 172.21.15.7:0/2657005614 >> 172.21.15.7:0/2657005614 conn(0x7fb8900ff190 msgr2=0x7fb8901015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 -- 172.21.15.7:0/2657005614 shutdown_connections 2024-06-13T07:01:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.247+0000 7fb896e00640 1 -- 172.21.15.7:0/2657005614 wait complete. 2024-06-13T07:01:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.249+0000 7fb896e00640 1 Processor -- start 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 -- start start 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890104e10 0x7fb890079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8901057d0 0x7fb8900762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010f8e0 con 0x7fb890104e10 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010f760 con 0x7fb8901057d0 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb896e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010fa60 con 0x7fb890103c20 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.250+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38652/0 (socket says 172.21.15.7:38652) 2024-06-13T07:01:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 -- 172.21.15.7:0/3796527520 learned_addr learned my addr 172.21.15.7:0/3796527520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 -- 172.21.15.7:0/3796527520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8901057d0 msgr2=0x7fb8900762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8901057d0 0x7fb8900762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 -- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890104e10 msgr2=0x7fb890079c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890104e10 0x7fb890079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.251+0000 7fb88fe00640 1 -- 172.21.15.7:0/3796527520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb890076b80 con 0x7fb890103c20 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb88fe00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb880031440 tx=0x7fb880032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb88cc00640 1 -- 172.21.15.7:0/3796527520 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88003f030 con 0x7fb890103c20 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb88cc00640 1 -- 172.21.15.7:0/3796527520 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880003ce0 con 0x7fb890103c20 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb890199b20 con 0x7fb890103c20 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb88cc00640 1 -- 172.21.15.7:0/3796527520 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880035070 con 0x7fb890103c20 2024-06-13T07:01:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.252+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89019a060 con 0x7fb890103c20 2024-06-13T07:01:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.264+0000 7fb88cc00640 1 -- 172.21.15.7:0/3796527520 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb880042030 con 0x7fb890103c20 2024-06-13T07:01:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.265+0000 7fb899c73740 1 Processor -- start 2024-06-13T07:01:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.267+0000 7fb88cc00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 0x7fb8740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.267+0000 7fb88cc00640 1 -- 172.21.15.7:0/3796527520 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb8800efc10 con 0x7fb890103c20 2024-06-13T07:01:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.272+0000 7fb894e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 0x7fb8740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.273+0000 7fb899c73740 1 -- start start 2024-06-13T07:01:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.275+0000 7fb899c73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb899c73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a45f0 0x56493b2a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb894e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 0x7fb8740a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb888005fd0 tx=0x7fb888009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb899c73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 0x56493b2abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb899c73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56493b207420 con 0x56493b2a3c00 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb899c73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56493b1aec50 con 0x56493b2a45f0 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb899c73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56493b2033e0 con 0x56493b2a9760 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51844/0 (socket says 172.21.15.7:51844) 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb88fe00640 1 -- 172.21.15.7:0/1899769337 learned_addr learned my addr 172.21.15.7:0/1899769337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.276+0000 7fb894e00640 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 0x56493b2abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb88fe00640 1 -- 172.21.15.7:0/1899769337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 msgr2=0x56493b2abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb88fe00640 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 0x56493b2abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb88fe00640 1 -- 172.21.15.7:0/1899769337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a45f0 msgr2=0x56493b2a90f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb88fe00640 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a45f0 0x56493b2a90f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb88fe00640 1 -- 172.21.15.7:0/1899769337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493af1b210 con 0x56493b2a3c00 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.277+0000 7fb894e00640 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 0x56493b2abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.278+0000 7fb88fe00640 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8800b8590 tx=0x7fb88003b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f69f0640abbc9e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.279+0000 7fb885600640 1 -- 172.21.15.7:0/1899769337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800d5070 con 0x56493b2a3c00 2024-06-13T07:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.279+0000 7fb885600640 1 -- 172.21.15.7:0/1899769337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8800bc830 con 0x56493b2a3c00 2024-06-13T07:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.279+0000 7fb885600640 1 -- 172.21.15.7:0/1899769337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800dc0c0 con 0x56493b2a3c00 2024-06-13T07:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.281+0000 7fb899c73740 1 -- 172.21.15.7:0/1899769337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 msgr2=0x56493b2a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.281+0000 7fb899c73740 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8800b8590 tx=0x7fb88003b720 comp rx=0 tx=0).stop 2024-06-13T07:01:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.282+0000 7fb899c73740 1 -- 172.21.15.7:0/1899769337 shutdown_connections 2024-06-13T07:01:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.282+0000 7fb899c73740 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a9760 0x56493b2abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.282+0000 7fb899c73740 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a45f0 0x56493b2a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.282+0000 7fb899c73740 1 --2- 172.21.15.7:0/1899769337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2a4020 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.282+0000 7fb899c73740 1 -- 172.21.15.7:0/1899769337 >> 172.21.15.7:0/1899769337 conn(0x56493b26bfb0 msgr2=0x56493b29f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.283+0000 7fb899c73740 1 -- 172.21.15.7:0/1899769337 shutdown_connections 2024-06-13T07:01:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.286+0000 7fb899c73740 1 -- 172.21.15.7:0/1899769337 wait complete. 2024-06-13T07:01:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.286+0000 7fb899c73740 1 -- start start 2024-06-13T07:01:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.287+0000 7fb899c73740 1 Processor -- start 2024-06-13T07:01:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 0x56493b2b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56493b2ac4d0 con 0x56493b2a9760 2024-06-13T07:01:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56493b1c20b0 con 0x56493b2a3c00 2024-06-13T07:01:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56493b2ac650 con 0x56493b2a45f0 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.288+0000 7fb899c73740 10 client.0 _ll_register_callbacks cb 0x56493b1adf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38674/0 (socket says 172.21.15.7:38674) 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 -- 172.21.15.7:0/863996571 learned_addr learned my addr 172.21.15.7:0/863996571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 -- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 msgr2=0x56493b2ade00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2ade00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb88f400640 1 -- 172.21.15.7:0/863996571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 msgr2=0x56493b2b5310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.289+0000 7fb894e00640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 0x56493b2b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.290+0000 7fb88f400640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 0x56493b2b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.290+0000 7fb88f400640 1 -- 172.21.15.7:0/863996571 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493afe7620 con 0x56493b2a45f0 2024-06-13T07:01:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.290+0000 7fb894e00640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 0x56493b2b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.291+0000 7fb88f400640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00e600 tx=0x7fb87c00eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f87475136ce35a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.291+0000 7fb87a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.291+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c004220 con 0x56493b2a45f0 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.292+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb87c0043c0 con 0x56493b2a45f0 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.292+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8800bc830 con 0x56493b2a45f0 2024-06-13T07:01:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.293+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb87c005020 con 0x56493b2a45f0 2024-06-13T07:01:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.293+0000 7fb899c73740 10 client.26113 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.293+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56493b2b8750 con 0x56493b2a45f0 2024-06-13T07:01:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8800bc830 con 0x56493b2a45f0 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb87c010730 con 0x56493b2a45f0 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 10 client.26113 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 20 client.26113 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2b9e40 0x56493b2bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 4 client.26113 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.294+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56493b214b50 con 0x56493b2b9e40 2024-06-13T07:01:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.296+0000 7fb88fe00640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2b9e40 0x56493b2bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.300+0000 7fb88fe00640 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2b9e40 0x56493b2bc2e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882a765516ea4e server_cookie=8b4705b39a74dacb in_seq=0 out_seq=0 2024-06-13T07:01:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.300+0000 7fb87a000640 10 client.26113 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.301+0000 7fb87a000640 1 -- 172.21.15.7:0/863996571 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56493b214b50 con 0x56493b2b9e40 2024-06-13T07:01:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.301+0000 7fb87a000640 10 client.26113 handle_command_reply: tid=0 2024-06-13T07:01:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.301+0000 7fb899c73740 1 client.26113 shutdown 2024-06-13T07:01:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 msgr2=0x56493b2ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00e600 tx=0x7fb87c00eac0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 shutdown_connections 2024-06-13T07:01:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2b9e40 0x56493b2bc2e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a9760 0x56493b2b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb87a000640 0 client.26113 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b2ae340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 --2- 172.21.15.7:0/863996571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 >> 172.21.15.7:0/863996571 conn(0x56493b26bfb0 msgr2=0x56493b29e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 shutdown_connections 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 1 -- 172.21.15.7:0/863996571 wait complete. 2024-06-13T07:01:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.302+0000 7fb899c73740 20 client.26113 trim_cache size 0 max 16384 2024-06-13T07:01:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.358+0000 7fb899c73740 1 Processor -- start 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 -- start start 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 0x56493b29f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b29f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901bbd00 con 0x56493b26bfb0 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b9570 con 0x56493b2a3c00 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb899c73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b73e0 con 0x56493b2a45f0 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb88f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb88f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36734/0 (socket says 172.21.15.7:36734) 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.359+0000 7fb88f400640 1 -- 172.21.15.7:0/428901117 learned_addr learned my addr 172.21.15.7:0/428901117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.360+0000 7fb894e00640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b29f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.360+0000 7fb88fe00640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 0x56493b29f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.360+0000 7fb88f400640 1 -- 172.21.15.7:0/428901117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 msgr2=0x56493b29f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.360+0000 7fb88f400640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b29f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.360+0000 7fb88f400640 1 -- 172.21.15.7:0/428901117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 msgr2=0x56493b29f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.361+0000 7fb88f400640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 0x56493b29f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.361+0000 7fb88f400640 1 -- 172.21.15.7:0/428901117 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493b2a8d60 con 0x56493b2a3c00 2024-06-13T07:01:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.361+0000 7fb88fe00640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 0x56493b29f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.364+0000 7fb88f400640 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb87c000c00 tx=0x7fb87c009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd753d2902331c80 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.365+0000 7fb884c00640 1 -- 172.21.15.7:0/428901117 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c010730 con 0x56493b2a3c00 2024-06-13T07:01:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.365+0000 7fb884c00640 1 -- 172.21.15.7:0/428901117 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c028040 con 0x56493b2a3c00 2024-06-13T07:01:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.365+0000 7fb884c00640 1 -- 172.21.15.7:0/428901117 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0047a0 con 0x56493b2a3c00 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 -- 172.21.15.7:0/428901117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 msgr2=0x56493b2b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb87c000c00 tx=0x7fb87c009d20 comp rx=0 tx=0).stop 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 -- 172.21.15.7:0/428901117 shutdown_connections 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b29f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b2a3c00 0x56493b2b52d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 --2- 172.21.15.7:0/428901117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b26bfb0 0x56493b29f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 -- 172.21.15.7:0/428901117 >> 172.21.15.7:0/428901117 conn(0x56493b2accb0 msgr2=0x56493b2a5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 -- 172.21.15.7:0/428901117 shutdown_connections 2024-06-13T07:01:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.368+0000 7fb899c73740 1 -- 172.21.15.7:0/428901117 wait complete. 2024-06-13T07:01:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.369+0000 7fb899c73740 1 -- start start 2024-06-13T07:01:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.369+0000 7fb899c73740 1 Processor -- start 2024-06-13T07:01:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.370+0000 7fb899c73740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.370+0000 7fb899c73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb899c73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b322bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb899c73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b7a30 con 0x56493b2a3c00 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb899c73740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b8ea0 con 0x56493b26bfb0 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb899c73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b8100 con 0x56493b2a45f0 2024-06-13T07:01:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb899c73740 10 client.0 _ll_register_callbacks cb 0x7fb87c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb88fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:36746/0 (socket says 172.21.15.7:36746) 2024-06-13T07:01:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.371+0000 7fb88fe00640 1 -- 172.21.15.7:0/2554182694 learned_addr learned my addr 172.21.15.7:0/2554182694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.372+0000 7fb88f400640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.372+0000 7fb88fe00640 1 -- 172.21.15.7:0/2554182694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 msgr2=0x56493b322bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.372+0000 7fb88fe00640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b322bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.372+0000 7fb88fe00640 1 -- 172.21.15.7:0/2554182694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 msgr2=0x56493b2b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.372+0000 7fb88fe00640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.373+0000 7fb88fe00640 1 -- 172.21.15.7:0/2554182694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493b3230f0 con 0x56493b26bfb0 2024-06-13T07:01:09.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.373+0000 7fb88f400640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.373+0000 7fb88fe00640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb880032fd0 tx=0x7fb880032e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf53c3e12b8ca7e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.373+0000 7fb87a000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.374+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800d5040 con 0x56493b26bfb0 2024-06-13T07:01:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.375+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb88003b460 con 0x56493b26bfb0 2024-06-13T07:01:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.375+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87c028040 con 0x56493b26bfb0 2024-06-13T07:01:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.376+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb8800c4da0 con 0x56493b26bfb0 2024-06-13T07:01:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.379+0000 7fb899c73740 10 client.26141 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.379+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56493b3242c0 con 0x56493b26bfb0 2024-06-13T07:01:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.380+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c028040 con 0x56493b26bfb0 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb8800b87a0 con 0x56493b26bfb0 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 10 client.26141 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 20 client.26141 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2a9760 0x56493b326a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 4 client.26141 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.381+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56493b20eea0 con 0x56493b2a9760 2024-06-13T07:01:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.383+0000 7fb88f400640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2a9760 0x56493b326a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.384+0000 7fb88f400640 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2a9760 0x56493b326a00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b37a854b973211 server_cookie=fd618bcd28c067f2 in_seq=0 out_seq=0 2024-06-13T07:01:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.384+0000 7fb87a000640 10 client.26141 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.513+0000 7fb87a000640 1 -- 172.21.15.7:0/2554182694 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56493b20eea0 con 0x56493b2a9760 2024-06-13T07:01:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.513+0000 7fb87a000640 10 client.26141 handle_command_reply: tid=0 2024-06-13T07:01:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.513+0000 7fb899c73740 1 client.26141 shutdown 2024-06-13T07:01:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 msgr2=0x56493b2b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb880032fd0 tx=0x7fb880032e50 comp rx=0 tx=0).stop 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 shutdown_connections 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56493b2a9760 0x56493b326a00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56493b2a45f0 0x56493b322bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56493b2a3c00 0x56493b2b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 --2- 172.21.15.7:0/2554182694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56493b26bfb0 0x56493b2b2180 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 >> 172.21.15.7:0/2554182694 conn(0x56493b2accb0 msgr2=0x56493b29f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.514+0000 7fb87a000640 0 client.26141 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.515+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 shutdown_connections 2024-06-13T07:01:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.515+0000 7fb899c73740 1 -- 172.21.15.7:0/2554182694 wait complete. 2024-06-13T07:01:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.516+0000 7fb899c73740 20 client.26141 trim_cache size 0 max 16384 2024-06-13T07:01:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:01:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 msgr2=0x7fb8740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 0x7fb8740a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb888005fd0 tx=0x7fb888009710 comp rx=0 tx=0).stop 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 msgr2=0x7fb890079750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb880031440 tx=0x7fb880032ce0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 shutdown_connections 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb8740a2cb0 0x7fb8740a5170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8901057d0 0x7fb8900762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb890104e10 0x7fb890079c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 --2- 172.21.15.7:0/3796527520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb890103c20 0x7fb890079750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 >> 172.21.15.7:0/3796527520 conn(0x7fb8900ff190 msgr2=0x7fb8901080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.517+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 shutdown_connections 2024-06-13T07:01:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:09.518+0000 7fb896e00640 1 -- 172.21.15.7:0/3796527520 wait complete. 2024-06-13T07:01:09.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:09 smithi007 ceph-mon[32077]: pgmap v662: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 698 op/s 2024-06-13T07:01:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:09 smithi189 ceph-mon[36616]: pgmap v662: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 698 op/s 2024-06-13T07:01:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:09 smithi175 ceph-mon[36886]: pgmap v662: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 698 op/s 2024-06-13T07:01:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 -- 172.21.15.7:0/3904016653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4102740 msgr2=0x7feae4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 --2- 172.21.15.7:0/3904016653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4102740 0x7feae4102b20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fead4009920 tx=0x7fead4009c30 comp rx=0 tx=0).stop 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 -- 172.21.15.7:0/3904016653 shutdown_connections 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 --2- 172.21.15.7:0/3904016653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae4107840 0x7feae410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 --2- 172.21.15.7:0/3904016653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4102740 0x7feae4102b20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 --2- 172.21.15.7:0/3904016653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae41030f0 0x7feae4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.631+0000 7feaeb200700 1 -- 172.21.15.7:0/3904016653 >> 172.21.15.7:0/3904016653 conn(0x7feae40795f0 msgr2=0x7feae4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.632+0000 7feaeb200700 1 -- 172.21.15.7:0/3904016653 shutdown_connections 2024-06-13T07:01:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.632+0000 7feaeb200700 1 -- 172.21.15.7:0/3904016653 wait complete. 2024-06-13T07:01:11.632 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:11 smithi007 ceph-mon[32077]: pgmap v663: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 484 op/s 2024-06-13T07:01:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.633+0000 7feaeb200700 1 Processor -- start 2024-06-13T07:01:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 -- start start 2024-06-13T07:01:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 0x7feae419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 0x7feae419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feae41127b0 con 0x7feae4107840 2024-06-13T07:01:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112630 con 0x7feae41030f0 2024-06-13T07:01:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.634+0000 7feaeb200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feae4112930 con 0x7feae4102740 2024-06-13T07:01:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.635+0000 7feae3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.635+0000 7feae3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 0x7feae419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.635+0000 7feae3e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43840/0 (socket says 172.21.15.7:43840) 2024-06-13T07:01:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.635+0000 7feae3e00700 1 -- 172.21.15.7:0/3555380331 learned_addr learned my addr 172.21.15.7:0/3555380331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.635+0000 7feae9200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 0x7feae419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 -- 172.21.15.7:0/3555380331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 msgr2=0x7feae419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 0x7feae419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 -- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 msgr2=0x7feae419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 0x7feae419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 -- 172.21.15.7:0/3555380331 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae419fe40 con 0x7feae4102740 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae9200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 0x7feae419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3400700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 0x7feae419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.636+0000 7feae3e00700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fead4000c00 tx=0x7fead400fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.637+0000 7feae0c00700 1 -- 172.21.15.7:0/3555380331 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4027070 con 0x7feae4102740 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.637+0000 7feae0c00700 1 -- 172.21.15.7:0/3555380331 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead400b560 con 0x7feae4102740 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.637+0000 7feae0c00700 1 -- 172.21.15.7:0/3555380331 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40049e0 con 0x7feae4102740 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.637+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41a00d0 con 0x7feae4102740 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.637+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41a79b0 con 0x7feae4102740 2024-06-13T07:01:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.640+0000 7feae0c00700 1 -- 172.21.15.7:0/3555380331 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fead402a070 con 0x7feae4102740 2024-06-13T07:01:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.641+0000 7feae0c00700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 0x7feac80a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.642+0000 7feae0c00700 1 -- 172.21.15.7:0/3555380331 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fead40cff30 con 0x7feae4102740 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.642+0000 7feaf5b50b80 1 Processor -- start 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feae3400700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 0x7feac80a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 -- start start 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4d7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55988d3dda10 con 0x55988d4dd3a0 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55988d420b00 con 0x55988d4ddcc0 2024-06-13T07:01:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.643+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55988d3db8f0 con 0x55988d4d7a50 2024-06-13T07:01:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.644+0000 7feae9200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.644+0000 7feae9200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43854/0 (socket says 172.21.15.7:43854) 2024-06-13T07:01:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.644+0000 7feae9200700 1 -- 172.21.15.7:0/1117810018 learned_addr learned my addr 172.21.15.7:0/1117810018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.645+0000 7feae3400700 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4d7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.646+0000 7feae3400700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 0x7feac80a51e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feae410f770 tx=0x7fead0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae9200700 1 -- 172.21.15.7:0/1117810018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 msgr2=0x55988d4d7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae9200700 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4d7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae9200700 1 -- 172.21.15.7:0/1117810018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 msgr2=0x55988d4dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae9200700 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae9200700 1 -- 172.21.15.7:0/1117810018 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d3c8f90 con 0x55988d4d7a50 2024-06-13T07:01:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.648+0000 7feae3400700 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4d7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.652+0000 7feae9200700 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feadc009440 tx=0x7feadc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a91a76e7ec81d1b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.653+0000 7fead9600700 1 -- 172.21.15.7:0/1117810018 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc013750 con 0x55988d4d7a50 2024-06-13T07:01:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.653+0000 7fead9600700 1 -- 172.21.15.7:0/1117810018 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc013dd0 con 0x55988d4d7a50 2024-06-13T07:01:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.653+0000 7fead9600700 1 -- 172.21.15.7:0/1117810018 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc014400 con 0x55988d4d7a50 2024-06-13T07:01:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.654+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1117810018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 msgr2=0x55988d4d7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.654+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feadc009440 tx=0x7feadc009920 comp rx=0 tx=0).stop 2024-06-13T07:01:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1117810018 shutdown_connections 2024-06-13T07:01:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4d7e90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1117810018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4d7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1117810018 >> 172.21.15.7:0/1117810018 conn(0x55988d3fb170 msgr2=0x55988d4c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1117810018 shutdown_connections 2024-06-13T07:01:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.656+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1117810018 wait complete. 2024-06-13T07:01:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.657+0000 7feaf5b50b80 1 -- start start 2024-06-13T07:01:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.657+0000 7feaf5b50b80 1 Processor -- start 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4ddcc0 0x55988d4e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55988d3bcf60 con 0x55988d4ddcc0 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55988d41cac0 con 0x55988d4dd3a0 2024-06-13T07:01:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55988d4e46a0 con 0x55988d4d7a50 2024-06-13T07:01:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.658+0000 7feaf5b50b80 10 client.0 _ll_register_callbacks cb 0x55988d3b6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37518/0 (socket says 172.21.15.7:37518) 2024-06-13T07:01:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 -- 172.21.15.7:0/580022367 learned_addr learned my addr 172.21.15.7:0/580022367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 -- 172.21.15.7:0/580022367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 msgr2=0x55988d4e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 -- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4ddcc0 msgr2=0x55988d4e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4ddcc0 0x55988d4e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.659+0000 7feae3400700 1 -- 172.21.15.7:0/580022367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988d36f8d0 con 0x55988d4dd3a0 2024-06-13T07:01:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.660+0000 7feae3400700 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fead0010f80 tx=0x7fead0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59089b45242a7a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.660+0000 7feace000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.660+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0004230 con 0x55988d4dd3a0 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.660+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fead00043d0 con 0x55988d4dd3a0 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.661+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc013dd0 con 0x55988d4dd3a0 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.661+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fead00036a0 con 0x55988d4dd3a0 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.661+0000 7feaf5b50b80 10 client.26150 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.661+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55988d4eef60 con 0x55988d4dd3a0 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.661+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc013dd0 con 0x55988d4dd3a0 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.662+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fead000f040 con 0x55988d4dd3a0 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.662+0000 7feaf5b50b80 10 client.26150 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.662+0000 7feaf5b50b80 10 client.26150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.662+0000 7feaf5b50b80 20 client.26150 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.662+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4f1980 0x55988d4f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.663+0000 7feaf5b50b80 4 client.26150 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.663+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988d42e000 con 0x55988d4f1980 2024-06-13T07:01:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.663+0000 7feae3e00700 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4f1980 0x55988d4f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.664+0000 7feae3e00700 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4f1980 0x55988d4f3e00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9efe05bb725f0954 server_cookie=6b269fa33cdc4445 in_seq=0 out_seq=0 2024-06-13T07:01:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.664+0000 7feace000700 10 client.26150 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:01:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.666+0000 7feace000700 1 -- 172.21.15.7:0/580022367 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55988d42e000 con 0x55988d4f1980 2024-06-13T07:01:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.666+0000 7feace000700 10 client.26150 handle_command_reply: tid=0 2024-06-13T07:01:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.667+0000 7feaf5b50b80 1 client.26150 shutdown 2024-06-13T07:01:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 msgr2=0x55988d4e0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fead0010f80 tx=0x7fead0004750 comp rx=0 tx=0).stop 2024-06-13T07:01:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 shutdown_connections 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4ddcc0 0x55988d4e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4f1980 0x55988d4f3e00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4d7a50 0x55988d4e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/580022367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4dd3a0 0x55988d4e0e00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 >> 172.21.15.7:0/580022367 conn(0x55988d3fb170 msgr2=0x55988d4c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feace000700 0 client.26150 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:01:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.668+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 shutdown_connections 2024-06-13T07:01:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.669+0000 7feaf5b50b80 1 -- 172.21.15.7:0/580022367 wait complete. 2024-06-13T07:01:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.669+0000 7feaf5b50b80 20 client.26150 trim_cache size 0 max 16384 2024-06-13T07:01:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.706+0000 7feaf5b50b80 1 Processor -- start 2024-06-13T07:01:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 -- start start 2024-06-13T07:01:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4d7a50 0x55988d4c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ae900 con 0x55988d4dd3a0 2024-06-13T07:01:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feae41ac130 con 0x55988d4d7a50 2024-06-13T07:01:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.707+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feae41a9fb0 con 0x55988d4ddcc0 2024-06-13T07:01:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43884/0 (socket says 172.21.15.7:43884) 2024-06-13T07:01:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 -- 172.21.15.7:0/3644092729 learned_addr learned my addr 172.21.15.7:0/3644092729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae3e00700 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4d7a50 0x55988d4c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 -- 172.21.15.7:0/3644092729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4d7a50 msgr2=0x55988d4c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4d7a50 0x55988d4c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 -- 172.21.15.7:0/3644092729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 msgr2=0x55988d4e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4e3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.708+0000 7feae9200700 1 -- 172.21.15.7:0/3644092729 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d4df370 con 0x55988d4ddcc0 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.709+0000 7feae9200700 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feadc004c00 tx=0x7feadc004c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6480b8a7ebb4c640 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.709+0000 7fead8c00700 1 -- 172.21.15.7:0/3644092729 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc023070 con 0x55988d4ddcc0 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.709+0000 7fead8c00700 1 -- 172.21.15.7:0/3644092729 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc004e50 con 0x55988d4ddcc0 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.709+0000 7fead8c00700 1 -- 172.21.15.7:0/3644092729 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc015990 con 0x55988d4ddcc0 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 -- 172.21.15.7:0/3644092729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 msgr2=0x55988d4c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feadc004c00 tx=0x7feadc004c30 comp rx=0 tx=0).stop 2024-06-13T07:01:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 -- 172.21.15.7:0/3644092729 shutdown_connections 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4ddcc0 0x55988d4c9a90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4d7a50 0x55988d4c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/3644092729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4dd3a0 0x55988d4e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.710+0000 7feaf5b50b80 1 -- 172.21.15.7:0/3644092729 >> 172.21.15.7:0/3644092729 conn(0x55988d3fb170 msgr2=0x55988d4df6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.711+0000 7feaf5b50b80 1 -- 172.21.15.7:0/3644092729 shutdown_connections 2024-06-13T07:01:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.711+0000 7feaf5b50b80 1 -- 172.21.15.7:0/3644092729 wait complete. 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.713+0000 7feaf5b50b80 1 -- start start 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.713+0000 7feaf5b50b80 1 Processor -- start 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 0x55988d4f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 0x55988d4f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4f6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feae41aa5f0 con 0x55988d4d7a50 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feae41aba60 con 0x55988d4ddcc0 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feae41aacc0 con 0x55988d4dd3a0 2024-06-13T07:01:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.714+0000 7feaf5b50b80 10 client.0 _ll_register_callbacks cb 0x55988d3b6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 0x55988d4f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae9200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae9200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37538/0 (socket says 172.21.15.7:37538) 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae9200700 1 -- 172.21.15.7:0/1202119109 learned_addr learned my addr 172.21.15.7:0/1202119109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 0x55988d4f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48528/0 (socket says 172.21.15.7:48528) 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.715+0000 7feae3400700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 0x55988d4f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.716+0000 7feae3400700 1 -- 172.21.15.7:0/1202119109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 msgr2=0x55988d4f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.716+0000 7feae3400700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.716+0000 7feae3400700 1 -- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 msgr2=0x55988d4f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.716+0000 7feae3400700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 0x55988d4f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.716+0000 7feae3400700 1 -- 172.21.15.7:0/1202119109 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988d4f6fe0 con 0x55988d4dd3a0 2024-06-13T07:01:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.718+0000 7feae3400700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 0x55988d4f63f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fead0000c00 tx=0x7fead0010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f12c55cf66e022f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.718+0000 7feace000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.719+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0036040 con 0x55988d4dd3a0 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.719+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fead001fd20 con 0x55988d4dd3a0 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.719+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc004e50 con 0x55988d4dd3a0 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.720+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fead0048050 con 0x55988d4dd3a0 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.720+0000 7feaf5b50b80 10 client.26131 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.720+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55988d4cef50 con 0x55988d4dd3a0 2024-06-13T07:01:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc004e50 con 0x55988d4dd3a0 2024-06-13T07:01:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fead0039070 con 0x55988d4dd3a0 2024-06-13T07:01:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 10 client.26131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:01:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 20 client.26131 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4cf630 0x55988d4d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 4 client.26131 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.721+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55988d428580 con 0x55988d4cf630 2024-06-13T07:01:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.724+0000 7feae3e00700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4cf630 0x55988d4d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.728+0000 7feace000700 10 client.26131 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:01:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.728+0000 7feae3e00700 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4cf630 0x55988d4d1a50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64dadd1326665030 server_cookie=196a7734d3c6f00b in_seq=0 out_seq=0 2024-06-13T07:01:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.729+0000 7feace000700 1 -- 172.21.15.7:0/1202119109 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55988d428580 con 0x55988d4cf630 2024-06-13T07:01:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.729+0000 7feace000700 10 client.26131 handle_command_reply: tid=0 2024-06-13T07:01:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.729+0000 7feaf5b50b80 1 client.26131 shutdown 2024-06-13T07:01:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 msgr2=0x55988d4f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 0x55988d4f63f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fead0000c00 tx=0x7fead0010790 comp rx=0 tx=0).stop 2024-06-13T07:01:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 shutdown_connections 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55988d4ddcc0 0x55988d4f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55988d4d7a50 0x55988d4f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55988d4cf630 0x55988d4d1a50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 --2- 172.21.15.7:0/1202119109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55988d4dd3a0 0x55988d4f63f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feace000700 0 client.26131 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:01:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 >> 172.21.15.7:0/1202119109 conn(0x55988d3fb170 msgr2=0x55988d4c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.734 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:01:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.732+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 shutdown_connections 2024-06-13T07:01:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.733+0000 7feaf5b50b80 1 -- 172.21.15.7:0/1202119109 wait complete. 2024-06-13T07:01:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.733+0000 7feaf5b50b80 20 client.26131 trim_cache size 0 max 16384 2024-06-13T07:01:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 msgr2=0x7feac80a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 0x7feac80a51e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feae410f770 tx=0x7fead0009310 comp rx=0 tx=0).stop 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 msgr2=0x7feae419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fead4000c00 tx=0x7fead400fb20 comp rx=0 tx=0).stop 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 shutdown_connections 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feae4107840 0x7feae419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae4102740 0x7feae419ae30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7feac80a2d20 0x7feac80a51e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 --2- 172.21.15.7:0/3555380331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feae41030f0 0x7feae419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.735+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 >> 172.21.15.7:0/3555380331 conn(0x7feae40795f0 msgr2=0x7feae4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.736+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 shutdown_connections 2024-06-13T07:01:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:11.736+0000 7feaeb200700 1 -- 172.21.15.7:0/3555380331 wait complete. 2024-06-13T07:01:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:11 smithi189 ceph-mon[36616]: pgmap v663: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 484 op/s 2024-06-13T07:01:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:11 smithi175 ceph-mon[36886]: pgmap v663: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 484 op/s 2024-06-13T07:01:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 310.0 s, 775.6 tps, lat 129.431 ms stddev 172.472 2024-06-13T07:01:13.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:01:13.723 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:13 smithi007 ceph-mon[32077]: pgmap v664: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 622 op/s 2024-06-13T07:01:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:13 smithi189 ceph-mon[36616]: pgmap v664: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 622 op/s 2024-06-13T07:01:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:13 smithi175 ceph-mon[36886]: pgmap v664: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 622 op/s 2024-06-13T07:01:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:15.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:15 smithi007 ceph-mon[32077]: pgmap v665: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-06-13T07:01:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:15 smithi189 ceph-mon[36616]: pgmap v665: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-06-13T07:01:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:15 smithi175 ceph-mon[36886]: pgmap v665: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-06-13T07:01:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.789+0000 7f6491200700 1 -- 172.21.15.7:0/3397880270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101520 msgr2=0x7f648c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.789+0000 7f6491200700 1 --2- 172.21.15.7:0/3397880270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101520 0x7f648c101900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f647c00b0d0 tx=0x7f647c00b3e0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.790+0000 7f6491200700 1 -- 172.21.15.7:0/3397880270 shutdown_connections 2024-06-13T07:01:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.790+0000 7f6491200700 1 --2- 172.21.15.7:0/3397880270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c10e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.791+0000 7f6491200700 1 --2- 172.21.15.7:0/3397880270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101520 0x7f648c101900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.791+0000 7f6491200700 1 --2- 172.21.15.7:0/3397880270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101ed0 0x7f648c105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.791+0000 7f6491200700 1 -- 172.21.15.7:0/3397880270 >> 172.21.15.7:0/3397880270 conn(0x7f648c0fcdb0 msgr2=0x7f648c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.791+0000 7f6491200700 1 -- 172.21.15.7:0/3397880270 shutdown_connections 2024-06-13T07:01:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.792+0000 7f6491200700 1 -- 172.21.15.7:0/3397880270 wait complete. 2024-06-13T07:01:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.793+0000 7f6491200700 1 Processor -- start 2024-06-13T07:01:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 -- start start 2024-06-13T07:01:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 0x7f648c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101ed0 0x7f648c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1105a0 con 0x7f648c101520 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f648c110420 con 0x7f648c106600 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6491200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f648c110720 con 0x7f648c101ed0 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f648b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 0x7f648c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f648b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 0x7f648c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48546/0 (socket says 172.21.15.7:48546) 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f648b400700 1 -- 172.21.15.7:0/3993066885 learned_addr learned my addr 172.21.15.7:0/3993066885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.794+0000 7f6482a00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101ed0 0x7f648c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 -- 172.21.15.7:0/3993066885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101ed0 msgr2=0x7f648c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101ed0 0x7f648c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 -- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 msgr2=0x7f648c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 0x7f648c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 -- 172.21.15.7:0/3993066885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19fde0 con 0x7f648c106600 2024-06-13T07:01:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.795+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c19f6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f648400a820 tx=0x7f648400abe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.796+0000 7f6488c00700 1 -- 172.21.15.7:0/3993066885 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484004830 con 0x7f648c106600 2024-06-13T07:01:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.796+0000 7f6488c00700 1 -- 172.21.15.7:0/3993066885 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6484005020 con 0x7f648c106600 2024-06-13T07:01:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.796+0000 7f6488c00700 1 -- 172.21.15.7:0/3993066885 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484007b60 con 0x7f648c106600 2024-06-13T07:01:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.796+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c1a00d0 con 0x7f648c106600 2024-06-13T07:01:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.796+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1a7950 con 0x7f648c106600 2024-06-13T07:01:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.802+0000 7f6488c00700 1 -- 172.21.15.7:0/3993066885 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f648401b020 con 0x7f648c106600 2024-06-13T07:01:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.802+0000 7f649d1c3b80 1 Processor -- start 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.803+0000 7f649d1c3b80 1 -- start start 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ace3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56507abf2a10 con 0x56507ace2d60 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56507ac35b00 con 0x56507ace3680 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56507abf08f0 con 0x56507acff540 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f648b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.804+0000 7f6482a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ace3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37564/0 (socket says 172.21.15.7:37564) 2024-06-13T07:01:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 -- 172.21.15.7:0/3554547999 learned_addr learned my addr 172.21.15.7:0/3554547999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f6482a00700 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 -- 172.21.15.7:0/3554547999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 msgr2=0x56507ad01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f6488c00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 0x7f64700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 -- 172.21.15.7:0/3554547999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 msgr2=0x56507ace3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.805+0000 7f648b400700 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ace3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.806+0000 7f6488c00700 1 -- 172.21.15.7:0/3993066885 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f64840ce2f0 con 0x7f648c106600 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.806+0000 7f648b400700 1 -- 172.21.15.7:0/3554547999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507abddf90 con 0x56507ace3680 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.806+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 0x7f64700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.807+0000 7f6482a00700 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.807+0000 7f648be00700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 0x7f64700a5160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6484096d10 tx=0x7f6484005710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.808+0000 7f648b400700 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f647c009180 tx=0x7f647c0091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155b6efef63aea45 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.808+0000 7f6480c00700 1 -- 172.21.15.7:0/3554547999 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c042070 con 0x56507ace3680 2024-06-13T07:01:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.809+0000 7f6480c00700 1 -- 172.21.15.7:0/3554547999 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c009480 con 0x56507ace3680 2024-06-13T07:01:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.809+0000 7f6480c00700 1 -- 172.21.15.7:0/3554547999 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c03e6b0 con 0x56507ace3680 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.809+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3554547999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 msgr2=0x56507acff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.809+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f647c009180 tx=0x7f647c0091b0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3554547999 shutdown_connections 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ace3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3554547999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acff000 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3554547999 >> 172.21.15.7:0/3554547999 conn(0x56507ac10170 msgr2=0x56507acdf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3554547999 shutdown_connections 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.810+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3554547999 wait complete. 2024-06-13T07:01:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.811+0000 7f649d1c3b80 1 -- start start 2024-06-13T07:01:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.811+0000 7f649d1c3b80 1 Processor -- start 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507acfefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507acf68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56507abd1f60 con 0x56507ace2d60 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56507ac31ac0 con 0x56507ace3680 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56507ace6090 con 0x56507acff540 2024-06-13T07:01:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.812+0000 7f649d1c3b80 10 client.0 _ll_register_callbacks cb 0x56507abcb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f6482a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507acf68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f6482a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507acf68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43920/0 (socket says 172.21.15.7:43920) 2024-06-13T07:01:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f6482a00700 1 -- 172.21.15.7:0/1568623220 learned_addr learned my addr 172.21.15.7:0/1568623220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f6482a00700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507acfefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 -- 172.21.15.7:0/1568623220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 msgr2=0x56507acf68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507acf68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 -- 172.21.15.7:0/1568623220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 msgr2=0x56507acfefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507acfefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.813+0000 7f648b400700 1 -- 172.21.15.7:0/1568623220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507ab848d0 con 0x56507ace3680 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f648b400700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acf61c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f647c00b420 tx=0x7f647c007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9e6fa958f26fb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f6479600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:16.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c042070 con 0x56507ace3680 2024-06-13T07:01:16.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f647c0076b0 con 0x56507ace3680 2024-06-13T07:01:16.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56507ac43000 con 0x56507ace3680 2024-06-13T07:01:16.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.814+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f647c056050 con 0x56507ace3680 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 10 client.26165 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56507ad03f20 con 0x56507ace3680 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c0078b0 con 0x56507ace3680 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f647c011070 con 0x56507ace3680 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 20 client.26165 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad063f0 0x56507ad087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 4 client.26165 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.815+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56507ac43000 con 0x56507ad063f0 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.816+0000 7f6482a00700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad063f0 0x56507ad087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.816+0000 7f6482a00700 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad063f0 0x56507ad087e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42651334ea3eb6e server_cookie=64eb0a55f23cae9f in_seq=0 out_seq=0 2024-06-13T07:01:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.816+0000 7f6479600700 10 client.26165 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:16.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f6479600700 1 -- 172.21.15.7:0/1568623220 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56507ac43000 con 0x56507ad063f0 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f6479600700 10 client.26165 handle_command_reply: tid=0 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 client.26165 shutdown 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 msgr2=0x56507acf61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acf61c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f647c00b420 tx=0x7f647c007f40 comp rx=0 tx=0).stop 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 shutdown_connections 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad063f0 0x56507ad087e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f6479600700 0 client.26165 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.817+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507acf68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507acfefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/1568623220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acf61c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 >> 172.21.15.7:0/1568623220 conn(0x56507ac10170 msgr2=0x56507acdee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 shutdown_connections 2024-06-13T07:01:16.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 1 -- 172.21.15.7:0/1568623220 wait complete. 2024-06-13T07:01:16.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.818+0000 7f649d1c3b80 20 client.26165 trim_cache size 0 max 16384 2024-06-13T07:01:16.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 Processor -- start 2024-06-13T07:01:16.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 -- start start 2024-06-13T07:01:16.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507ace2d60 0x56507acfaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 0x56507acfeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f648c0729e0 con 0x56507acff540 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f648c070210 con 0x56507ace3680 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.846+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f648c10b240 con 0x56507ace2d60 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f648b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f6482a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507ace2d60 0x56507acfaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f648b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37580/0 (socket says 172.21.15.7:37580) 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f648b400700 1 -- 172.21.15.7:0/3153521312 learned_addr learned my addr 172.21.15.7:0/3153521312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:16.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f6482a00700 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 0x56507acfeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.847+0000 7f648b400700 1 -- 172.21.15.7:0/3153521312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507ace2d60 msgr2=0x56507acfaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f648b400700 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507ace2d60 0x56507acfaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f648b400700 1 -- 172.21.15.7:0/3153521312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 msgr2=0x56507acfeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f648b400700 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 0x56507acfeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f648b400700 1 -- 172.21.15.7:0/3153521312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507acdfd50 con 0x56507ace3680 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f6482a00700 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 0x56507acfeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f648b400700 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f647c006280 tx=0x7f647c00b060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3399cae49d86191 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f647be00700 1 -- 172.21.15.7:0/3153521312 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c042040 con 0x56507ace3680 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f647be00700 1 -- 172.21.15.7:0/3153521312 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c003e50 con 0x56507ace3680 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.848+0000 7f647be00700 1 -- 172.21.15.7:0/3153521312 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c0053b0 con 0x56507ace3680 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3153521312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 msgr2=0x56507acdf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f647c006280 tx=0x7f647c00b060 comp rx=0 tx=0).stop 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3153521312 shutdown_connections 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507acff540 0x56507acfeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507ace2d60 0x56507acfaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/3153521312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507acdf390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3153521312 >> 172.21.15.7:0/3153521312 conn(0x56507ac10170 msgr2=0x56507ace5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3153521312 shutdown_connections 2024-06-13T07:01:16.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.849+0000 7f649d1c3b80 1 -- 172.21.15.7:0/3153521312 wait complete. 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 -- start start 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 Processor -- start 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507ad090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f648c06e6d0 con 0x56507ace2d60 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f648c06fb40 con 0x56507ace3680 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f648c06eda0 con 0x56507acff540 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.850+0000 7f649d1c3b80 10 client.0 _ll_register_callbacks cb 0x56507abcb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507ad090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f6482a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f6482a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48602/0 (socket says 172.21.15.7:48602) 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f6482a00700 1 -- 172.21.15.7:0/4156239942 learned_addr learned my addr 172.21.15.7:0/4156239942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f6482a00700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 -- 172.21.15.7:0/4156239942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 msgr2=0x56507ad095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 -- 172.21.15.7:0/4156239942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 msgr2=0x56507ad0bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.851+0000 7f648b400700 1 -- 172.21.15.7:0/4156239942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507ad09ba0 con 0x56507ace3680 2024-06-13T07:01:16.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f6482a00700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f648b400700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507ad090a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x56507acfb500 tx=0x7f647c03e450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ed5553e945507c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f6479600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c042040 con 0x56507ace3680 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f647c005c60 con 0x56507ace3680 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.852+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647c003e50 con 0x56507ace3680 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.853+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f647c03eb10 con 0x56507ace3680 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.853+0000 7f649d1c3b80 10 client.26171 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.853+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56507ad70a70 con 0x56507ace3680 2024-06-13T07:01:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.853+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c003e50 con 0x56507ace3680 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.853+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f647c011070 con 0x56507ace3680 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 20 client.26171 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad70ed0 0x56507ad732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 4 client.26171 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56507ac3d580 con 0x56507ad70ed0 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f6482a00700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad70ed0 0x56507ad732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.854+0000 7f6482a00700 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad70ed0 0x56507ad732c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc56f15f0ea8c48d server_cookie=b67514932762357c in_seq=0 out_seq=0 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f6479600700 10 client.26171 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f6479600700 1 -- 172.21.15.7:0/4156239942 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56507ac3d580 con 0x56507ad70ed0 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f6479600700 10 client.26171 handle_command_reply: tid=0 2024-06-13T07:01:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 client.26171 shutdown 2024-06-13T07:01:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 msgr2=0x56507ad090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507ad090a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x56507acfb500 tx=0x7f647c03e450 comp rx=0 tx=0).stop 2024-06-13T07:01:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 shutdown_connections 2024-06-13T07:01:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56507ad70ed0 0x56507ad732c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56507acff540 0x56507ad095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56507ace2d60 0x56507ad0bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 --2- 172.21.15.7:0/4156239942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56507ace3680 0x56507ad090a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 >> 172.21.15.7:0/4156239942 conn(0x56507ac10170 msgr2=0x56507ace4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.855+0000 7f6479600700 0 client.26171 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.856+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 shutdown_connections 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.856+0000 7f649d1c3b80 1 -- 172.21.15.7:0/4156239942 wait complete. 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.856+0000 7f649d1c3b80 20 client.26171 trim_cache size 0 max 16384 2024-06-13T07:01:16.857 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 msgr2=0x7f64700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 0x7f64700a5160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6484096d10 tx=0x7f6484005710 comp rx=0 tx=0).stop 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 msgr2=0x7f648c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c19f6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f648400a820 tx=0x7f648400abe0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 shutdown_connections 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f64700a2ca0 0x7f64700a5160 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f648c106600 0x7f648c19f6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c101520 0x7f648c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 --2- 172.21.15.7:0/3993066885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f648c101ed0 0x7f648c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 >> 172.21.15.7:0/3993066885 conn(0x7f648c0fcdb0 msgr2=0x7f648c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 shutdown_connections 2024-06-13T07:01:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:16.857+0000 7f6491200700 1 -- 172.21.15.7:0/3993066885 wait complete. 2024-06-13T07:01:17.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:17 smithi189 ceph-mon[36616]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 538 op/s 2024-06-13T07:01:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:17 smithi175 ceph-mon[36886]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 538 op/s 2024-06-13T07:01:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 315.0 s, 609.6 tps, lat 161.168 ms stddev 221.301 2024-06-13T07:01:17.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:17 smithi007 ceph-mon[32077]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 538 op/s 2024-06-13T07:01:19.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:19 smithi007 ceph-mon[32077]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-06-13T07:01:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:19 smithi189 ceph-mon[36616]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-06-13T07:01:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:19 smithi175 ceph-mon[36886]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-06-13T07:01:20.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:01:20.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:01:21.421 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:21 smithi007 ceph-mon[32077]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 518 op/s 2024-06-13T07:01:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:21 smithi189 ceph-mon[36616]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 518 op/s 2024-06-13T07:01:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:21 smithi175 ceph-mon[36886]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 518 op/s 2024-06-13T07:01:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.737+0000 7f69df200640 1 -- 172.21.15.7:0/934189997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e0102a00 msgr2=0x7f69e0104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.737+0000 7f69df200640 1 --2- 172.21.15.7:0/934189997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e0102a00 0x7f69e0104e70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009860 tx=0x7f69cc02a340 comp rx=0 tx=0).stop 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.739+0000 7f69df200640 1 -- 172.21.15.7:0/934189997 shutdown_connections 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.739+0000 7f69df200640 1 --2- 172.21.15.7:0/934189997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e01053b0 0x7f69e0105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.739+0000 7f69df200640 1 --2- 172.21.15.7:0/934189997 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e0102a00 0x7f69e0104e70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.739+0000 7f69df200640 1 --2- 172.21.15.7:0/934189997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0100090 0x7f69e01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.739+0000 7f69df200640 1 -- 172.21.15.7:0/934189997 >> 172.21.15.7:0/934189997 conn(0x7f69e00fbc90 msgr2=0x7f69e00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.742+0000 7f69df200640 1 -- 172.21.15.7:0/934189997 shutdown_connections 2024-06-13T07:01:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.742+0000 7f69df200640 1 -- 172.21.15.7:0/934189997 wait complete. 2024-06-13T07:01:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.744+0000 7f69df200640 1 Processor -- start 2024-06-13T07:01:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.745+0000 7f69df200640 1 -- start start 2024-06-13T07:01:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0100090 0x7f69e019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 0x7f69e019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0113a70 con 0x7f69e0100090 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01138f0 con 0x7f69e01053b0 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69df200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0113bf0 con 0x7f69e0102a00 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 0x7f69e019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 0x7f69e019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45112/0 (socket says 172.21.15.7:45112) 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48346/0 (socket says 172.21.15.7:48346) 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69dd400640 1 -- 172.21.15.7:0/2401979582 learned_addr learned my addr 172.21.15.7:0/2401979582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69dde00640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0100090 0x7f69e019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 -- 172.21.15.7:0/2401979582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 msgr2=0x7f69e019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 0x7f69e019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 -- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0100090 msgr2=0x7f69e019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.747+0000 7f69de800640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0100090 0x7f69e019e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69de800640 1 -- 172.21.15.7:0/2401979582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e01a5d60 con 0x7f69e01053b0 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69de800640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f69d800da10 tx=0x7f69d800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69d6a00640 1 -- 172.21.15.7:0/2401979582 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800bd70 con 0x7f69e01053b0 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69d6a00640 1 -- 172.21.15.7:0/2401979582 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d8003e50 con 0x7f69e01053b0 2024-06-13T07:01:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69d6a00640 1 -- 172.21.15.7:0/2401979582 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80106a0 con 0x7f69e01053b0 2024-06-13T07:01:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.748+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e01a5ff0 con 0x7f69e01053b0 2024-06-13T07:01:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.749+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e01a64c0 con 0x7f69e01053b0 2024-06-13T07:01:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.758+0000 7f69d6a00640 1 -- 172.21.15.7:0/2401979582 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f69d8003970 con 0x7f69e01053b0 2024-06-13T07:01:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.758+0000 7f69e7df1740 1 Processor -- start 2024-06-13T07:01:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.758+0000 7f69e7df1740 1 -- start start 2024-06-13T07:01:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52e7b0 0x557e8a5332b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a535db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557e8a491420 con 0x557e8a533920 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e8a438c50 con 0x557e8a52e7b0 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69e7df1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557e8a48d3e0 con 0x557e8a52ddc0 2024-06-13T07:01:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52e7b0 0x557e8a5332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45128/0 (socket says 172.21.15.7:45128) 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.759+0000 7f69dd400640 1 -- 172.21.15.7:0/810619831 learned_addr learned my addr 172.21.15.7:0/810619831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a535db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 -- 172.21.15.7:0/810619831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52e7b0 msgr2=0x557e8a5332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52e7b0 0x557e8a5332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 -- 172.21.15.7:0/810619831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 msgr2=0x557e8a535db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a535db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.760+0000 7f69dd400640 1 -- 172.21.15.7:0/810619831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8a1a5210 con 0x557e8a52ddc0 2024-06-13T07:01:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.761+0000 7f69dd400640 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005760 tx=0x7f69cc0056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b212505adc49e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69d6a00640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 0x7f69c40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69d6a00640 1 -- 172.21.15.7:0/2401979582 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f69d80cf6c0 con 0x7f69e01053b0 2024-06-13T07:01:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69dde00640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 0x7f69c40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69cb400640 1 -- 172.21.15.7:0/810619831 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc037040 con 0x557e8a52ddc0 2024-06-13T07:01:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69cb400640 1 -- 172.21.15.7:0/810619831 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69cc045070 con 0x557e8a52ddc0 2024-06-13T07:01:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.763+0000 7f69cb400640 1 -- 172.21.15.7:0/810619831 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc003d30 con 0x557e8a52ddc0 2024-06-13T07:01:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.764+0000 7f69e7df1740 1 -- 172.21.15.7:0/810619831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 msgr2=0x557e8a52e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.764+0000 7f69e7df1740 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005760 tx=0x7f69cc0056f0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.765+0000 7f69e7df1740 1 -- 172.21.15.7:0/810619831 shutdown_connections 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.765+0000 7f69e7df1740 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a535db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.765+0000 7f69e7df1740 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52e7b0 0x557e8a5332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.765+0000 7f69e7df1740 1 --2- 172.21.15.7:0/810619831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52ddc0 0x557e8a52e1e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.765+0000 7f69e7df1740 1 -- 172.21.15.7:0/810619831 >> 172.21.15.7:0/810619831 conn(0x557e8a4f5fb0 msgr2=0x557e8a529840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.766+0000 7f69e7df1740 1 -- 172.21.15.7:0/810619831 shutdown_connections 2024-06-13T07:01:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.766+0000 7f69e7df1740 1 -- 172.21.15.7:0/810619831 wait complete. 2024-06-13T07:01:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.766+0000 7f69dde00640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 0x7f69c40a5210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005dc0 tx=0x7f69d0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.766+0000 7f69e7df1740 1 -- start start 2024-06-13T07:01:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.767+0000 7f69e7df1740 1 Processor -- start 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.767+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a538150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.767+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.767+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a53f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69e7df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557e8a536870 con 0x557e8a533920 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a538150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45142/0 (socket says 172.21.15.7:45142) 2024-06-13T07:01:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 -- 172.21.15.7:0/2247999736 learned_addr learned my addr 172.21.15.7:0/2247999736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69e7df1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557e8a44c0b0 con 0x557e8a52ddc0 2024-06-13T07:01:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557e8a5369f0 con 0x557e8a52e7b0 2024-06-13T07:01:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69e7df1740 10 client.0 _ll_register_callbacks cb 0x557e8a437f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 -- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 msgr2=0x557e8a538150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a538150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 -- 172.21.15.7:0/2247999736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 msgr2=0x557e8a53f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a53f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.768+0000 7f69dd400640 1 -- 172.21.15.7:0/2247999736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8a271620 con 0x557e8a52e7b0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.769+0000 7f69dd400640 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69cc02a9e0 tx=0x7f69cc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da539dc2e5e9d3ce server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.769+0000 7f69bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.769+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc037040 con 0x557e8a52e7b0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.769+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f69cc003ab0 con 0x557e8a52e7b0 2024-06-13T07:01:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.769+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69cc045070 con 0x557e8a52e7b0 2024-06-13T07:01:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.771+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69cc057050 con 0x557e8a52e7b0 2024-06-13T07:01:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.771+0000 7f69e7df1740 10 client.26158 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.772+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e8a536870 con 0x557e8a52e7b0 2024-06-13T07:01:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.772+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69cc045070 con 0x557e8a52e7b0 2024-06-13T07:01:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.772+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69cc03d040 con 0x557e8a52e7b0 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 10 client.26158 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 20 client.26158 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a543d20 0x557e8a546110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 4 client.26158 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e8a49eb50 con 0x557e8a543d20 2024-06-13T07:01:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.775+0000 7f69dde00640 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a543d20 0x557e8a546110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.776+0000 7f69dde00640 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a543d20 0x557e8a546110 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb74f4a9a92b8fa server_cookie=653e5fa12ef8b7a2 in_seq=0 out_seq=0 2024-06-13T07:01:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.776+0000 7f69bfe00640 10 client.26158 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.777+0000 7f69bfe00640 1 -- 172.21.15.7:0/2247999736 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557e8a49eb50 con 0x557e8a543d20 2024-06-13T07:01:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.777+0000 7f69bfe00640 10 client.26158 handle_command_reply: tid=0 2024-06-13T07:01:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.777+0000 7f69e7df1740 1 client.26158 shutdown 2024-06-13T07:01:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.777+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 msgr2=0x557e8a538690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69cc02a9e0 tx=0x7f69cc005850 comp rx=0 tx=0).stop 2024-06-13T07:01:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 shutdown_connections 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a543d20 0x557e8a546110 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a53f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a538690 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 --2- 172.21.15.7:0/2247999736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a538150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 >> 172.21.15.7:0/2247999736 conn(0x557e8a4f5fb0 msgr2=0x557e8a5289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69bfe00640 0 client.26158 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.778+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 shutdown_connections 2024-06-13T07:01:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.779+0000 7f69e7df1740 1 -- 172.21.15.7:0/2247999736 wait complete. 2024-06-13T07:01:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.779+0000 7f69e7df1740 20 client.26158 trim_cache size 0 max 16384 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 Processor -- start 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 -- start start 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a5292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01b3080 con 0x557e8a533920 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01b08f0 con 0x557e8a52ddc0 2024-06-13T07:01:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.829+0000 7f69e7df1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01ae740 con 0x557e8a52e7b0 2024-06-13T07:01:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.830+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.830+0000 7f69dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.830+0000 7f69dde00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48366/0 (socket says 172.21.15.7:48366) 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.830+0000 7f69dde00640 1 -- 172.21.15.7:0/651587326 learned_addr learned my addr 172.21.15.7:0/651587326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.830+0000 7f69de800640 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a5292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 -- 172.21.15.7:0/651587326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 msgr2=0x557e8a53f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 -- 172.21.15.7:0/651587326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 msgr2=0x557e8a5292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a5292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 -- 172.21.15.7:0/651587326 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8a546740 con 0x557e8a52ddc0 2024-06-13T07:01:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.831+0000 7f69dde00640 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f69d0028fd0 tx=0x7f69d0015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4687d94fe904fb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.833+0000 7f69caa00640 1 -- 172.21.15.7:0/651587326 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0005020 con 0x557e8a52ddc0 2024-06-13T07:01:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.833+0000 7f69caa00640 1 -- 172.21.15.7:0/651587326 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d00047a0 con 0x557e8a52ddc0 2024-06-13T07:01:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.833+0000 7f69caa00640 1 -- 172.21.15.7:0/651587326 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0028810 con 0x557e8a52ddc0 2024-06-13T07:01:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 -- 172.21.15.7:0/651587326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 msgr2=0x557e8a5331e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f69d0028fd0 tx=0x7f69d0015000 comp rx=0 tx=0).stop 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 -- 172.21.15.7:0/651587326 shutdown_connections 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a533920 0x557e8a5292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 --2- 172.21.15.7:0/651587326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a52ddc0 0x557e8a5331e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.835+0000 7f69e7df1740 1 -- 172.21.15.7:0/651587326 >> 172.21.15.7:0/651587326 conn(0x557e8a4f5fb0 msgr2=0x557e8a535d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.836+0000 7f69e7df1740 1 -- 172.21.15.7:0/651587326 shutdown_connections 2024-06-13T07:01:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.836+0000 7f69e7df1740 1 -- 172.21.15.7:0/651587326 wait complete. 2024-06-13T07:01:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.836+0000 7f69e7df1740 1 -- start start 2024-06-13T07:01:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.837+0000 7f69e7df1740 1 Processor -- start 2024-06-13T07:01:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.838+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 0x557e8a53f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.838+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69e7df1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a533920 0x557e8a53c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69e7df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01aedb0 con 0x557e8a52ddc0 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69e7df1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01b0220 con 0x557e8a533920 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69e7df1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01af480 con 0x557e8a52e7b0 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69e7df1740 10 client.0 _ll_register_callbacks cb 0x7f69cc030e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45164/0 (socket says 172.21.15.7:45164) 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69dd400640 1 -- 172.21.15.7:0/524000814 learned_addr learned my addr 172.21.15.7:0/524000814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.839+0000 7f69de800640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a533920 0x557e8a53c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dd400640 1 -- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a533920 msgr2=0x557e8a53c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dd400640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a533920 0x557e8a53c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dde00640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 0x557e8a53f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dd400640 1 -- 172.21.15.7:0/524000814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 msgr2=0x557e8a53f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dd400640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 0x557e8a53f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dd400640 1 -- 172.21.15.7:0/524000814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8a5ac990 con 0x557e8a52e7b0 2024-06-13T07:01:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.840+0000 7f69dde00640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 0x557e8a53f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.841+0000 7f69dd400640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69cc030410 tx=0x7f69cc005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de6586d83d5f879 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.841+0000 7f69bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.842+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc037040 con 0x557e8a52e7b0 2024-06-13T07:01:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.842+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f69cc049c00 con 0x557e8a52e7b0 2024-06-13T07:01:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.842+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d00047a0 con 0x557e8a52e7b0 2024-06-13T07:01:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.846+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f69cc005040 con 0x557e8a52e7b0 2024-06-13T07:01:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.846+0000 7f69e7df1740 10 client.26164 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.846+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e8a5adc50 con 0x557e8a52e7b0 2024-06-13T07:01:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.846+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d00047a0 con 0x557e8a52e7b0 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.846+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f69cc032440 con 0x557e8a52e7b0 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.847+0000 7f69e7df1740 10 client.26164 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.848+0000 7f69e7df1740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.848+0000 7f69e7df1740 20 client.26164 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.848+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a5ae580 0x557e8a5b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.848+0000 7f69e7df1740 4 client.26164 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.848+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e8a498ea0 con 0x557e8a5ae580 2024-06-13T07:01:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.849+0000 7f69dde00640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a5ae580 0x557e8a5b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.850+0000 7f69dde00640 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a5ae580 0x557e8a5b09a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b746894892a691a4 server_cookie=f4817b2f6f7ab72b in_seq=0 out_seq=0 2024-06-13T07:01:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.850+0000 7f69bfe00640 10 client.26164 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 320.0 s, 583.6 tps, lat 170.713 ms stddev 228.770 2024-06-13T07:01:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.891+0000 7f69bfe00640 1 -- 172.21.15.7:0/524000814 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557e8a498ea0 con 0x557e8a5ae580 2024-06-13T07:01:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.891+0000 7f69bfe00640 10 client.26164 handle_command_reply: tid=0 2024-06-13T07:01:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.891+0000 7f69e7df1740 1 client.26164 shutdown 2024-06-13T07:01:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 msgr2=0x557e8a53c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69cc030410 tx=0x7f69cc005b20 comp rx=0 tx=0).stop 2024-06-13T07:01:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 shutdown_connections 2024-06-13T07:01:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557e8a5ae580 0x557e8a5b09a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557e8a533920 0x557e8a53c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e8a52e7b0 0x557e8a53c0e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 --2- 172.21.15.7:0/524000814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557e8a52ddc0 0x557e8a53f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 >> 172.21.15.7:0/524000814 conn(0x557e8a4f5fb0 msgr2=0x557e8a52fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.892+0000 7f69bfe00640 0 client.26164 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.893+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 shutdown_connections 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.893+0000 7f69e7df1740 1 -- 172.21.15.7:0/524000814 wait complete. 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.893+0000 7f69e7df1740 20 client.26164 trim_cache size 0 max 16384 2024-06-13T07:01:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:01:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:01:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 msgr2=0x7f69c40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 0x7f69c40a5210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005dc0 tx=0x7f69d0005d30 comp rx=0 tx=0).stop 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 msgr2=0x7f69e01a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f69d800da10 tx=0x7f69d800ded0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 shutdown_connections 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f69c40a2d50 0x7f69c40a5210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e01053b0 0x7f69e01a56f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e0102a00 0x7f69e019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 --2- 172.21.15.7:0/2401979582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69e0100090 0x7f69e019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.896+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 >> 172.21.15.7:0/2401979582 conn(0x7f69e00fbc90 msgr2=0x7f69e0100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.898+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 shutdown_connections 2024-06-13T07:01:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:22.898+0000 7f69df200640 1 -- 172.21.15.7:0/2401979582 wait complete. 2024-06-13T07:01:23.803 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:23 smithi007 ceph-mon[32077]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 609 op/s 2024-06-13T07:01:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:23 smithi189 ceph-mon[36616]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 609 op/s 2024-06-13T07:01:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:23 smithi175 ceph-mon[36886]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 609 op/s 2024-06-13T07:01:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.038+0000 7fa590a00700 1 -- 172.21.15.7:0/2749797995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580003c20 con 0x7fa594102fb0 2024-06-13T07:01:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.039+0000 7fa593200700 1 -- 172.21.15.7:0/2749797995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa594102fb0 msgr2=0x7fa594107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.039+0000 7fa593200700 1 --2- 172.21.15.7:0/2749797995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa594102fb0 0x7fa594107000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5800099c0 tx=0x7fa580009cd0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 -- 172.21.15.7:0/2749797995 shutdown_connections 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 --2- 172.21.15.7:0/2749797995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5941076e0 0x7fa59410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 --2- 172.21.15.7:0/2749797995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa5941029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 --2- 172.21.15.7:0/2749797995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa594102fb0 0x7fa594107000 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 -- 172.21.15.7:0/2749797995 >> 172.21.15.7:0/2749797995 conn(0x7fa5940fded0 msgr2=0x7fa5941002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 -- 172.21.15.7:0/2749797995 shutdown_connections 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.040+0000 7fa593200700 1 -- 172.21.15.7:0/2749797995 wait complete. 2024-06-13T07:01:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.041+0000 7fa593200700 1 Processor -- start 2024-06-13T07:01:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.041+0000 7fa593200700 1 -- start start 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5941076e0 0x7fa594198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941126c0 con 0x7fa5941076e0 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112540 con 0x7fa594102600 2024-06-13T07:01:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.043+0000 7fa593200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112840 con 0x7fa594102fb0 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.044+0000 7fa591400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.044+0000 7fa591e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.044+0000 7fa591400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45174/0 (socket says 172.21.15.7:45174) 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.044+0000 7fa591400700 1 -- 172.21.15.7:0/1013656797 learned_addr learned my addr 172.21.15.7:0/1013656797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.044+0000 7fa591e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48388/0 (socket says 172.21.15.7:48388) 2024-06-13T07:01:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591400700 1 -- 172.21.15.7:0/1013656797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 msgr2=0x7fa59419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591400700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591400700 1 -- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5941076e0 msgr2=0x7fa594198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591400700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5941076e0 0x7fa594198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591400700 1 -- 172.21.15.7:0/1013656797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941991a0 con 0x7fa594102fb0 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.045+0000 7fa591e00700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.046+0000 7fa591400700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa580006010 tx=0x7fa580003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.047+0000 7fa58aa00700 1 -- 172.21.15.7:0/1013656797 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580003680 con 0x7fa594102fb0 2024-06-13T07:01:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.048+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa594199430 con 0x7fa594102fb0 2024-06-13T07:01:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.048+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5940746c0 con 0x7fa594102fb0 2024-06-13T07:01:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.048+0000 7fa58aa00700 1 -- 172.21.15.7:0/1013656797 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5800055c0 con 0x7fa594102fb0 2024-06-13T07:01:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.048+0000 7fa58aa00700 1 -- 172.21.15.7:0/1013656797 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58000f6c0 con 0x7fa594102fb0 2024-06-13T07:01:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.052+0000 7fa58aa00700 1 -- 172.21.15.7:0/1013656797 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa58000f860 con 0x7fa594102fb0 2024-06-13T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.054+0000 7fa5a3969b80 1 Processor -- start 2024-06-13T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.054+0000 7fa5a3969b80 1 -- start start 2024-06-13T07:01:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.054+0000 7fa58aa00700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 0x7fa5780a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.054+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef99a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef99ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa58aa00700 1 -- 172.21.15.7:0/1013656797 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa5800d4320 con 0x7fa594102fb0 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef8a7a90 con 0x55e8ef99a9e0 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef8eab80 con 0x55e8ef9a35f0 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef8a5970 con 0x55e8ef914ec0 2024-06-13T07:01:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa592800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa592800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48394/0 (socket says 172.21.15.7:48394) 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa592800700 1 -- 172.21.15.7:0/2318033080 learned_addr learned my addr 172.21.15.7:0/2318033080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.055+0000 7fa592800700 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef99a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 -- 172.21.15.7:0/2318033080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 msgr2=0x55e8ef99a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef99a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 -- 172.21.15.7:0/2318033080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 msgr2=0x55e8ef99ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef99ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 -- 172.21.15.7:0/2318033080 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ef893010 con 0x55e8ef9a35f0 2024-06-13T07:01:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.056+0000 7fa592800700 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0060b0 tx=0x7fa58c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0835c543e3ab02d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.057+0000 7fa591e00700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 0x7fa5780a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.057+0000 7fa57f400700 1 -- 172.21.15.7:0/2318033080 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c014040 con 0x55e8ef9a35f0 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.057+0000 7fa57f400700 1 -- 172.21.15.7:0/2318033080 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58c004db0 con 0x55e8ef9a35f0 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.058+0000 7fa57f400700 1 -- 172.21.15.7:0/2318033080 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c005380 con 0x55e8ef9a35f0 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.058+0000 7fa591e00700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 0x7fa5780a5300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa5941074c0 tx=0x7fa584009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2318033080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 msgr2=0x55e8ef9a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0060b0 tx=0x7fa58c009ea0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2318033080 shutdown_connections 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a59e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef99a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2318033080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef99ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2318033080 >> 172.21.15.7:0/2318033080 conn(0x55e8ef8c51f0 msgr2=0x55e8ef9948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2318033080 shutdown_connections 2024-06-13T07:01:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.059+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2318033080 wait complete. 2024-06-13T07:01:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.060+0000 7fa5a3969b80 1 -- start start 2024-06-13T07:01:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.060+0000 7fa5a3969b80 1 Processor -- start 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef886fe0 con 0x55e8ef99a9e0 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef8e6b40 con 0x55e8ef9a35f0 2024-06-13T07:01:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.061+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef9a6200 con 0x55e8ef914ec0 2024-06-13T07:01:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa5a3969b80 10 client.0 _ll_register_callbacks cb 0x55e8ef8847a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa592800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45198/0 (socket says 172.21.15.7:45198) 2024-06-13T07:01:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 -- 172.21.15.7:0/3685279536 learned_addr learned my addr 172.21.15.7:0/3685279536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 -- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 msgr2=0x55e8ef9a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.062+0000 7fa591e00700 1 -- 172.21.15.7:0/3685279536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 msgr2=0x55e8ef9a00c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.063+0000 7fa591400700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.063+0000 7fa591e00700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.063+0000 7fa591e00700 1 -- 172.21.15.7:0/3685279536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ef839950 con 0x55e8ef914ec0 2024-06-13T07:01:24.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.063+0000 7fa591400700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.064+0000 7fa573e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.064+0000 7fa591e00700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa584003aa0 tx=0x7fa5840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e694e23adac372 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.064+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584003e60 con 0x55e8ef914ec0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.064+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa58400ebd0 con 0x55e8ef914ec0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.064+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c004db0 con 0x55e8ef914ec0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.065+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa58400edd0 con 0x55e8ef914ec0 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.065+0000 7fa5a3969b80 10 client.26176 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:24.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.065+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8ef9b0af0 con 0x55e8ef914ec0 2024-06-13T07:01:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.066+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58c004db0 con 0x55e8ef914ec0 2024-06-13T07:01:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa58401e710 con 0x55e8ef914ec0 2024-06-13T07:01:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 10 client.26176 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 20 client.26176 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8ef9b3130 0x55e8ef9b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 4 client.26176 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.067+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8ef8f8080 con 0x55e8ef9b3130 2024-06-13T07:01:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.068+0000 7fa591400700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8ef9b3130 0x55e8ef9b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.070+0000 7fa591400700 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8ef9b3130 0x55e8ef9b5580 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48638aacdcbdac36 server_cookie=ec2c0399ca945b42 in_seq=0 out_seq=0 2024-06-13T07:01:24.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.070+0000 7fa573e00700 10 client.26176 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:24.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.071+0000 7fa573e00700 1 -- 172.21.15.7:0/3685279536 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e8ef8f8080 con 0x55e8ef9b3130 2024-06-13T07:01:24.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.071+0000 7fa573e00700 10 client.26176 handle_command_reply: tid=0 2024-06-13T07:01:24.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.072+0000 7fa5a3969b80 1 client.26176 shutdown 2024-06-13T07:01:24.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.073+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 msgr2=0x55e8ef9a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.073+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa584003aa0 tx=0x7fa5840047c0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.073+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 shutdown_connections 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.073+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8ef9b3130 0x55e8ef9b5580 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef914ec0 0x55e8ef9a2dc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/3685279536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 >> 172.21.15.7:0/3685279536 conn(0x55e8ef8c51f0 msgr2=0x55e8ef994490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa573e00700 0 client.26176 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.074+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 shutdown_connections 2024-06-13T07:01:24.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.075+0000 7fa5a3969b80 1 -- 172.21.15.7:0/3685279536 wait complete. 2024-06-13T07:01:24.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.075+0000 7fa5a3969b80 20 client.26176 trim_cache size 0 max 16384 2024-06-13T07:01:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.118+0000 7fa5a3969b80 1 Processor -- start 2024-06-13T07:01:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.118+0000 7fa5a3969b80 1 -- start start 2024-06-13T07:01:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 0x55e8ef9af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa574011220 con 0x55e8ef914ec0 2024-06-13T07:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400ea50 con 0x55e8ef9a35f0 2024-06-13T07:01:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.119+0000 7fa5a3969b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400c8a0 con 0x55e8ef99a9e0 2024-06-13T07:01:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.120+0000 7fa592800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.120+0000 7fa592800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48418/0 (socket says 172.21.15.7:48418) 2024-06-13T07:01:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.120+0000 7fa592800700 1 -- 172.21.15.7:0/4056178492 learned_addr learned my addr 172.21.15.7:0/4056178492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.120+0000 7fa591e00700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 0x55e8ef9af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.120+0000 7fa592800700 1 -- 172.21.15.7:0/4056178492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 msgr2=0x55e8ef9a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa591400700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa592800700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa592800700 1 -- 172.21.15.7:0/4056178492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 msgr2=0x55e8ef9af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa592800700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 0x55e8ef9af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa592800700 1 -- 172.21.15.7:0/4056178492 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ef9a0ff0 con 0x55e8ef9a35f0 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa591e00700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 0x55e8ef9af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa591400700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.121+0000 7fa592800700 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009c10 tx=0x7fa58c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48285ae2822d7d96 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.122+0000 7fa57ea00700 1 -- 172.21.15.7:0/4056178492 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c008e70 con 0x55e8ef9a35f0 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.122+0000 7fa57ea00700 1 -- 172.21.15.7:0/4056178492 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58c009380 con 0x55e8ef9a35f0 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.122+0000 7fa57ea00700 1 -- 172.21.15.7:0/4056178492 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c004030 con 0x55e8ef9a35f0 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 -- 172.21.15.7:0/4056178492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 msgr2=0x55e8ef9940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009c10 tx=0x7fa58c015040 comp rx=0 tx=0).stop 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 -- 172.21.15.7:0/4056178492 shutdown_connections 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9940e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef914ec0 0x55e8ef9af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.123+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/4056178492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.124+0000 7fa5a3969b80 1 -- 172.21.15.7:0/4056178492 >> 172.21.15.7:0/4056178492 conn(0x55e8ef8c51f0 msgr2=0x55e8ef9939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.124+0000 7fa5a3969b80 1 -- 172.21.15.7:0/4056178492 shutdown_connections 2024-06-13T07:01:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.124+0000 7fa5a3969b80 1 -- 172.21.15.7:0/4056178492 wait complete. 2024-06-13T07:01:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.125+0000 7fa5a3969b80 1 -- start start 2024-06-13T07:01:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.125+0000 7fa5a3969b80 1 Processor -- start 2024-06-13T07:01:24.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.127+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.127+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.127+0000 7fa5a3969b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.127+0000 7fa591e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa591e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48426/0 (socket says 172.21.15.7:48426) 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa591e00700 1 -- 172.21.15.7:0/2279356412 learned_addr learned my addr 172.21.15.7:0/2279356412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa591400700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400cf10 con 0x55e8ef99a9e0 2024-06-13T07:01:24.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400e380 con 0x55e8ef914ec0 2024-06-13T07:01:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa57400d5e0 con 0x55e8ef9a35f0 2024-06-13T07:01:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa5a3969b80 10 client.0 _ll_register_callbacks cb 0x55e8ef8847a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:24.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.128+0000 7fa592800700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 -- 172.21.15.7:0/2279356412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef9a35f0 msgr2=0x55e8ef9ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9ab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 -- 172.21.15.7:0/2279356412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 msgr2=0x55e8ef9aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:24.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ef9aba00 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.129+0000 7fa591e00700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa584005a80 tx=0x7fa584004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab19b7c01e791a32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa573e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58401d380 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa58401e3f0 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c009380 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa58401e5f0 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa5a3969b80 10 client.26204 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.130+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e8efa2f6c0 con 0x55e8ef914ec0 2024-06-13T07:01:24.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.131+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58c009380 con 0x55e8ef914ec0 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa58403b030 con 0x55e8ef914ec0 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 20 client.26204 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8efa2fdd0 0x55e8efa321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 4 client.26204 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.132+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8ef8f2600 con 0x55e8efa2fdd0 2024-06-13T07:01:24.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.133+0000 7fa591400700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8efa2fdd0 0x55e8efa321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.134+0000 7fa591400700 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8efa2fdd0 0x55e8efa321f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fdd3fbe8da8a43 server_cookie=acdace52eed6db7d in_seq=0 out_seq=0 2024-06-13T07:01:24.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.134+0000 7fa573e00700 10 client.26204 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.137+0000 7fa573e00700 1 -- 172.21.15.7:0/2279356412 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8ef8f2600 con 0x55e8efa2fdd0 2024-06-13T07:01:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.137+0000 7fa573e00700 10 client.26204 handle_command_reply: tid=0 2024-06-13T07:01:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.137+0000 7fa5a3969b80 1 client.26204 shutdown 2024-06-13T07:01:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.137+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 msgr2=0x55e8ef9aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.137+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa584005a80 tx=0x7fa584004680 comp rx=0 tx=0).stop 2024-06-13T07:01:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 shutdown_connections 2024-06-13T07:01:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e8efa2fdd0 0x55e8efa321f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8ef9a35f0 0x55e8ef9ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e8ef914ec0 0x55e8ef9aa570 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 --2- 172.21.15.7:0/2279356412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ef99a9e0 0x55e8ef9aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 >> 172.21.15.7:0/2279356412 conn(0x55e8ef8c51f0 msgr2=0x55e8ef9aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa573e00700 0 client.26204 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 shutdown_connections 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 1 -- 172.21.15.7:0/2279356412 wait complete. 2024-06-13T07:01:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.138+0000 7fa5a3969b80 20 client.26204 trim_cache size 0 max 16384 2024-06-13T07:01:24.139 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:01:24.139 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:01:24.139 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "4f2c68d0-417e-4a88-ac6c-2d256d7030fa", 2024-06-13T07:01:24.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:01:24.139 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:01:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 msgr2=0x7fa5780a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 0x7fa5780a5300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa5941074c0 tx=0x7fa584009260 comp rx=0 tx=0).stop 2024-06-13T07:01:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 msgr2=0x7fa59419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa580006010 tx=0x7fa580003a90 comp rx=0 tx=0).stop 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 shutdown_connections 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5941076e0 0x7fa594198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa5780a2e40 0x7fa5780a5300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa594102600 0x7fa59419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 --2- 172.21.15.7:0/1013656797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa594102fb0 0x7fa59419ed70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.140+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 >> 172.21.15.7:0/1013656797 conn(0x7fa5940fded0 msgr2=0x7fa5940feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.141+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 shutdown_connections 2024-06-13T07:01:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:24.141+0000 7fa593200700 1 -- 172.21.15.7:0/1013656797 wait complete. 2024-06-13T07:01:24.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:24 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:24.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:24 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:24.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:24.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:24.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:24.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:24.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:24 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:25.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:01:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:01:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 567 op/s 2024-06-13T07:01:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:25 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 567 op/s 2024-06-13T07:01:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:01:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:25.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:01:25.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:25 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 567 op/s 2024-06-13T07:01:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:25 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:01:27.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:27 smithi007 ceph-mon[32077]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 494 op/s 2024-06-13T07:01:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:27 smithi189 ceph-mon[36616]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 494 op/s 2024-06-13T07:01:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:27 smithi175 ceph-mon[36886]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 494 op/s 2024-06-13T07:01:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 325.0 s, 658.4 tps, lat 152.829 ms stddev 175.412 2024-06-13T07:01:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.391+0000 7f80fa000700 1 -- 172.21.15.7:0/705480076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4075180 msgr2=0x7f80f410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.391+0000 7f80fa000700 1 --2- 172.21.15.7:0/705480076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4075180 0x7f80f410e810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f80e40098f0 tx=0x7f80e4009c00 comp rx=0 tx=0).stop 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 -- 172.21.15.7:0/705480076 shutdown_connections 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 --2- 172.21.15.7:0/705480076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f4111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 --2- 172.21.15.7:0/705480076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4074860 0x7f80f4074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 --2- 172.21.15.7:0/705480076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4075180 0x7f80f410e810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 -- 172.21.15.7:0/705480076 >> 172.21.15.7:0/705480076 conn(0x7f80f40fdf60 msgr2=0x7f80f4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 -- 172.21.15.7:0/705480076 shutdown_connections 2024-06-13T07:01:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.392+0000 7f80fa000700 1 -- 172.21.15.7:0/705480076 wait complete. 2024-06-13T07:01:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.393+0000 7f80fa000700 1 Processor -- start 2024-06-13T07:01:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 -- start start 2024-06-13T07:01:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 0x7f80f419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4075180 0x7f80f419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4114ab0 con 0x7f80f410ed50 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4114930 con 0x7f80f4075180 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.394+0000 7f80fa000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80f4114c30 con 0x7f80f4074860 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.395+0000 7f80f3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 0x7f80f419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.395+0000 7f80f3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 0x7f80f419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45234/0 (socket says 172.21.15.7:45234) 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.395+0000 7f80f3400700 1 -- 172.21.15.7:0/3327916804 learned_addr learned my addr 172.21.15.7:0/3327916804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.395+0000 7f80f3e00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.395+0000 7f80f2a00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4075180 0x7f80f419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.396+0000 7f80f3e00700 1 -- 172.21.15.7:0/3327916804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 msgr2=0x7f80f419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.396+0000 7f80f3e00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 0x7f80f419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.396+0000 7f80f3e00700 1 -- 172.21.15.7:0/3327916804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4075180 msgr2=0x7f80f419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.396+0000 7f80f3e00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4075180 0x7f80f419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.396+0000 7f80f3e00700 1 -- 172.21.15.7:0/3327916804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419ff70 con 0x7f80f410ed50 2024-06-13T07:01:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.397+0000 7f80f3e00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f419f830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3c0 tx=0x7f80e000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.398+0000 7f80ebe00700 1 -- 172.21.15.7:0/3327916804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0015020 con 0x7f80f410ed50 2024-06-13T07:01:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.399+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f41a0260 con 0x7f80f410ed50 2024-06-13T07:01:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.399+0000 7f80ebe00700 1 -- 172.21.15.7:0/3327916804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0011b00 con 0x7f80f410ed50 2024-06-13T07:01:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.400+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41023a0 con 0x7f80f410ed50 2024-06-13T07:01:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.400+0000 7f80ebe00700 1 -- 172.21.15.7:0/3327916804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0010be0 con 0x7f80f410ed50 2024-06-13T07:01:28.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.407+0000 7f80ebe00700 1 -- 172.21.15.7:0/3327916804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f80e0018020 con 0x7f80f410ed50 2024-06-13T07:01:28.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.409+0000 7f80ebe00700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 0x7f80d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.410+0000 7f80ebe00700 1 -- 172.21.15.7:0/3327916804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80e00ce8a0 con 0x7f80f410ed50 2024-06-13T07:01:28.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.410+0000 7f80f3400700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 0x7f80d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.410+0000 7f81049b2b80 1 Processor -- start 2024-06-13T07:01:28.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.411+0000 7f81049b2b80 1 -- start start 2024-06-13T07:01:28.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.411+0000 7f80f3400700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 0x7f80d80a5160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fd0 tx=0x7f80ec016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.411+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 0x555e9dd21f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.412+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 0x555e9dd1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.412+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.412+0000 7f81049b2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555e9dc2aa10 con 0x555e9dd1c240 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.412+0000 7f81049b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e9dc6db00 con 0x555e9dd21b90 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.412+0000 7f81049b2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e9dc288f0 con 0x555e9dd224b0 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.413+0000 7f80f3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 0x555e9dd21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.413+0000 7f80f3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 0x555e9dd21f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48452/0 (socket says 172.21.15.7:48452) 2024-06-13T07:01:28.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.413+0000 7f80f3400700 1 -- 172.21.15.7:0/2001899359 learned_addr learned my addr 172.21.15.7:0/2001899359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:28.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.413+0000 7f80f2a00700 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 0x555e9dd1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 -- 172.21.15.7:0/2001899359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 msgr2=0x555e9dd21f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 0x555e9dd21f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 -- 172.21.15.7:0/2001899359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 msgr2=0x555e9dd1c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 -- 172.21.15.7:0/2001899359 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9dc15f90 con 0x555e9dd224b0 2024-06-13T07:01:28.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.414+0000 7f80f2a00700 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 0x555e9dd1bd00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009fd0 tx=0x7f80e400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6c6d2bd160ed server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.415+0000 7f80e8c00700 1 -- 172.21.15.7:0/2001899359 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4026070 con 0x555e9dd224b0 2024-06-13T07:01:28.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.415+0000 7f80e8c00700 1 -- 172.21.15.7:0/2001899359 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e400f8d0 con 0x555e9dd224b0 2024-06-13T07:01:28.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.415+0000 7f80e8c00700 1 -- 172.21.15.7:0/2001899359 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003cc0 con 0x555e9dd224b0 2024-06-13T07:01:28.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 -- 172.21.15.7:0/2001899359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 msgr2=0x555e9dd1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 0x555e9dd1bd00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009fd0 tx=0x7f80e400bf90 comp rx=0 tx=0).stop 2024-06-13T07:01:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 -- 172.21.15.7:0/2001899359 shutdown_connections 2024-06-13T07:01:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd1c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd21b90 0x555e9dd21f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2001899359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd224b0 0x555e9dd1bd00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.416+0000 7f81049b2b80 1 -- 172.21.15.7:0/2001899359 >> 172.21.15.7:0/2001899359 conn(0x555e9dc48170 msgr2=0x555e9dd16ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.417+0000 7f81049b2b80 1 -- 172.21.15.7:0/2001899359 shutdown_connections 2024-06-13T07:01:28.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.418+0000 7f81049b2b80 1 -- 172.21.15.7:0/2001899359 wait complete. 2024-06-13T07:01:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.418+0000 7f81049b2b80 1 -- start start 2024-06-13T07:01:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.419+0000 7f81049b2b80 1 Processor -- start 2024-06-13T07:01:28.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 0x555e9dd25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd25b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555e9dc09f60 con 0x555e9dd21b90 2024-06-13T07:01:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e9dc69ac0 con 0x555e9dd224b0 2024-06-13T07:01:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e9dd28e30 con 0x555e9dd1c240 2024-06-13T07:01:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f81049b2b80 10 client.0 _ll_register_callbacks cb 0x555e9dc03380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f80f3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f80f3400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45260/0 (socket says 172.21.15.7:45260) 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.420+0000 7f80f3400700 1 -- 172.21.15.7:0/1916434866 learned_addr learned my addr 172.21.15.7:0/1916434866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.421+0000 7f80f3e00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd25b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.421+0000 7f80f2a00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 0x555e9dd25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.421+0000 7f80f3e00700 1 -- 172.21.15.7:0/1916434866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 msgr2=0x555e9dd282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.421+0000 7f80f3e00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.421+0000 7f80f3e00700 1 -- 172.21.15.7:0/1916434866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 msgr2=0x555e9dd25630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80f3e00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 0x555e9dd25630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80f3e00700 1 -- 172.21.15.7:0/1916434866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9dbbc8d0 con 0x555e9dd224b0 2024-06-13T07:01:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80f3400700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80f2a00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 0x555e9dd25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80f3e00700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd25b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80e0010790 tx=0x7f80e009c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed16204edb03be4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.422+0000 7f80dd600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00a4aa0 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80e0010380 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e400f8d0 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f80e0010580 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f81049b2b80 10 client.26216 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e9dd33720 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.423+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e400f8d0 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f80e009b280 con 0x555e9dd224b0 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 20 client.26216 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dd35d60 0x555e9dd38180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 4 client.26216 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e9dc7b000 con 0x555e9dd35d60 2024-06-13T07:01:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.424+0000 7f80f3400700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dd35d60 0x555e9dd38180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.425+0000 7f80f3400700 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dd35d60 0x555e9dd38180 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df99e17c6101677 server_cookie=6ae880bbe1172040 in_seq=0 out_seq=0 2024-06-13T07:01:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.425+0000 7f80dd600700 10 client.26216 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.427+0000 7f80dd600700 1 -- 172.21.15.7:0/1916434866 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e9dc7b000 con 0x555e9dd35d60 2024-06-13T07:01:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.427+0000 7f80dd600700 10 client.26216 handle_command_reply: tid=0 2024-06-13T07:01:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.427+0000 7f81049b2b80 1 client.26216 shutdown 2024-06-13T07:01:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 msgr2=0x555e9dd25b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd25b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80e0010790 tx=0x7f80e009c690 comp rx=0 tx=0).stop 2024-06-13T07:01:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 shutdown_connections 2024-06-13T07:01:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dd35d60 0x555e9dd38180 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f80dd600700 0 client.26216 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd25b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd1c240 0x555e9dd282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 --2- 172.21.15.7:0/1916434866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd21b90 0x555e9dd25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.428+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 >> 172.21.15.7:0/1916434866 conn(0x555e9dc48170 msgr2=0x555e9dd10790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.429+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 shutdown_connections 2024-06-13T07:01:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.429+0000 7f81049b2b80 1 -- 172.21.15.7:0/1916434866 wait complete. 2024-06-13T07:01:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.429+0000 7f81049b2b80 20 client.26216 trim_cache size 0 max 16384 2024-06-13T07:01:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.463+0000 7f81049b2b80 1 Processor -- start 2024-06-13T07:01:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.463+0000 7f81049b2b80 1 -- start start 2024-06-13T07:01:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd23060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd224b0 0x555e9dd27e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40112a0 con 0x555e9dd224b0 2024-06-13T07:01:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80d400ead0 con 0x555e9dd1c240 2024-06-13T07:01:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f81049b2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80d400c920 con 0x555e9dd21b90 2024-06-13T07:01:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48464/0 (socket says 172.21.15.7:48464) 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd23060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 -- 172.21.15.7:0/2363280147 learned_addr learned my addr 172.21.15.7:0/2363280147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3e00700 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd224b0 0x555e9dd27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 -- 172.21.15.7:0/2363280147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 msgr2=0x555e9dd23060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd23060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 -- 172.21.15.7:0/2363280147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd224b0 msgr2=0x555e9dd27e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd224b0 0x555e9dd27e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.464+0000 7f80f3400700 1 -- 172.21.15.7:0/2363280147 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9dd387e0 con 0x555e9dd1c240 2024-06-13T07:01:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.465+0000 7f80f3400700 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00b010 tx=0x7f80ec004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35eb80c718f9c93 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.465+0000 7f80dfe00700 1 -- 172.21.15.7:0/2363280147 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0087d0 con 0x555e9dd1c240 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.465+0000 7f80dfe00700 1 -- 172.21.15.7:0/2363280147 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec008e50 con 0x555e9dd1c240 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.465+0000 7f80dfe00700 1 -- 172.21.15.7:0/2363280147 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec038cc0 con 0x555e9dd1c240 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 -- 172.21.15.7:0/2363280147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 msgr2=0x555e9dd11130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00b010 tx=0x7f80ec004b40 comp rx=0 tx=0).stop 2024-06-13T07:01:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 -- 172.21.15.7:0/2363280147 shutdown_connections 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd224b0 0x555e9dd27e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd1c240 0x555e9dd11130 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 --2- 172.21.15.7:0/2363280147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd23060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.466+0000 7f81049b2b80 1 -- 172.21.15.7:0/2363280147 >> 172.21.15.7:0/2363280147 conn(0x555e9dc48170 msgr2=0x555e9dd22950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.467+0000 7f81049b2b80 1 -- 172.21.15.7:0/2363280147 shutdown_connections 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.467+0000 7f81049b2b80 1 -- 172.21.15.7:0/2363280147 wait complete. 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.467+0000 7f81049b2b80 1 -- start start 2024-06-13T07:01:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.468+0000 7f81049b2b80 1 Processor -- start 2024-06-13T07:01:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd2e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80d400cf90 con 0x555e9dd1c240 2024-06-13T07:01:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80d400e400 con 0x555e9dd224b0 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.470+0000 7f81049b2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80d400d660 con 0x555e9dd21b90 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f81049b2b80 10 client.0 _ll_register_callbacks cb 0x555e9dc03380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd2e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48466/0 (socket says 172.21.15.7:48466) 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 -- 172.21.15.7:0/4191054324 learned_addr learned my addr 172.21.15.7:0/4191054324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f2a00700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 -- 172.21.15.7:0/4191054324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 msgr2=0x555e9dd2ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd2ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.471+0000 7f80f3e00700 1 -- 172.21.15.7:0/4191054324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 msgr2=0x555e9dd2e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80f3e00700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd2e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80f3e00700 1 -- 172.21.15.7:0/4191054324 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9dd3da50 con 0x555e9dd224b0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80f3400700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd2e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80f3e00700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a03a0 tx=0x7f80e00a4680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1aa4022ba0bdaac server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80dd600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00ac050 con 0x555e9dd224b0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.472+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ec008e50 con 0x555e9dd224b0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.473+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80e0097d90 con 0x555e9dd224b0 2024-06-13T07:01:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.473+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f80e00a4aa0 con 0x555e9dd224b0 2024-06-13T07:01:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.473+0000 7f81049b2b80 10 client.26222 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.473+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e9dda0700 con 0x555e9dd224b0 2024-06-13T07:01:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.473+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec008e50 con 0x555e9dd224b0 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f80e00183a0 con 0x555e9dd224b0 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 10 client.26222 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 20 client.26222 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dda0ca0 0x555e9dda30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 4 client.26222 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e9dc75580 con 0x555e9dda0ca0 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.474+0000 7f80f3400700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dda0ca0 0x555e9dda30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.475+0000 7f80f3400700 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dda0ca0 0x555e9dda30c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb3847c9cb21dd server_cookie=de8a68c8a7a499dd in_seq=0 out_seq=0 2024-06-13T07:01:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.475+0000 7f80dd600700 10 client.26222 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.475+0000 7f80dd600700 1 -- 172.21.15.7:0/4191054324 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e9dc75580 con 0x555e9dda0ca0 2024-06-13T07:01:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.475+0000 7f80dd600700 10 client.26222 handle_command_reply: tid=0 2024-06-13T07:01:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.475+0000 7f81049b2b80 1 client.26222 shutdown 2024-06-13T07:01:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 msgr2=0x555e9dd3d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a03a0 tx=0x7f80e00a4680 comp rx=0 tx=0).stop 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 shutdown_connections 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x555e9dda0ca0 0x555e9dda30c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e9dd224b0 0x555e9dd3d240 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e9dd1c240 0x555e9dd2e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 --2- 172.21.15.7:0/4191054324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e9dd21b90 0x555e9dd2ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f80dd600700 0 client.26222 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 >> 172.21.15.7:0/4191054324 conn(0x555e9dc48170 msgr2=0x555e9dd23120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 shutdown_connections 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 1 -- 172.21.15.7:0/4191054324 wait complete. 2024-06-13T07:01:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.476+0000 7f81049b2b80 20 client.26222 trim_cache size 0 max 16384 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "4f2c68d0-417e-4a88-ac6c-2d256d7030fa": { 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "4f2c68d0-417e-4a88-ac6c-2d256d7030fa", 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:01:28.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:01:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:01:28.480 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:01:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 msgr2=0x7f80d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 0x7f80d80a5160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fd0 tx=0x7f80ec016040 comp rx=0 tx=0).stop 2024-06-13T07:01:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 msgr2=0x7f80f419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f419f830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3c0 tx=0x7f80e000b6d0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 shutdown_connections 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80f410ed50 0x7f80f419f830 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80f4074860 0x7f80f419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f80d80a2ca0 0x7f80d80a5160 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 --2- 172.21.15.7:0/3327916804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80f4075180 0x7f80f419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 >> 172.21.15.7:0/3327916804 conn(0x7f80f40fdf60 msgr2=0x7f80f40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 shutdown_connections 2024-06-13T07:01:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:01:28.478+0000 7f80fa000700 1 -- 172.21.15.7:0/3327916804 wait complete. 2024-06-13T07:01:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:29 smithi189 ceph-mon[36616]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 713 op/s 2024-06-13T07:01:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:29 smithi175 ceph-mon[36886]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 713 op/s 2024-06-13T07:01:29.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:29 smithi007 ceph-mon[32077]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 713 op/s 2024-06-13T07:01:31.569 INFO:tasks.cephfs.filesystem:scrub status for tag:4f2c68d0-417e-4a88-ac6c-2d256d7030fa - {'path': '/', 'tag': '4f2c68d0-417e-4a88-ac6c-2d256d7030fa', 'options': 'recursive,force'} 2024-06-13T07:01:31.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:31 smithi007 ceph-mon[32077]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 535 op/s 2024-06-13T07:01:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:31 smithi189 ceph-mon[36616]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 535 op/s 2024-06-13T07:01:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:31 smithi175 ceph-mon[36886]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.8 MiB/s wr, 535 op/s 2024-06-13T07:01:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 330.0 s, 837.8 tps, lat 119.295 ms stddev 144.257 2024-06-13T07:01:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:33 smithi189 ceph-mon[36616]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 611 op/s 2024-06-13T07:01:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:33 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:01:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:33 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:01:33.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:33 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:01:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:33 smithi175 ceph-mon[36886]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 611 op/s 2024-06-13T07:01:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:33 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:01:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:33 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:01:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:33 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:01:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:33 smithi007 ceph-mon[32077]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 611 op/s 2024-06-13T07:01:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:33 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:01:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:33 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:01:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:33 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:01:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:35 smithi189 ceph-mon[36616]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-06-13T07:01:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:35 smithi175 ceph-mon[36886]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-06-13T07:01:35.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:35 smithi007 ceph-mon[32077]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-06-13T07:01:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 -- 172.21.15.7:0/1014739914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a01035b0 msgr2=0x7f50a0109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1014739914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a01035b0 0x7f50a0109e40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5098009be0 tx=0x7f509802a6e0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 -- 172.21.15.7:0/1014739914 shutdown_connections 2024-06-13T07:01:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1014739914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a01035b0 0x7f50a0109e40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1014739914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0102bf0 0x7f50a0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1014739914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a0101a00 0x7f50a0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.109+0000 7f50a7e00640 1 -- 172.21.15.7:0/1014739914 >> 172.21.15.7:0/1014739914 conn(0x7f50a00fcf90 msgr2=0x7f50a00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.111+0000 7f50a7e00640 1 -- 172.21.15.7:0/1014739914 shutdown_connections 2024-06-13T07:01:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.112+0000 7f50a7e00640 1 -- 172.21.15.7:0/1014739914 wait complete. 2024-06-13T07:01:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.113+0000 7f50a7e00640 1 Processor -- start 2024-06-13T07:01:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.113+0000 7f50a7e00640 1 -- start start 2024-06-13T07:01:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 0x7f50a019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 0x7f50a01a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010d680 con 0x7f50a0101a00 2024-06-13T07:01:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010d500 con 0x7f50a0102bf0 2024-06-13T07:01:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.114+0000 7f50a7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010d800 con 0x7f50a01035b0 2024-06-13T07:01:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.115+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 0x7f50a01a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.115+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.115+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49006/0 (socket says 172.21.15.7:49006) 2024-06-13T07:01:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.115+0000 7f50a4a00640 1 -- 172.21.15.7:0/800781746 learned_addr learned my addr 172.21.15.7:0/800781746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.115+0000 7f50a5400640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 0x7f50a019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.117+0000 7f50a5400640 1 -- 172.21.15.7:0/800781746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 msgr2=0x7f50a01a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.117+0000 7f50a5400640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 0x7f50a01a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.117+0000 7f50a5400640 1 -- 172.21.15.7:0/800781746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 msgr2=0x7f50a019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.117+0000 7f50a5400640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.117+0000 7f50a5400640 1 -- 172.21.15.7:0/800781746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a01a5e30 con 0x7f50a0101a00 2024-06-13T07:01:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.118+0000 7f50a5400640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 0x7f50a019e210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50a0102a50 tx=0x7f509400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.118+0000 7f50a4a00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.118+0000 7f50a5e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 0x7f50a01a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:01:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.119+0000 7f509e000640 1 -- 172.21.15.7:0/800781746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094017070 con 0x7f50a0101a00 2024-06-13T07:01:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.120+0000 7f509e000640 1 -- 172.21.15.7:0/800781746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5094010460 con 0x7f50a0101a00 2024-06-13T07:01:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.120+0000 7f509e000640 1 -- 172.21.15.7:0/800781746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509400f620 con 0x7f50a0101a00 2024-06-13T07:01:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.120+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a01a6120 con 0x7f50a0101a00 2024-06-13T07:01:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.120+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a01a6620 con 0x7f50a0101a00 2024-06-13T07:01:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.124+0000 7f50aac31740 1 Processor -- start 2024-06-13T07:01:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.125+0000 7f50aac31740 1 -- start start 2024-06-13T07:01:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.125+0000 7f50aac31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d3f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50aac31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d3f5f0 0x561793d440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50aac31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50aac31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561793ca2420 con 0x561793d3ec00 2024-06-13T07:01:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50aac31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561793c49c50 con 0x561793d44760 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50aac31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561793c9e3e0 con 0x561793d3f5f0 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49018/0 (socket says 172.21.15.7:49018) 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50a4a00640 1 -- 172.21.15.7:0/4184850295 learned_addr learned my addr 172.21.15.7:0/4184850295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.126+0000 7f50a5e00640 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d3f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 -- 172.21.15.7:0/4184850295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d3f5f0 msgr2=0x561793d440f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d3f5f0 0x561793d440f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 -- 172.21.15.7:0/4184850295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 msgr2=0x561793d3f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d3f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 -- 172.21.15.7:0/4184850295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617939b6210 con 0x561793d44760 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.127+0000 7f50a4a00640 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f508c0115f0 tx=0x7f508c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4757c322897e42 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.128+0000 7f5092600640 1 -- 172.21.15.7:0/4184850295 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c005020 con 0x561793d44760 2024-06-13T07:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.128+0000 7f5092600640 1 -- 172.21.15.7:0/4184850295 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508c0047a0 con 0x561793d44760 2024-06-13T07:01:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.128+0000 7f5092600640 1 -- 172.21.15.7:0/4184850295 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c00fac0 con 0x561793d44760 2024-06-13T07:01:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.129+0000 7f50aac31740 1 -- 172.21.15.7:0/4184850295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 msgr2=0x561793d46bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.129+0000 7f50aac31740 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f508c0115f0 tx=0x7f508c009f90 comp rx=0 tx=0).stop 2024-06-13T07:01:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 -- 172.21.15.7:0/4184850295 shutdown_connections 2024-06-13T07:01:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d46bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d3f5f0 0x561793d440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 --2- 172.21.15.7:0/4184850295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d3f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 -- 172.21.15.7:0/4184850295 >> 172.21.15.7:0/4184850295 conn(0x561793d06fb0 msgr2=0x561793d3a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f509e000640 1 -- 172.21.15.7:0/800781746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f509401a070 con 0x7f50a0101a00 2024-06-13T07:01:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.130+0000 7f50aac31740 1 -- 172.21.15.7:0/4184850295 shutdown_connections 2024-06-13T07:01:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.133+0000 7f509e000640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 0x7f50880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.133+0000 7f509e000640 1 -- 172.21.15.7:0/800781746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f50940cec10 con 0x7f50a0101a00 2024-06-13T07:01:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.133+0000 7f50a4a00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 0x7f50880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.136+0000 7f50a4a00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 0x7f50880a5240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f508c011f40 tx=0x7f508c004f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.136+0000 7f50aac31740 1 -- 172.21.15.7:0/4184850295 wait complete. 2024-06-13T07:01:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.136+0000 7f50aac31740 1 -- start start 2024-06-13T07:01:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.136+0000 7f50aac31740 1 Processor -- start 2024-06-13T07:01:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d44760 0x561793d49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d4e570 0x561793d50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561793d47710 con 0x561793d4e570 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561793c5d0b0 con 0x561793d3ec00 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.137+0000 7f50aac31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561793d47890 con 0x561793d44760 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50aac31740 10 client.0 _ll_register_callbacks cb 0x561793c48f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49030/0 (socket says 172.21.15.7:49030) 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 -- 172.21.15.7:0/2485443922 learned_addr learned my addr 172.21.15.7:0/2485443922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 -- 172.21.15.7:0/2485443922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d44760 msgr2=0x561793d49540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d44760 0x561793d49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.138+0000 7f50a5400640 1 -- 172.21.15.7:0/2485443922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d4e570 msgr2=0x561793d50960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.139+0000 7f50a5400640 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d4e570 0x561793d50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.139+0000 7f50a5400640 1 -- 172.21.15.7:0/2485443922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561793a82620 con 0x561793d3ec00 2024-06-13T07:01:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.139+0000 7f50a5400640 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5094010fd0 tx=0x7f509409b0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=598343bd8b9cefb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.141+0000 7f5086a00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:01:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.141+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094021070 con 0x561793d3ec00 2024-06-13T07:01:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.141+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f509409f160 con 0x561793d3ec00 2024-06-13T07:01:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.143+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508c0047a0 con 0x561793d3ec00 2024-06-13T07:01:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.143+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5094010610 con 0x561793d3ec00 2024-06-13T07:01:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.143+0000 7f50aac31740 10 client.26234 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561793d53d40 con 0x561793d3ec00 2024-06-13T07:01:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508c0047a0 con 0x561793d3ec00 2024-06-13T07:01:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f50940a3940 con 0x561793d3ec00 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 10 client.26234 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 20 client.26234 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793d558c0 0x561793d57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 4 client.26234 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.144+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561793cafb50 con 0x561793d558c0 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.145+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793d558c0 0x561793d57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.146+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793d558c0 0x561793d57d80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626a9781fe58fec6 server_cookie=f132724b09153439 in_seq=0 out_seq=0 2024-06-13T07:01:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.146+0000 7f5086a00640 10 client.26234 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f5086a00640 1 -- 172.21.15.7:0/2485443922 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561793cafb50 con 0x561793d558c0 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f5086a00640 10 client.26234 handle_command_reply: tid=0 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 client.26234 shutdown 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 msgr2=0x561793d49000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5094010fd0 tx=0x7f509409b0d0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 shutdown_connections 2024-06-13T07:01:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793d558c0 0x561793d57d80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d4e570 0x561793d50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d44760 0x561793d49540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 --2- 172.21.15.7:0/2485443922 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d49000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 >> 172.21.15.7:0/2485443922 conn(0x561793d06fb0 msgr2=0x561793d39840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.147+0000 7f5086a00640 0 client.26234 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.148+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 shutdown_connections 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.148+0000 7f50aac31740 1 -- 172.21.15.7:0/2485443922 wait complete. 2024-06-13T07:01:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.148+0000 7f50aac31740 20 client.26234 trim_cache size 0 max 16384 2024-06-13T07:01:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.182+0000 7f50aac31740 1 Processor -- start 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.183+0000 7f50aac31740 1 -- start start 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.183+0000 7f50aac31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d48db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.183+0000 7f50aac31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.183+0000 7f50aac31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d3d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.183+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49034/0 (socket says 172.21.15.7:49034) 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50aac31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5080012430 con 0x561793d3ec00 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50aac31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f508000fca0 con 0x561793d44760 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50aac31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f508000daf0 con 0x561793d4e570 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50a4a00640 1 -- 172.21.15.7:0/2606360257 learned_addr learned my addr 172.21.15.7:0/2606360257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.184+0000 7f50a5e00640 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d3d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.185+0000 7f50a4a00640 1 -- 172.21.15.7:0/2606360257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 msgr2=0x561793d3d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.185+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d3d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.185+0000 7f50a4a00640 1 -- 172.21.15.7:0/2606360257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 msgr2=0x561793d48db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.185+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d48db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.185+0000 7f50a4a00640 1 -- 172.21.15.7:0/2606360257 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561793d3b330 con 0x561793d44760 2024-06-13T07:01:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.186+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d40 tx=0x7f508c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f4cadd5a930c50 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.186+0000 7f5091c00640 1 -- 172.21.15.7:0/2606360257 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c005560 con 0x561793d44760 2024-06-13T07:01:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.187+0000 7f5091c00640 1 -- 172.21.15.7:0/2606360257 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508c029040 con 0x561793d44760 2024-06-13T07:01:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.187+0000 7f5091c00640 1 -- 172.21.15.7:0/2606360257 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c003850 con 0x561793d44760 2024-06-13T07:01:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.189+0000 7f50aac31740 1 -- 172.21.15.7:0/2606360257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 msgr2=0x561793d492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.189+0000 7f50aac31740 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d40 tx=0x7f508c009f90 comp rx=0 tx=0).stop 2024-06-13T07:01:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 -- 172.21.15.7:0/2606360257 shutdown_connections 2024-06-13T07:01:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d3d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d44760 0x561793d492f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 --2- 172.21.15.7:0/2606360257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d3ec00 0x561793d48db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 -- 172.21.15.7:0/2606360257 >> 172.21.15.7:0/2606360257 conn(0x561793d06fb0 msgr2=0x561793d40910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 -- 172.21.15.7:0/2606360257 shutdown_connections 2024-06-13T07:01:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.190+0000 7f50aac31740 1 -- 172.21.15.7:0/2606360257 wait complete. 2024-06-13T07:01:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.191+0000 7f50aac31740 1 -- start start 2024-06-13T07:01:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 Processor -- start 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 0x561793d675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f508000e160 con 0x561793d44760 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f508000f5d0 con 0x561793d3ec00 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f508000e830 con 0x561793d4e570 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.192+0000 7f50aac31740 10 client.0 _ll_register_callbacks cb 0x7f5094097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.193+0000 7f50a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.193+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 0x561793d675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.193+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.193+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33080/0 (socket says 172.21.15.7:33080) 2024-06-13T07:01:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.193+0000 7f50a5e00640 1 -- 172.21.15.7:0/2508991263 learned_addr learned my addr 172.21.15.7:0/2508991263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.194+0000 7f50a5e00640 1 -- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 msgr2=0x561793d67090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.194+0000 7f50a5e00640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d67090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.194+0000 7f50a5e00640 1 -- 172.21.15.7:0/2508991263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 msgr2=0x561793d675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.194+0000 7f50a5e00640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 0x561793d675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.194+0000 7f50a5e00640 1 -- 172.21.15.7:0/2508991263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561793d68b10 con 0x561793d4e570 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.195+0000 7f50a5400640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d67090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.195+0000 7f50a4a00640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 0x561793d675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.195+0000 7f50a5e00640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5098009be0 tx=0x7f5098009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aedeaa61fd9026e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.195+0000 7f5086a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098036040 con 0x561793d4e570 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508c029040 con 0x561793d4e570 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5098031440 con 0x561793d4e570 2024-06-13T07:01:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5098030e90 con 0x561793d4e570 2024-06-13T07:01:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f50aac31740 10 client.26206 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.196+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561793d69ee0 con 0x561793d4e570 2024-06-13T07:01:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.197+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508c029040 con 0x561793d4e570 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f5098048cc0 con 0x561793d4e570 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 10 client.26206 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 20 client.26206 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793dd1b40 0x561793dd3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 4 client.26206 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.198+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561793ca9ea0 con 0x561793dd1b40 2024-06-13T07:01:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.199+0000 7f50a5400640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793dd1b40 0x561793dd3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.199+0000 7f50a5400640 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793dd1b40 0x561793dd3f60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525482920b3fcf40 server_cookie=cf3e9430edef4e3c in_seq=0 out_seq=0 2024-06-13T07:01:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.200+0000 7f5086a00640 10 client.26206 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f5086a00640 1 -- 172.21.15.7:0/2508991263 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561793ca9ea0 con 0x561793dd1b40 2024-06-13T07:01:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f5086a00640 10 client.26206 handle_command_reply: tid=0 2024-06-13T07:01:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 client.26206 shutdown 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 msgr2=0x561793d68520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5098009be0 tx=0x7f5098009800 comp rx=0 tx=0).stop 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 shutdown_connections 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561793dd1b40 0x561793dd3f60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561793d4e570 0x561793d68520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561793d44760 0x561793d675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 --2- 172.21.15.7:0/2508991263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561793d3ec00 0x561793d67090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f5086a00640 0 client.26206 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.294+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 >> 172.21.15.7:0/2508991263 conn(0x561793d06fb0 msgr2=0x561793d40910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.296+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 shutdown_connections 2024-06-13T07:01:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.296+0000 7f50aac31740 1 -- 172.21.15.7:0/2508991263 wait complete. 2024-06-13T07:01:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.297+0000 7f50aac31740 20 client.26206 trim_cache size 0 max 16384 2024-06-13T07:01:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:01:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:01:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.299+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 msgr2=0x7f50880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.299+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 0x7f50880a5240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f508c011f40 tx=0x7f508c004f50 comp rx=0 tx=0).stop 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.299+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 msgr2=0x7f50a019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.299+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 0x7f50a019e210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50a0102a50 tx=0x7f509400dbb0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 shutdown_connections 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f50880a2d80 0x7f50880a5240 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50a01035b0 0x7f50a01a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a0102bf0 0x7f50a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 --2- 172.21.15.7:0/800781746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a0101a00 0x7f50a019e210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 >> 172.21.15.7:0/800781746 conn(0x7f50a00fcf90 msgr2=0x7f50a0103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 shutdown_connections 2024-06-13T07:01:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:36.300+0000 7f50a7e00640 1 -- 172.21.15.7:0/800781746 wait complete. 2024-06-13T07:01:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:37 smithi189 ceph-mon[36616]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-06-13T07:01:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:37 smithi175 ceph-mon[36886]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-06-13T07:01:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 335.0 s, 649.0 tps, lat 156.636 ms stddev 211.888 2024-06-13T07:01:37.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:37 smithi007 ceph-mon[32077]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-06-13T07:01:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:39 smithi175 ceph-mon[36886]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 734 op/s 2024-06-13T07:01:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:39 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:39.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:39 smithi007 ceph-mon[32077]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 734 op/s 2024-06-13T07:01:39.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:39 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:39 smithi189 ceph-mon[36616]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 734 op/s 2024-06-13T07:01:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:39 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:01:40.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:40.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:40.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:40 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:01:40.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:40.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:40 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:01:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:01:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:01:41.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:40 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:01:41.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:41 smithi007 ceph-mon[32077]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 519 op/s 2024-06-13T07:01:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:41 smithi189 ceph-mon[36616]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 519 op/s 2024-06-13T07:01:42.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:41 smithi175 ceph-mon[36886]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 519 op/s 2024-06-13T07:01:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 340.0 s, 694.8 tps, lat 145.439 ms stddev 197.686 2024-06-13T07:01:43.888 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:43 smithi189 ceph-mon[36616]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 577 op/s 2024-06-13T07:01:43.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:43 smithi007 ceph-mon[32077]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 577 op/s 2024-06-13T07:01:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:43 smithi175 ceph-mon[36886]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 577 op/s 2024-06-13T07:01:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:45 smithi007 ceph-mon[32077]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 650 op/s 2024-06-13T07:01:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:45 smithi189 ceph-mon[36616]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 650 op/s 2024-06-13T07:01:46.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:45 smithi175 ceph-mon[36886]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 650 op/s 2024-06-13T07:01:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 345.0 s, 701.5 tps, lat 140.172 ms stddev 186.998 2024-06-13T07:01:47.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:47 smithi007 ceph-mon[32077]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 542 op/s 2024-06-13T07:01:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:47 smithi189 ceph-mon[36616]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 542 op/s 2024-06-13T07:01:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:47 smithi175 ceph-mon[36886]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 542 op/s 2024-06-13T07:01:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.502+0000 7fe8eca00640 1 -- 172.21.15.7:0/3084165378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e8074400 msgr2=0x7fe8e8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.502+0000 7fe8eca00640 1 --2- 172.21.15.7:0/3084165378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e8074400 0x7fe8e8074880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009930 tx=0x7fe8d402a440 comp rx=0 tx=0).stop 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 -- 172.21.15.7:0/3084165378 shutdown_connections 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 --2- 172.21.15.7:0/3084165378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e810dff0 0x7fe8e81103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 --2- 172.21.15.7:0/3084165378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e8074400 0x7fe8e8074880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 --2- 172.21.15.7:0/3084165378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 0x7fe8e8073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 -- 172.21.15.7:0/3084165378 >> 172.21.15.7:0/3084165378 conn(0x7fe8e80fcf90 msgr2=0x7fe8e80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 -- 172.21.15.7:0/3084165378 shutdown_connections 2024-06-13T07:01:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.503+0000 7fe8eca00640 1 -- 172.21.15.7:0/3084165378 wait complete. 2024-06-13T07:01:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.504+0000 7fe8eca00640 1 Processor -- start 2024-06-13T07:01:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.504+0000 7fe8eca00640 1 -- start start 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.504+0000 7fe8eca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8eca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 0x7fe8e819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8eca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8eca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8113cb0 con 0x7fe8e80759e0 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8eca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8113b30 con 0x7fe8e810dff0 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8eca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8113e30 con 0x7fe8e8074400 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 0x7fe8e819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:46440/0 (socket says 172.21.15.7:46440) 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51892/0 (socket says 172.21.15.7:51892) 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 -- 172.21.15.7:0/1889013552 learned_addr learned my addr 172.21.15.7:0/1889013552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 -- 172.21.15.7:0/1889013552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 msgr2=0x7fe8e819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 -- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 msgr2=0x7fe8e819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 0x7fe8e819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.505+0000 7fe8e6800640 1 -- 172.21.15.7:0/1889013552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e81a5e40 con 0x7fe8e810dff0 2024-06-13T07:01:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:01:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000eb10 tx=0x7fe8e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8dea00640 1 -- 172.21.15.7:0/1889013552 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000ce20 con 0x7fe8e810dff0 2024-06-13T07:01:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e81a6130 con 0x7fe8e810dff0 2024-06-13T07:01:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8dea00640 1 -- 172.21.15.7:0/1889013552 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0005020 con 0x7fe8e810dff0 2024-06-13T07:01:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8dea00640 1 -- 172.21.15.7:0/1889013552 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0010690 con 0x7fe8e810dff0 2024-06-13T07:01:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.506+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81a66a0 con 0x7fe8e810dff0 2024-06-13T07:01:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.511+0000 7fe8ef877740 1 Processor -- start 2024-06-13T07:01:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.511+0000 7fe8dea00640 1 -- 172.21.15.7:0/1889013552 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe8e00108f0 con 0x7fe8e810dff0 2024-06-13T07:01:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 -- start start 2024-06-13T07:01:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e46fc00 0x55b21e470020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e475760 0x55b21e477bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b21e3d3420 con 0x55b21e4705f0 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b21e37ac50 con 0x55b21e475760 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.512+0000 7fe8ef877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b21e3cf3e0 con 0x55b21e46fc00 2024-06-13T07:01:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36418/0 (socket says 172.21.15.7:36418) 2024-06-13T07:01:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 -- 172.21.15.7:0/2766123699 learned_addr learned my addr 172.21.15.7:0/2766123699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 -- 172.21.15.7:0/2766123699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e46fc00 msgr2=0x55b21e470020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e46fc00 0x55b21e470020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 -- 172.21.15.7:0/2766123699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e475760 msgr2=0x55b21e477bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e475760 0x55b21e477bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.513+0000 7fe8e5e00640 1 -- 172.21.15.7:0/2766123699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21e0e7210 con 0x55b21e4705f0 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8dea00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 0x7fe8cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800f4c0 tx=0x7fe8d800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1c298fc76a0b5f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8e5400640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 0x7fe8cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8dea00640 1 -- 172.21.15.7:0/1889013552 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe8e00cf880 con 0x7fe8e810dff0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8d3000640 1 -- 172.21.15.7:0/2766123699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8005220 con 0x55b21e4705f0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8d3000640 1 -- 172.21.15.7:0/2766123699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8004c80 con 0x55b21e4705f0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.514+0000 7fe8d3000640 1 -- 172.21.15.7:0/2766123699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8010b60 con 0x55b21e4705f0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 -- 172.21.15.7:0/2766123699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 msgr2=0x55b21e4750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800f4c0 tx=0x7fe8d800f980 comp rx=0 tx=0).stop 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8e5400640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 0x7fe8cc0a51f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4005ab0 tx=0x7fe8d4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 -- 172.21.15.7:0/2766123699 shutdown_connections 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e475760 0x55b21e477bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e4705f0 0x55b21e4750f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 --2- 172.21.15.7:0/2766123699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e46fc00 0x55b21e470020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.515+0000 7fe8ef877740 1 -- 172.21.15.7:0/2766123699 >> 172.21.15.7:0/2766123699 conn(0x55b21e437fb0 msgr2=0x55b21e46b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.516+0000 7fe8ef877740 1 -- 172.21.15.7:0/2766123699 shutdown_connections 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.516+0000 7fe8ef877740 1 -- 172.21.15.7:0/2766123699 wait complete. 2024-06-13T07:01:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.516+0000 7fe8ef877740 1 -- start start 2024-06-13T07:01:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.516+0000 7fe8ef877740 1 Processor -- start 2024-06-13T07:01:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e48af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e48b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b21e489720 con 0x55b21e475760 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b21e38e0b0 con 0x55b21e46fc00 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b21e4898a0 con 0x55b21e4705f0 2024-06-13T07:01:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.517+0000 7fe8ef877740 10 client.0 _ll_register_callbacks cb 0x55b21e379f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.518+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e48af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.518+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.518+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36420/0 (socket says 172.21.15.7:36420) 2024-06-13T07:01:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.518+0000 7fe8e6800640 1 -- 172.21.15.7:0/1127374917 learned_addr learned my addr 172.21.15.7:0/1127374917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.518+0000 7fe8e5400640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e48b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e6800640 1 -- 172.21.15.7:0/1127374917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 msgr2=0x55b21e48b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e48b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e6800640 1 -- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 msgr2=0x55b21e48af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e48af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e6800640 1 -- 172.21.15.7:0/1127374917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21e1b3620 con 0x55b21e475760 2024-06-13T07:01:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.519+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e48af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:01:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.520+0000 7fe8e6800640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0098710 tx=0x7fe8e0005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35b610c1b15f6a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.520+0000 7fe8c7400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:01:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.520+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0010de0 con 0x55b21e475760 2024-06-13T07:01:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.520+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe8e00ba530 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.521+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8004c80 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.522+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8e00a00f0 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.522+0000 7fe8ef877740 10 client.15798 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.522+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b21e4a6910 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.522+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8004c80 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe8e00a4520 con 0x55b21e475760 2024-06-13T07:01:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 10 client.15798 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 20 client.15798 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e4a8030 0x55b21e4aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 4 client.15798 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b21e3e0b50 con 0x55b21e4a8030 2024-06-13T07:01:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.523+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e4a8030 0x55b21e4aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.524+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e4a8030 0x55b21e4aa440 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df1f9061a60c029 server_cookie=c6d23c3c8d9bbb6b in_seq=0 out_seq=0 2024-06-13T07:01:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.524+0000 7fe8c7400640 10 client.15798 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.524+0000 7fe8c7400640 1 -- 172.21.15.7:0/1127374917 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b21e3e0b50 con 0x55b21e4a8030 2024-06-13T07:01:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.524+0000 7fe8c7400640 10 client.15798 handle_command_reply: tid=0 2024-06-13T07:01:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.525+0000 7fe8ef877740 1 client.15798 shutdown 2024-06-13T07:01:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.530+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 msgr2=0x55b21e4924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.530+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0098710 tx=0x7fe8e0005310 comp rx=0 tx=0).stop 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 shutdown_connections 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e4a8030 0x55b21e4aa440 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e475760 0x55b21e4924c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e48b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1127374917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e48af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8c7400640 0 client.15798 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.531+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 >> 172.21.15.7:0/1127374917 conn(0x55b21e437fb0 msgr2=0x55b21e46a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.532+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 shutdown_connections 2024-06-13T07:01:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.532+0000 7fe8ef877740 1 -- 172.21.15.7:0/1127374917 wait complete. 2024-06-13T07:01:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.532+0000 7fe8ef877740 20 client.15798 trim_cache size 0 max 16384 2024-06-13T07:01:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.570+0000 7fe8ef877740 1 Processor -- start 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 -- start start 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 0x55b21e46b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e46fc00 0x55b21e492480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e4afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e806f6a0 con 0x55b21e46fc00 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e806cf50 con 0x55b21e437fb0 2024-06-13T07:01:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.571+0000 7fe8ef877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e810d990 con 0x55b21e4705f0 2024-06-13T07:01:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.572+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e4afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.572+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e4afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51928/0 (socket says 172.21.15.7:51928) 2024-06-13T07:01:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.572+0000 7fe8e6800640 1 -- 172.21.15.7:0/1896600006 learned_addr learned my addr 172.21.15.7:0/1896600006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:49.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.572+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 0x55b21e46b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5400640 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e46fc00 0x55b21e492480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5e00640 1 -- 172.21.15.7:0/1896600006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 msgr2=0x55b21e4afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e4afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5e00640 1 -- 172.21.15.7:0/1896600006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e46fc00 msgr2=0x55b21e492480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e46fc00 0x55b21e492480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.575+0000 7fe8e5e00640 1 -- 172.21.15.7:0/1896600006 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21e474be0 con 0x55b21e437fb0 2024-06-13T07:01:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.576+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 0x55b21e46b870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55b21e46c530 tx=0x7fe8d800feb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144a1dbfcaa62ff7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.576+0000 7fe8d2600640 1 -- 172.21.15.7:0/1896600006 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8010760 con 0x55b21e437fb0 2024-06-13T07:01:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.576+0000 7fe8d2600640 1 -- 172.21.15.7:0/1896600006 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d802f070 con 0x55b21e437fb0 2024-06-13T07:01:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.576+0000 7fe8d2600640 1 -- 172.21.15.7:0/1896600006 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8018c40 con 0x55b21e437fb0 2024-06-13T07:01:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.577+0000 7fe8ef877740 1 -- 172.21.15.7:0/1896600006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 msgr2=0x55b21e46b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.577+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 0x55b21e46b870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55b21e46c530 tx=0x7fe8d800feb0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 -- 172.21.15.7:0/1896600006 shutdown_connections 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e4afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e46fc00 0x55b21e492480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 --2- 172.21.15.7:0/1896600006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e437fb0 0x55b21e46b870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 -- 172.21.15.7:0/1896600006 >> 172.21.15.7:0/1896600006 conn(0x55b21e48ab00 msgr2=0x55b21e46b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 -- 172.21.15.7:0/1896600006 shutdown_connections 2024-06-13T07:01:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.578+0000 7fe8ef877740 1 -- 172.21.15.7:0/1896600006 wait complete. 2024-06-13T07:01:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.579+0000 7fe8ef877740 1 -- start start 2024-06-13T07:01:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.579+0000 7fe8ef877740 1 Processor -- start 2024-06-13T07:01:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e4904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e491450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e806b410 con 0x55b21e437fb0 2024-06-13T07:01:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e806c880 con 0x55b21e46fc00 2024-06-13T07:01:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8ef877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e806bae0 con 0x55b21e4705f0 2024-06-13T07:01:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36428/0 (socket says 172.21.15.7:36428) 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.580+0000 7fe8e5e00640 1 -- 172.21.15.7:0/3169514381 learned_addr learned my addr 172.21.15.7:0/3169514381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8ef877740 10 client.0 _ll_register_callbacks cb 0x7fe8e009b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8e5e00640 1 -- 172.21.15.7:0/3169514381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 msgr2=0x55b21e491450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e491450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8e5e00640 1 -- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 msgr2=0x55b21e4904b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e4904b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.581+0000 7fe8e5e00640 1 -- 172.21.15.7:0/3169514381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21e491a40 con 0x55b21e437fb0 2024-06-13T07:01:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.582+0000 7fe8c7400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:01:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.582+0000 7fe8e5e00640 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800ae50 tx=0x7fe8d800a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc317805b6bb988f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:01:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.582+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d803a060 con 0x55b21e437fb0 2024-06-13T07:01:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.582+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe8d8018b50 con 0x55b21e437fb0 2024-06-13T07:01:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.583+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d802f070 con 0x55b21e437fb0 2024-06-13T07:01:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.583+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8d8018d50 con 0x55b21e437fb0 2024-06-13T07:01:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.583+0000 7fe8ef877740 10 client.15801 fetch_fsmap learned FSMap version 34 2024-06-13T07:01:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.583+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b21e5130b0 con 0x55b21e437fb0 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d802f070 con 0x55b21e437fb0 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe8d80358c0 con 0x55b21e437fb0 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 10 client.15801 fetch_fsmap finished waiting for FSMap version 34 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 20 client.15801 populate_metadata read hostname 'smithi007' 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e475760 0x55b21e515860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 4 client.15801 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.586+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b21e3daea0 con 0x55b21e475760 2024-06-13T07:01:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.587+0000 7fe8e5400640 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e475760 0x55b21e515860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:01:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.589+0000 7fe8e5400640 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e475760 0x55b21e515860 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23ed5980d79398d server_cookie=81be8fc104a7a16b in_seq=0 out_seq=0 2024-06-13T07:01:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.589+0000 7fe8c7400640 10 client.15801 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.721+0000 7fe8c7400640 1 -- 172.21.15.7:0/3169514381 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b21e3daea0 con 0x55b21e475760 2024-06-13T07:01:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.721+0000 7fe8c7400640 10 client.15801 handle_command_reply: tid=0 2024-06-13T07:01:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.722+0000 7fe8ef877740 1 client.15801 shutdown 2024-06-13T07:01:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.722+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 msgr2=0x55b21e48ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.722+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800ae50 tx=0x7fe8d800a550 comp rx=0 tx=0).stop 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.722+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 shutdown_connections 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.722+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b21e475760 0x55b21e515860 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b21e4705f0 0x55b21e491450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b21e46fc00 0x55b21e4904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 --2- 172.21.15.7:0/3169514381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b21e437fb0 0x55b21e48ff70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 >> 172.21.15.7:0/3169514381 conn(0x55b21e48ab00 msgr2=0x55b21e46e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8c7400640 0 client.15801 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 shutdown_connections 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 1 -- 172.21.15.7:0/3169514381 wait complete. 2024-06-13T07:01:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.723+0000 7fe8ef877740 20 client.15801 trim_cache size 0 max 16384 2024-06-13T07:01:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:01:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:01:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:01:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 msgr2=0x7fe8cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 0x7fe8cc0a51f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4005ab0 tx=0x7fe8d4005d50 comp rx=0 tx=0).stop 2024-06-13T07:01:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 msgr2=0x7fe8e81a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:01:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000eb10 tx=0x7fe8e000ef10 comp rx=0 tx=0).stop 2024-06-13T07:01:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 shutdown_connections 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe8cc0a2d30 0x7fe8cc0a51f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e810dff0 0x7fe8e81a57d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8e80759e0 0x7fe8e819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 --2- 172.21.15.7:0/1889013552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8e8074400 0x7fe8e819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.725+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 >> 172.21.15.7:0/1889013552 conn(0x7fe8e80fcf90 msgr2=0x7fe8e8101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.726+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 shutdown_connections 2024-06-13T07:01:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:01:49.726+0000 7fe8eca00640 1 -- 172.21.15.7:0/1889013552 wait complete. 2024-06-13T07:01:49.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:49 smithi007 ceph-mon[32077]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 697 op/s 2024-06-13T07:01:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:49 smithi189 ceph-mon[36616]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 697 op/s 2024-06-13T07:01:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:49 smithi175 ceph-mon[36886]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 697 op/s 2024-06-13T07:01:51.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:51 smithi007 ceph-mon[32077]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 514 op/s 2024-06-13T07:01:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:51 smithi189 ceph-mon[36616]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 514 op/s 2024-06-13T07:01:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:51 smithi175 ceph-mon[36886]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 514 op/s 2024-06-13T07:01:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 350.0 s, 602.5 tps, lat 162.990 ms stddev 225.996 2024-06-13T07:01:53.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:53 smithi007 ceph-mon[32077]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-13T07:01:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:53 smithi189 ceph-mon[36616]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-13T07:01:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:53 smithi175 ceph-mon[36886]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 534 op/s 2024-06-13T07:01:56.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:55 smithi189 ceph-mon[36616]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 625 op/s 2024-06-13T07:01:56.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:55 smithi175 ceph-mon[36886]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 625 op/s 2024-06-13T07:01:56.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:55 smithi007 ceph-mon[32077]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 625 op/s 2024-06-13T07:01:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 355.0 s, 695.4 tps, lat 147.994 ms stddev 184.374 2024-06-13T07:01:58.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:57 smithi189 ceph-mon[36616]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 486 op/s 2024-06-13T07:01:58.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:57 smithi175 ceph-mon[36886]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 486 op/s 2024-06-13T07:01:58.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:57 smithi007 ceph-mon[32077]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 486 op/s 2024-06-13T07:01:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:01:58 smithi189 ceph-mon[36616]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-06-13T07:01:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:01:58 smithi175 ceph-mon[36886]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-06-13T07:01:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:01:58 smithi007 ceph-mon[32077]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 635 op/s 2024-06-13T07:02:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:00 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:00 smithi189 ceph-mon[36616]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 466 op/s 2024-06-13T07:02:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:00 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:00 smithi175 ceph-mon[36886]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 466 op/s 2024-06-13T07:02:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:00 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:01.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:00 smithi007 ceph-mon[32077]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 466 op/s 2024-06-13T07:02:01.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:02:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:01 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:01 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:02.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:01 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:02.292 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 360.0 s, 767.6 tps, lat 129.547 ms stddev 150.852 2024-06-13T07:02:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 -- 172.21.15.7:0/1067626452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 msgr2=0x7fc634111220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 --2- 172.21.15.7:0/1067626452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc634111220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009be0 tx=0x7fc62c02a6c0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 -- 172.21.15.7:0/1067626452 shutdown_connections 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 --2- 172.21.15.7:0/1067626452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc634111220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 --2- 172.21.15.7:0/1067626452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63410c3e0 0x7fc63410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 --2- 172.21.15.7:0/1067626452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc634109ab0 0x7fc63410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 -- 172.21.15.7:0/1067626452 >> 172.21.15.7:0/1067626452 conn(0x7fc63406ea10 msgr2=0x7fc634070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 -- 172.21.15.7:0/1067626452 shutdown_connections 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.932+0000 7fc639e00640 1 -- 172.21.15.7:0/1067626452 wait complete. 2024-06-13T07:02:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.933+0000 7fc639e00640 1 Processor -- start 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.933+0000 7fc639e00640 1 -- start start 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109ab0 0x7fc6341a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 0x7fc6341a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341180f0 con 0x7fc634109ab0 2024-06-13T07:02:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc634117f70 con 0x7fc63410c3e0 2024-06-13T07:02:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc639e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc634118270 con 0x7fc63410ed60 2024-06-13T07:02:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc633400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc62be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 0x7fc6341a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc633400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42196/0 (socket says 172.21.15.7:42196) 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc633400640 1 -- 172.21.15.7:0/3203605061 learned_addr learned my addr 172.21.15.7:0/3203605061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.934+0000 7fc632a00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109ab0 0x7fc6341a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 -- 172.21.15.7:0/3203605061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 msgr2=0x7fc6341a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 0x7fc6341a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 -- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109ab0 msgr2=0x7fc6341a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109ab0 0x7fc6341a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 -- 172.21.15.7:0/3203605061 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341aa270 con 0x7fc63410ed60 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc62be00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 0x7fc6341a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc633400640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009b30 tx=0x7fc62c030710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc638a00640 1 -- 172.21.15.7:0/3203605061 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c041020 con 0x7fc63410ed60 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6341aa500 con 0x7fc63410ed60 2024-06-13T07:02:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc638a00640 1 -- 172.21.15.7:0/3203605061 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c031920 con 0x7fc63410ed60 2024-06-13T07:02:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.935+0000 7fc638a00640 1 -- 172.21.15.7:0/3203605061 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c036040 con 0x7fc63410ed60 2024-06-13T07:02:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.936+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341aaa70 con 0x7fc63410ed60 2024-06-13T07:02:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.938+0000 7fc638a00640 1 -- 172.21.15.7:0/3203605061 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc62c044020 con 0x7fc63410ed60 2024-06-13T07:02:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.940+0000 7fc638a00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 0x7fc6180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.940+0000 7fc638a00640 1 -- 172.21.15.7:0/3203605061 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc62c0efe30 con 0x7fc63410ed60 2024-06-13T07:02:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.940+0000 7fc632a00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 0x7fc6180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 Processor -- start 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 -- start start 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 0x55d2a8ed3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8e26420 con 0x55d2a8ecbe00 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8dcdc50 con 0x55d2a8ecb410 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc63cc16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8e223e0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.941+0000 7fc632a00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 0x7fc6180a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0045f0 tx=0x7fc61c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc632a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc632a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56816/0 (socket says 172.21.15.7:56816) 2024-06-13T07:02:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc632a00640 1 -- 172.21.15.7:0/1959975953 learned_addr learned my addr 172.21.15.7:0/1959975953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 0x55d2a8ed3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc62be00640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 -- 172.21.15.7:0/1959975953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 msgr2=0x55d2a8ecb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 -- 172.21.15.7:0/1959975953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 msgr2=0x55d2a8ed0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 -- 172.21.15.7:0/1959975953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a8b3a210 con 0x55d2a8ed0f70 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc62be00640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc632a00640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.942+0000 7fc633400640 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 0x55d2a8ed3400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02abe0 tx=0x7fc62c0b8680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660b6556d4283f6b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.943+0000 7fc623e00640 1 -- 172.21.15.7:0/1959975953 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0bc9f0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.943+0000 7fc623e00640 1 -- 172.21.15.7:0/1959975953 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c0bc2f0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.943+0000 7fc623e00640 1 -- 172.21.15.7:0/1959975953 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0b8960 con 0x55d2a8ed0f70 2024-06-13T07:02:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 -- 172.21.15.7:0/1959975953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 msgr2=0x55d2a8ed3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 0x55d2a8ed3400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02abe0 tx=0x7fc62c0b8680 comp rx=0 tx=0).stop 2024-06-13T07:02:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 -- 172.21.15.7:0/1959975953 shutdown_connections 2024-06-13T07:02:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ed0f70 0x55d2a8ed3400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1959975953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ecb410 0x55d2a8ecb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 -- 172.21.15.7:0/1959975953 >> 172.21.15.7:0/1959975953 conn(0x55d2a8e8afb0 msgr2=0x55d2a8ec6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 -- 172.21.15.7:0/1959975953 shutdown_connections 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.944+0000 7fc63cc16740 1 -- 172.21.15.7:0/1959975953 wait complete. 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.945+0000 7fc63cc16740 1 -- start start 2024-06-13T07:02:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.945+0000 7fc63cc16740 1 Processor -- start 2024-06-13T07:02:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8ed3ce0 con 0x55d2a8ecbe00 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8de10b0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d2a8ed3e60 con 0x55d2a8ecb410 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc63cc16740 10 client.0 _ll_register_callbacks cb 0x55d2a8dccf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc632a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc633400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc632a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42216/0 (socket says 172.21.15.7:42216) 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc633400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56818/0 (socket says 172.21.15.7:56818) 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc632a00640 1 -- 172.21.15.7:0/320885081 learned_addr learned my addr 172.21.15.7:0/320885081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.946+0000 7fc62be00640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 -- 172.21.15.7:0/320885081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 msgr2=0x55d2a8ed54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 -- 172.21.15.7:0/320885081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 msgr2=0x55d2a8ed5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 -- 172.21.15.7:0/320885081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a8c06620 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc633400640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0c5120 tx=0x7fc62c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c1efbf6fbf0a66 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc620c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0c4f60 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62c0bc2f0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc62c0b8960 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc62c0bc9f0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc63cc16740 10 client.26264 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.947+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d2a8ed3ce0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c0bc2f0 con 0x55d2a8ed0f70 2024-06-13T07:02:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fc62c0bb740 con 0x55d2a8ed0f70 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 10 client.26264 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 20 client.26264 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8ee1230 0x55d2a8ee3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 4 client.26264 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a8e33b50 con 0x55d2a8ee1230 2024-06-13T07:02:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.948+0000 7fc632a00640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8ee1230 0x55d2a8ee3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.949+0000 7fc632a00640 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8ee1230 0x55d2a8ee3620 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33ceedc5dda5439 server_cookie=5aa07d0752605e3a in_seq=0 out_seq=0 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.949+0000 7fc620c00640 10 client.26264 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc620c00640 1 -- 172.21.15.7:0/320885081 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2a8e33b50 con 0x55d2a8ee1230 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc620c00640 10 client.26264 handle_command_reply: tid=0 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 client.26264 shutdown 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 msgr2=0x55d2a8edca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0c5120 tx=0x7fc62c02a6c0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 shutdown_connections 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8ee1230 0x55d2a8ee3620 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 --2- 172.21.15.7:0/320885081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 >> 172.21.15.7:0/320885081 conn(0x55d2a8e8afb0 msgr2=0x55d2a8ebd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.950+0000 7fc620c00640 0 client.26264 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.951+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 shutdown_connections 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.951+0000 7fc63cc16740 1 -- 172.21.15.7:0/320885081 wait complete. 2024-06-13T07:02:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.951+0000 7fc63cc16740 20 client.26264 trim_cache size 0 max 16384 2024-06-13T07:02:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 Processor -- start 2024-06-13T07:02:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 -- start start 2024-06-13T07:02:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b33e0 con 0x55d2a8ecb410 2024-06-13T07:02:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b0c50 con 0x55d2a8ed0f70 2024-06-13T07:02:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc63cc16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341aeaa0 con 0x55d2a8ecbe00 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc62be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc632a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc62be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42224/0 (socket says 172.21.15.7:42224) 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc62be00640 1 -- 172.21.15.7:0/4154407471 learned_addr learned my addr 172.21.15.7:0/4154407471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.972+0000 7fc633400640 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 -- 172.21.15.7:0/4154407471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 msgr2=0x55d2a8edca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 -- 172.21.15.7:0/4154407471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecb410 msgr2=0x55d2a8ed54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 -- 172.21.15.7:0/4154407471 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a8ec7b40 con 0x55d2a8ecbe00 2024-06-13T07:02:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.973+0000 7fc62be00640 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6340736e0 tx=0x7fc624009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faedae30a8db40b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.974+0000 7fc623400640 1 -- 172.21.15.7:0/4154407471 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624019030 con 0x55d2a8ecbe00 2024-06-13T07:02:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.974+0000 7fc623400640 1 -- 172.21.15.7:0/4154407471 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624012c60 con 0x55d2a8ecbe00 2024-06-13T07:02:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.974+0000 7fc623400640 1 -- 172.21.15.7:0/4154407471 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62400f040 con 0x55d2a8ecbe00 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.974+0000 7fc63cc16740 1 -- 172.21.15.7:0/4154407471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 msgr2=0x55d2a8ed5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.974+0000 7fc63cc16740 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc6340736e0 tx=0x7fc624009110 comp rx=0 tx=0).stop 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 -- 172.21.15.7:0/4154407471 shutdown_connections 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecbe00 0x55d2a8ed5a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 --2- 172.21.15.7:0/4154407471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 -- 172.21.15.7:0/4154407471 >> 172.21.15.7:0/4154407471 conn(0x55d2a8e8afb0 msgr2=0x55d2a8ed4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 -- 172.21.15.7:0/4154407471 shutdown_connections 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 -- 172.21.15.7:0/4154407471 wait complete. 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 -- start start 2024-06-13T07:02:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.975+0000 7fc63cc16740 1 Processor -- start 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341af110 con 0x55d2a8ecbe00 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341b0580 con 0x55d2a8ed0f70 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341af7e0 con 0x55d2a8ecb410 2024-06-13T07:02:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc63cc16740 10 client.0 _ll_register_callbacks cb 0x7fc61c004240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc62be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc632a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc62be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47602/0 (socket says 172.21.15.7:47602) 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc62be00640 1 -- 172.21.15.7:0/1260882772 learned_addr learned my addr 172.21.15.7:0/1260882772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.976+0000 7fc633400640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 -- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 msgr2=0x55d2a8edafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 -- 172.21.15.7:0/1260882772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 msgr2=0x55d2a8eda000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 -- 172.21.15.7:0/1260882772 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a8edb5b0 con 0x55d2a8ecb410 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc62be00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc632a00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed9ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00bef0 tx=0x7fc61c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d238a3e1c04511c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.977+0000 7fc620c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.978+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c02a610 con 0x55d2a8ecb410 2024-06-13T07:02:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.978+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc61c02a7b0 con 0x55d2a8ecb410 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.978+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624012c60 con 0x55d2a8ecb410 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.979+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc61c02a9b0 con 0x55d2a8ecb410 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.979+0000 7fc63cc16740 10 client.26236 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.979+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d2a8edc9c0 con 0x55d2a8ecb410 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.980+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624012c60 con 0x55d2a8ecb410 2024-06-13T07:02:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.980+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fc61c031030 con 0x55d2a8ecb410 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 10 client.26236 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 20 client.26236 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8f4c290 0x55d2a8f4e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 4 client.26236 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2a8e2dea0 con 0x55d2a8f4c290 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.981+0000 7fc62be00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8f4c290 0x55d2a8f4e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.982+0000 7fc620c00640 10 client.26236 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:02.982+0000 7fc62be00640 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8f4c290 0x55d2a8f4e6b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230bf6704091339f server_cookie=5c5148a647ad98ec in_seq=0 out_seq=0 2024-06-13T07:02:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc620c00640 1 -- 172.21.15.7:0/1260882772 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d2a8e2dea0 con 0x55d2a8f4c290 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc620c00640 10 client.26236 handle_command_reply: tid=0 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 client.26236 shutdown 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 msgr2=0x55d2a8ed9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed9ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00bef0 tx=0x7fc61c004730 comp rx=0 tx=0).stop 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 shutdown_connections 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d2a8f4c290 0x55d2a8f4e6b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d2a8ed0f70 0x55d2a8edafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2a8ecbe00 0x55d2a8eda000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc620c00640 0 client.26236 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 --2- 172.21.15.7:0/1260882772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2a8ecb410 0x55d2a8ed9ac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.005+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 >> 172.21.15.7:0/1260882772 conn(0x55d2a8e8afb0 msgr2=0x55d2a8ec9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.006+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 shutdown_connections 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.006+0000 7fc63cc16740 1 -- 172.21.15.7:0/1260882772 wait complete. 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.006+0000 7fc63cc16740 20 client.26236 trim_cache size 0 max 16384 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:02:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:02:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:02:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 msgr2=0x7fc6180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 0x7fc6180a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0045f0 tx=0x7fc61c00b390 comp rx=0 tx=0).stop 2024-06-13T07:02:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 msgr2=0x7fc6341a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009b30 tx=0x7fc62c030710 comp rx=0 tx=0).stop 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 shutdown_connections 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc6180a2c30 0x7fc6180a50f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc63410ed60 0x7fc6341a9c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc63410c3e0 0x7fc6341a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 --2- 172.21.15.7:0/3203605061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc634109ab0 0x7fc6341a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 >> 172.21.15.7:0/3203605061 conn(0x7fc63406ea10 msgr2=0x7fc63410bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 shutdown_connections 2024-06-13T07:02:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:03.007+0000 7fc639e00640 1 -- 172.21.15.7:0/3203605061 wait complete. 2024-06-13T07:02:03.066 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:02 smithi007 ceph-mon[32077]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 490 op/s 2024-06-13T07:02:03.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:02 smithi189 ceph-mon[36616]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 490 op/s 2024-06-13T07:02:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:02 smithi175 ceph-mon[36886]: pgmap v689: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 490 op/s 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 -- 172.21.15.7:0/1204688860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 msgr2=0x7f36c806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 --2- 172.21.15.7:0/1204688860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c806ab40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009990 tx=0x7f36b4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 -- 172.21.15.7:0/1204688860 shutdown_connections 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 --2- 172.21.15.7:0/1204688860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c8104ed0 0x7f36c8105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 --2- 172.21.15.7:0/1204688860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c81073c0 0x7f36c806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 --2- 172.21.15.7:0/1204688860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c806ab40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 -- 172.21.15.7:0/1204688860 >> 172.21.15.7:0/1204688860 conn(0x7f36c8078340 msgr2=0x7f36c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 -- 172.21.15.7:0/1204688860 shutdown_connections 2024-06-13T07:02:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.117+0000 7f36cee00700 1 -- 172.21.15.7:0/1204688860 wait complete. 2024-06-13T07:02:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.119+0000 7f36cee00700 1 Processor -- start 2024-06-13T07:02:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.119+0000 7f36cee00700 1 -- start start 2024-06-13T07:02:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.120+0000 7f36cee00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c8115740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.120+0000 7f36cee00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 0x7f36c8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.120 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:05 smithi007 ceph-mon[32077]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 615 op/s 2024-06-13T07:02:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.120+0000 7f36cee00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 0x7f36c810e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.120+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c8115740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.120+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c8115740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56846/0 (socket says 172.21.15.7:56846) 2024-06-13T07:02:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8118d70 con 0x7f36c81073c0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36c7e00700 1 -- 172.21.15.7:0/897523999 learned_addr learned my addr 172.21.15.7:0/897523999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8118bf0 con 0x7f36c806a6e0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8118ef0 con 0x7f36c8104ed0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36c7400700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 0x7f36c8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.121+0000 7f36cce00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 0x7f36c810e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 -- 172.21.15.7:0/897523999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 msgr2=0x7f36c8115740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c8115740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 -- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 msgr2=0x7f36c810e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 0x7f36c810e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c810f2a0 con 0x7f36c8104ed0 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36cce00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 0x7f36c810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.122+0000 7f36c7400700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 0x7f36c8115c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006010 tx=0x7f36b400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.123+0000 7f36c4c00700 1 -- 172.21.15.7:0/897523999 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4027070 con 0x7f36c8104ed0 2024-06-13T07:02:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.123+0000 7f36c4c00700 1 -- 172.21.15.7:0/897523999 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b4003d70 con 0x7f36c8104ed0 2024-06-13T07:02:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.123+0000 7f36c4c00700 1 -- 172.21.15.7:0/897523999 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400f460 con 0x7f36c8104ed0 2024-06-13T07:02:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.123+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c810f530 con 0x7f36c8104ed0 2024-06-13T07:02:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.123+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c8113440 con 0x7f36c8104ed0 2024-06-13T07:02:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.130+0000 7f36c4c00700 1 -- 172.21.15.7:0/897523999 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f36b402a070 con 0x7f36c8104ed0 2024-06-13T07:02:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.130+0000 7f36d977eb80 1 Processor -- start 2024-06-13T07:02:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.131+0000 7f36d977eb80 1 -- start start 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.131+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1948070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c1948a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36d977eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1857a10 con 0x55c6c18c4e40 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36d977eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c6c189ab00 con 0x55c6c1953300 2024-06-13T07:02:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36d977eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6c18558f0 con 0x55c6c19485b0 2024-06-13T07:02:05.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36c4c00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c1948a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36cce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1948070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c1948a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42256/0 (socket says 172.21.15.7:42256) 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.132+0000 7f36c7e00700 1 -- 172.21.15.7:0/3608349969 learned_addr learned my addr 172.21.15.7:0/3608349969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36c4c00700 1 -- 172.21.15.7:0/897523999 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36b40d01e0 con 0x7f36c8104ed0 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36cce00700 1 -- 172.21.15.7:0/3608349969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 msgr2=0x55c6c1948a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36cce00700 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c1948a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36cce00700 1 -- 172.21.15.7:0/3608349969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 msgr2=0x55c6c19556f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36cce00700 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19556f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.133+0000 7f36cce00700 1 -- 172.21.15.7:0/3608349969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c1842f90 con 0x55c6c18c4e40 2024-06-13T07:02:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.134+0000 7f36cce00700 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1948070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f36c00094e0 tx=0x7f36c0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3b940c79fcfad7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.134+0000 7f36cce00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.134+0000 7f36bd600700 1 -- 172.21.15.7:0/3608349969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0012040 con 0x55c6c18c4e40 2024-06-13T07:02:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.135+0000 7f36bd600700 1 -- 172.21.15.7:0/3608349969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c0016bb0 con 0x55c6c18c4e40 2024-06-13T07:02:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.135+0000 7f36bd600700 1 -- 172.21.15.7:0/3608349969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c001ee50 con 0x55c6c18c4e40 2024-06-13T07:02:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36cce00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f36c0018040 tx=0x7f36c0015860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 -- 172.21.15.7:0/3608349969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 msgr2=0x55c6c1948070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1948070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f36c00094e0 tx=0x7f36c0009510 comp rx=0 tx=0).stop 2024-06-13T07:02:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 -- 172.21.15.7:0/3608349969 shutdown_connections 2024-06-13T07:02:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1948070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 --2- 172.21.15.7:0/3608349969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c1948a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.136+0000 7f36d977eb80 1 -- 172.21.15.7:0/3608349969 >> 172.21.15.7:0/3608349969 conn(0x55c6c1875170 msgr2=0x55c6c19444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.137+0000 7f36d977eb80 1 -- 172.21.15.7:0/3608349969 shutdown_connections 2024-06-13T07:02:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.137+0000 7f36d977eb80 1 -- 172.21.15.7:0/3608349969 wait complete. 2024-06-13T07:02:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.137+0000 7f36d977eb80 1 -- start start 2024-06-13T07:02:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.138+0000 7f36d977eb80 1 Processor -- start 2024-06-13T07:02:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1952ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c194fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1836f60 con 0x55c6c18c4e40 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1896ac0 con 0x55c6c1953300 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6c1966fd0 con 0x55c6c19485b0 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.139+0000 7f36d977eb80 10 client.0 _ll_register_callbacks cb 0x55c6c1834720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.140+0000 7f36cce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.140+0000 7f36cce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56870/0 (socket says 172.21.15.7:56870) 2024-06-13T07:02:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.140+0000 7f36cce00700 1 -- 172.21.15.7:0/2926225506 learned_addr learned my addr 172.21.15.7:0/2926225506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.141+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1952ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.141+0000 7f36c7400700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c194fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.141+0000 7f36c7400700 1 -- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 msgr2=0x55c6c19503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7400700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7400700 1 -- 172.21.15.7:0/2926225506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 msgr2=0x55c6c1952ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7400700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1952ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7400700 1 -- 172.21.15.7:0/2926225506 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c17e98d0 con 0x55c6c19485b0 2024-06-13T07:02:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36cce00700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1952ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36c7400700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c194fe70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36b40cd940 tx=0x7f36b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e6cf1786b33729 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.142+0000 7f36b2000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.143+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b409c390 con 0x55c6c19485b0 2024-06-13T07:02:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.143+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36b40cda50 con 0x55c6c19485b0 2024-06-13T07:02:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.144+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c0016bb0 con 0x55c6c19485b0 2024-06-13T07:02:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.144+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36b409c1b0 con 0x55c6c19485b0 2024-06-13T07:02:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.144+0000 7f36d977eb80 10 client.26248 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:05.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.144+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6c1971710 con 0x55c6c19485b0 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.144+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c0016bb0 con 0x55c6c19485b0 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f36b40b0ce0 con 0x55c6c19485b0 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 10 client.26248 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 20 client.26248 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c1973be0 0x55c6c1975fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 4 client.26248 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.145+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c18a8000 con 0x55c6c1973be0 2024-06-13T07:02:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.146+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c1973be0 0x55c6c1975fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.146+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c1973be0 0x55c6c1975fd0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0d93b07881baa0 server_cookie=83ecbc2ec6bc83ce in_seq=0 out_seq=0 2024-06-13T07:02:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.147+0000 7f36b2000700 10 client.26248 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.148+0000 7f36b2000700 1 -- 172.21.15.7:0/2926225506 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6c18a8000 con 0x55c6c1973be0 2024-06-13T07:02:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.148+0000 7f36b2000700 10 client.26248 handle_command_reply: tid=0 2024-06-13T07:02:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.148+0000 7f36d977eb80 1 client.26248 shutdown 2024-06-13T07:02:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 msgr2=0x55c6c194fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c194fe70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36b40cd940 tx=0x7f36b4000bc0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 shutdown_connections 2024-06-13T07:02:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c1973be0 0x55c6c1975fd0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c19503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c1952ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36b2000700 0 client.26248 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2926225506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c194fe70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.149+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 >> 172.21.15.7:0/2926225506 conn(0x55c6c1875170 msgr2=0x55c6c1943740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.151+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 shutdown_connections 2024-06-13T07:02:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.151+0000 7f36d977eb80 1 -- 172.21.15.7:0/2926225506 wait complete. 2024-06-13T07:02:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.152+0000 7f36d977eb80 20 client.26248 trim_cache size 0 max 16384 2024-06-13T07:02:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 Processor -- start 2024-06-13T07:02:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 -- start start 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c19485b0 0x55c6c1954c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 0x55c6c194a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81b6ec0 con 0x55c6c1953300 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8073b40 con 0x55c6c19485b0 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.201+0000 7f36d977eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8071a40 con 0x55c6c18c4e40 2024-06-13T07:02:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42282/0 (socket says 172.21.15.7:42282) 2024-06-13T07:02:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 -- 172.21.15.7:0/20697825 learned_addr learned my addr 172.21.15.7:0/20697825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7400700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c19485b0 0x55c6c1954c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36cce00700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 0x55c6c194a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 -- 172.21.15.7:0/20697825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c19485b0 msgr2=0x55c6c1954c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c19485b0 0x55c6c1954c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 -- 172.21.15.7:0/20697825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 msgr2=0x55c6c194a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 0x55c6c194a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36c7e00700 1 -- 172.21.15.7:0/20697825 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c193be10 con 0x55c6c18c4e40 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.202+0000 7f36cce00700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 0x55c6c194a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.204+0000 7f36c7e00700 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c6c19766a0 tx=0x7f36b8014800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcfe952fe7f28585 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.205+0000 7f36bcc00700 1 -- 172.21.15.7:0/20697825 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80140d0 con 0x55c6c18c4e40 2024-06-13T07:02:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.205+0000 7f36bcc00700 1 -- 172.21.15.7:0/20697825 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b8014270 con 0x55c6c18c4e40 2024-06-13T07:02:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.205+0000 7f36bcc00700 1 -- 172.21.15.7:0/20697825 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b801d460 con 0x55c6c18c4e40 2024-06-13T07:02:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 -- 172.21.15.7:0/20697825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 msgr2=0x55c6c19546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c6c19766a0 tx=0x7f36b8014800 comp rx=0 tx=0).stop 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 -- 172.21.15.7:0/20697825 shutdown_connections 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c1953300 0x55c6c194a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c18c4e40 0x55c6c19546f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 --2- 172.21.15.7:0/20697825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c19485b0 0x55c6c1954c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.206+0000 7f36d977eb80 1 -- 172.21.15.7:0/20697825 >> 172.21.15.7:0/20697825 conn(0x55c6c1875170 msgr2=0x55c6c194e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.207+0000 7f36d977eb80 1 -- 172.21.15.7:0/20697825 shutdown_connections 2024-06-13T07:02:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.207+0000 7f36d977eb80 1 -- 172.21.15.7:0/20697825 wait complete. 2024-06-13T07:02:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.207+0000 7f36d977eb80 1 -- start start 2024-06-13T07:02:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 Processor -- start 2024-06-13T07:02:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c194d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c196c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c196c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8072000 con 0x55c6c18c4e40 2024-06-13T07:02:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8073470 con 0x55c6c1953300 2024-06-13T07:02:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.208+0000 7f36d977eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80726d0 con 0x55c6c19485b0 2024-06-13T07:02:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.209+0000 7f36d977eb80 10 client.0 _ll_register_callbacks cb 0x55c6c1834720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.209+0000 7f36cce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c196c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.209+0000 7f36cce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c196c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56890/0 (socket says 172.21.15.7:56890) 2024-06-13T07:02:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.210+0000 7f36cce00700 1 -- 172.21.15.7:0/2468289585 learned_addr learned my addr 172.21.15.7:0/2468289585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.209+0000 7f36c7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c194d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.209+0000 7f36c7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c196c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.211+0000 7f36c7400700 1 -- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 msgr2=0x55c6c196c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.211+0000 7f36c7400700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c196c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.211+0000 7f36c7400700 1 -- 172.21.15.7:0/2468289585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 msgr2=0x55c6c194d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36c7400700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c194d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36c7400700 1 -- 172.21.15.7:0/2468289585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c196cca0 con 0x55c6c19485b0 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c194d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36c7400700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c196c220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f36b409ab00 tx=0x7f36b4022470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb56bb6fe90041e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:05.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36b2000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b409c390 con 0x55c6c19485b0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.212+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f36b409beb0 con 0x55c6c19485b0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.213+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b8014270 con 0x55c6c19485b0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.213+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f36b40a5230 con 0x55c6c19485b0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.213+0000 7f36d977eb80 10 client.26254 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.213+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6c19de780 con 0x55c6c19485b0 2024-06-13T07:02:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.213+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b8014270 con 0x55c6c19485b0 2024-06-13T07:02:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.214+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f36b40b0ce0 con 0x55c6c19485b0 2024-06-13T07:02:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 10 client.26254 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 20 client.26254 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c19dead0 0x55c6c19e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 4 client.26254 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6c18a2580 con 0x55c6c19dead0 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.215+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c19dead0 0x55c6c19e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.216+0000 7f36b2000700 10 client.26254 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.216+0000 7f36c7e00700 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c19dead0 0x55c6c19e0ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a5dcbbe9e83a5 server_cookie=4d89fa2084e24507 in_seq=0 out_seq=0 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.216+0000 7f36b2000700 1 -- 172.21.15.7:0/2468289585 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c6c18a2580 con 0x55c6c19dead0 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.216+0000 7f36b2000700 10 client.26254 handle_command_reply: tid=0 2024-06-13T07:02:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.216+0000 7f36d977eb80 1 client.26254 shutdown 2024-06-13T07:02:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.217+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 msgr2=0x55c6c196c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.217+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c196c220 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f36b409ab00 tx=0x7f36b4022470 comp rx=0 tx=0).stop 2024-06-13T07:02:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 shutdown_connections 2024-06-13T07:02:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c6c19dead0 0x55c6c19e0ef0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36b2000700 0 client.26254 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c6c1953300 0x55c6c196c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6c18c4e40 0x55c6c194d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 --2- 172.21.15.7:0/2468289585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6c19485b0 0x55c6c196c220 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.218+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 >> 172.21.15.7:0/2468289585 conn(0x55c6c1875170 msgr2=0x55c6c1942eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.219+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 shutdown_connections 2024-06-13T07:02:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.219+0000 7f36d977eb80 1 -- 172.21.15.7:0/2468289585 wait complete. 2024-06-13T07:02:05.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.221+0000 7f36d977eb80 20 client.26254 trim_cache size 0 max 16384 2024-06-13T07:02:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:02:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:02:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:02:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.223+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 msgr2=0x7f36ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.223+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f36c0018040 tx=0x7f36c0015860 comp rx=0 tx=0).stop 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.223+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 msgr2=0x7f36c8115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.223+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 0x7f36c8115c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006010 tx=0x7f36b400bab0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 shutdown_connections 2024-06-13T07:02:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36c81073c0 0x7f36c810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c806a6e0 0x7f36c8115740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 --2- 172.21.15.7:0/897523999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36c8104ed0 0x7f36c8115c80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 >> 172.21.15.7:0/897523999 conn(0x7f36c8078340 msgr2=0x7f36c8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 shutdown_connections 2024-06-13T07:02:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:05.224+0000 7f36cee00700 1 -- 172.21.15.7:0/897523999 wait complete. 2024-06-13T07:02:05.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:05 smithi189 ceph-mon[36616]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 615 op/s 2024-06-13T07:02:05.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:05 smithi175 ceph-mon[36886]: pgmap v690: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 615 op/s 2024-06-13T07:02:06.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:02:06.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:02:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:07.233 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:07 smithi007 ceph-mon[32077]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.3 MiB/s wr, 478 op/s 2024-06-13T07:02:07.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:07 smithi189 ceph-mon[36616]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.3 MiB/s wr, 478 op/s 2024-06-13T07:02:07.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:07 smithi175 ceph-mon[36886]: pgmap v691: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.3 MiB/s wr, 478 op/s 2024-06-13T07:02:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 365.0 s, 549.2 tps, lat 179.898 ms stddev 221.590 2024-06-13T07:02:09.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:09 smithi007 ceph-mon[32077]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 625 op/s 2024-06-13T07:02:09.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:09 smithi189 ceph-mon[36616]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 625 op/s 2024-06-13T07:02:09.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:09 smithi175 ceph-mon[36886]: pgmap v692: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 625 op/s 2024-06-13T07:02:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.617+0000 7fb673000700 1 -- 172.21.15.7:0/3261359320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 msgr2=0x7fb66c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.617+0000 7fb673000700 1 --2- 172.21.15.7:0/3261359320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c074aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009990 tx=0x7fb65c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 -- 172.21.15.7:0/3261359320 shutdown_connections 2024-06-13T07:02:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 --2- 172.21.15.7:0/3261359320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb66c10eec0 0x7fb66c1112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 --2- 172.21.15.7:0/3261359320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c074aa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 --2- 172.21.15.7:0/3261359320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb66c074fe0 0x7fb66c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 -- 172.21.15.7:0/3261359320 >> 172.21.15.7:0/3261359320 conn(0x7fb66c079560 msgr2=0x7fb66c079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 -- 172.21.15.7:0/3261359320 shutdown_connections 2024-06-13T07:02:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.620+0000 7fb673000700 1 -- 172.21.15.7:0/3261359320 wait complete. 2024-06-13T07:02:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.622+0000 7fb673000700 1 Processor -- start 2024-06-13T07:02:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 -- start start 2024-06-13T07:02:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb66c074fe0 0x7fb66c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb66c10eec0 0x7fb66c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c114b30 con 0x7fb66c074fe0 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c1149b0 con 0x7fb66c0746c0 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.623+0000 7fb673000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c114cb0 con 0x7fb66c10eec0 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:56918/0 (socket says 172.21.15.7:56918) 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 -- 172.21.15.7:0/2633504835 learned_addr learned my addr 172.21.15.7:0/2633504835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 -- 172.21.15.7:0/2633504835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb66c10eec0 msgr2=0x7fb66c19f8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb66c10eec0 0x7fb66c19f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 -- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb66c074fe0 msgr2=0x7fb66c19b5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb66c074fe0 0x7fb66c19b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.624+0000 7fb66be00700 1 -- 172.21.15.7:0/2633504835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c1a0020 con 0x7fb66c0746c0 2024-06-13T07:02:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.627+0000 7fb66be00700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0057f0 tx=0x7fb65c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.627+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c027070 con 0x7fb66c0746c0 2024-06-13T07:02:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.627+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb65c0041b0 con 0x7fb66c0746c0 2024-06-13T07:02:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.627+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c027070 con 0x7fb66c0746c0 2024-06-13T07:02:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.627+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66c1a02b0 con 0x7fb66c0746c0 2024-06-13T07:02:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.628+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66c1a0680 con 0x7fb66c0746c0 2024-06-13T07:02:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.630+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66c069180 con 0x7fb66c0746c0 2024-06-13T07:02:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.634+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb65c02a070 con 0x7fb66c0746c0 2024-06-13T07:02:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.636+0000 7fb668c00700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 0x7fb6500a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.637+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb65c09a7e0 con 0x7fb66c0746c0 2024-06-13T07:02:10.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.638+0000 7fb66b400700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 0x7fb6500a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:10.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.639+0000 7fb66b400700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 0x7fb6500a5360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb66c110ba0 tx=0x7fb658006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:10.646+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb65c05e850 con 0x7fb66c0746c0 2024-06-13T07:02:11.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.130+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb66c02dca0 con 0x7fb66c0746c0 2024-06-13T07:02:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.131+0000 7fb668c00700 1 -- 172.21.15.7:0/2633504835 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11870 (secure 0 0 0) 0x7fb65c013070 con 0x7fb66c0746c0 2024-06-13T07:02:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:02:11.131 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:02:00.092034+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24521,24527]},"id":1}]} 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.134+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 msgr2=0x7fb6500a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.134+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 0x7fb6500a5360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb66c110ba0 tx=0x7fb658006c60 comp rx=0 tx=0).stop 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.134+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 msgr2=0x7fb66c19b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.134+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0057f0 tx=0x7fb65c003680 comp rx=0 tx=0).stop 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 shutdown_connections 2024-06-13T07:02:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb66c10eec0 0x7fb66c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:11.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb6500a2ea0 0x7fb6500a5360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:11.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb66c0746c0 0x7fb66c19b060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:11.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 --2- 172.21.15.7:0/2633504835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb66c074fe0 0x7fb66c19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:11.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.135+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 >> 172.21.15.7:0/2633504835 conn(0x7fb66c079560 msgr2=0x7fb66c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:11.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.136+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 shutdown_connections 2024-06-13T07:02:11.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:11.136+0000 7fb673000700 1 -- 172.21.15.7:0/2633504835 wait complete. 2024-06-13T07:02:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-06-13T07:02:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:11 smithi189 ceph-mon[36616]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-06-13T07:02:11.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:11 smithi175 ceph-mon[36886]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-06-13T07:02:11.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:11 smithi007 ceph-mon[32077]: pgmap v693: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.9 MiB/s wr, 459 op/s 2024-06-13T07:02:12.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:12 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2633504835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:12.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:12 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2633504835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:12.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:12 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2633504835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:02:12.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:02:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 370.0 s, 487.8 tps, lat 199.612 ms stddev 289.490 2024-06-13T07:02:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:13 smithi189 ceph-mon[36616]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-06-13T07:02:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:13 smithi175 ceph-mon[36886]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-06-13T07:02:13.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:13 smithi007 ceph-mon[32077]: pgmap v694: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-06-13T07:02:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:15 smithi189 ceph-mon[36616]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-06-13T07:02:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:15 smithi175 ceph-mon[36886]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-06-13T07:02:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:15 smithi007 ceph-mon[32077]: pgmap v695: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-06-13T07:02:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.221+0000 7f54c5000640 1 -- 172.21.15.7:0/1706844460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0104e20 msgr2=0x7f54c01052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.221+0000 7f54c5000640 1 --2- 172.21.15.7:0/1706844460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0104e20 0x7f54c01052a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009900 tx=0x7f54a802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.222+0000 7f54c5000640 1 -- 172.21.15.7:0/1706844460 shutdown_connections 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.222+0000 7f54c5000640 1 --2- 172.21.15.7:0/1706844460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c01057e0 0x7f54c010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.222+0000 7f54c5000640 1 --2- 172.21.15.7:0/1706844460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0104e20 0x7f54c01052a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.222+0000 7f54c5000640 1 --2- 172.21.15.7:0/1706844460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c0103c30 0x7f54c0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.222+0000 7f54c5000640 1 -- 172.21.15.7:0/1706844460 >> 172.21.15.7:0/1706844460 conn(0x7f54c00ff1c0 msgr2=0x7f54c01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.223+0000 7f54c5000640 1 -- 172.21.15.7:0/1706844460 shutdown_connections 2024-06-13T07:02:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.227+0000 7f54c5000640 1 -- 172.21.15.7:0/1706844460 wait complete. 2024-06-13T07:02:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.228+0000 7f54c5000640 1 Processor -- start 2024-06-13T07:02:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.229+0000 7f54c5000640 1 -- start start 2024-06-13T07:02:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c0104e20 0x7f54c006c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c01057e0 0x7f54c00733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54c010f960 con 0x7f54c01057e0 2024-06-13T07:02:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54c010f7e0 con 0x7f54c0104e20 2024-06-13T07:02:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.230+0000 7f54c5000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c010fae0 con 0x7f54c0103c30 2024-06-13T07:02:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33964/0 (socket says 172.21.15.7:33964) 2024-06-13T07:02:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 -- 172.21.15.7:0/814974734 learned_addr learned my addr 172.21.15.7:0/814974734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 -- 172.21.15.7:0/814974734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c0104e20 msgr2=0x7f54c006c3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c0104e20 0x7f54c006c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 -- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c01057e0 msgr2=0x7f54c00733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c01057e0 0x7f54c00733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 -- 172.21.15.7:0/814974734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01b7060 con 0x7f54c0103c30 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.231+0000 7f54bde00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b000d770 tx=0x7f54b000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.232+0000 7f54b6a00640 1 -- 172.21.15.7:0/814974734 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0017070 con 0x7f54c0103c30 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.232+0000 7f54b6a00640 1 -- 172.21.15.7:0/814974734 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b0009c30 con 0x7f54c0103c30 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.232+0000 7f54b6a00640 1 -- 172.21.15.7:0/814974734 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0003dd0 con 0x7f54c0103c30 2024-06-13T07:02:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.232+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01b7230 con 0x7f54c0103c30 2024-06-13T07:02:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.232+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01b7780 con 0x7f54c0103c30 2024-06-13T07:02:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.238+0000 7f54c7dc4740 1 Processor -- start 2024-06-13T07:02:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.238+0000 7f54c7dc4740 1 -- start start 2024-06-13T07:02:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fdedf28420 con 0x55fdedfca760 2024-06-13T07:02:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fdedecfc50 con 0x55fdedfc4c00 2024-06-13T07:02:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.239+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdedf243e0 con 0x55fdedfc55f0 2024-06-13T07:02:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.240+0000 7f54b6a00640 1 -- 172.21.15.7:0/814974734 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f54b0003f70 con 0x7f54c0103c30 2024-06-13T07:02:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:32984/0 (socket says 172.21.15.7:32984) 2024-06-13T07:02:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 -- 172.21.15.7:0/2370330535 learned_addr learned my addr 172.21.15.7:0/2370330535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54bde00640 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 -- 172.21.15.7:0/2370330535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 msgr2=0x55fdedfca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 -- 172.21.15.7:0/2370330535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 msgr2=0x55fdedfccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.241+0000 7f54be800640 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.242+0000 7f54be800640 1 -- 172.21.15.7:0/2370330535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdedc3c210 con 0x55fdedfc4c00 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.243+0000 7f54be800640 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009d00 tx=0x7f54b800b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe4f6d11a7fad0c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.243+0000 7f54af400640 1 -- 172.21.15.7:0/2370330535 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800bd40 con 0x55fdedfc4c00 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.243+0000 7f54b6a00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 0x7f54a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.243+0000 7f54af400640 1 -- 172.21.15.7:0/2370330535 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x55fdedfc4c00 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.243+0000 7f54af400640 1 -- 172.21.15.7:0/2370330535 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800bd40 con 0x55fdedfc4c00 2024-06-13T07:02:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.244+0000 7f54bde00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 0x7f54a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.244+0000 7f54b6a00640 1 -- 172.21.15.7:0/814974734 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54b00cea30 con 0x7f54c0103c30 2024-06-13T07:02:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.244+0000 7f54c7dc4740 1 -- 172.21.15.7:0/2370330535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 msgr2=0x55fdedfc5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.244+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009d00 tx=0x7f54b800b420 comp rx=0 tx=0).stop 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 -- 172.21.15.7:0/2370330535 shutdown_connections 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/2370330535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc4c00 0x55fdedfc5020 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 -- 172.21.15.7:0/2370330535 >> 172.21.15.7:0/2370330535 conn(0x55fdedf8cfb0 msgr2=0x55fdedfc0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.245+0000 7f54c7dc4740 1 -- 172.21.15.7:0/2370330535 shutdown_connections 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.246+0000 7f54bde00640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 0x7f54a40a5170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cc560 tx=0x7f54b00bb040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.246+0000 7f54c7dc4740 1 -- 172.21.15.7:0/2370330535 wait complete. 2024-06-13T07:02:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.246+0000 7f54c7dc4740 1 -- start start 2024-06-13T07:02:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.247+0000 7f54c7dc4740 1 Processor -- start 2024-06-13T07:02:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.247+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.248+0000 7f54be800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54be800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58496/0 (socket says 172.21.15.7:58496) 2024-06-13T07:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54be800640 1 -- 172.21.15.7:0/683250335 learned_addr learned my addr 172.21.15.7:0/683250335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfcf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdedfd6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fdedfcd710 con 0x55fdedfc4c00 2024-06-13T07:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fdedee30b0 con 0x55fdedfca760 2024-06-13T07:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdedfcd890 con 0x55fdedfc55f0 2024-06-13T07:02:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.249+0000 7f54c7dc4740 10 client.0 _ll_register_callbacks cb 0x55fdedecef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.250+0000 7f54be800640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdedfd6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.250+0000 7f54b7e00640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfcf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.251+0000 7f54be800640 1 -- 172.21.15.7:0/683250335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 msgr2=0x55fdedfcf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.251+0000 7f54be800640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfcf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.251+0000 7f54be800640 1 -- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 msgr2=0x55fdedfd6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.251+0000 7f54be800640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdedfd6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.251+0000 7f54be800640 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdedd08620 con 0x55fdedfc4c00 2024-06-13T07:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.254+0000 7f54be800640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009900 tx=0x7f54b8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1743f910831496 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.254+0000 7f54a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.254+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8020e20 con 0x55fdedfc4c00 2024-06-13T07:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.254+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54b8029450 con 0x55fdedfc4c00 2024-06-13T07:02:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.255+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b8017070 con 0x55fdedfc4c00 2024-06-13T07:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.255+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f54b8029650 con 0x55fdedfc4c00 2024-06-13T07:02:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.256+0000 7f54c7dc4740 10 client.15831 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.256+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fdedfd97f0 con 0x55fdedfc4c00 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x55fdedfc4c00 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f54b8011070 con 0x55fdedfc4c00 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 10 client.15831 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 20 client.15831 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdedfdac00 0x55fdedfdcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 4 client.15831 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.257+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdedf35b50 con 0x55fdedfdac00 2024-06-13T07:02:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.260+0000 7f54b7e00640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdedfdac00 0x55fdedfdcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.262+0000 7f54b7e00640 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdedfdac00 0x55fdedfdcff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc90603282f9086b server_cookie=274474d06190dfb3 in_seq=0 out_seq=0 2024-06-13T07:02:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54a3e00640 10 client.15831 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54a3e00640 1 -- 172.21.15.7:0/683250335 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdedf35b50 con 0x55fdedfdac00 2024-06-13T07:02:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54a3e00640 10 client.15831 handle_command_reply: tid=0 2024-06-13T07:02:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 client.15831 shutdown 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 msgr2=0x55fdedfcef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009900 tx=0x7f54b8009850 comp rx=0 tx=0).stop 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 shutdown_connections 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdedfdac00 0x55fdedfdcff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdedfd6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfcf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/683250335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfcef00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 >> 172.21.15.7:0/683250335 conn(0x55fdedf8cfb0 msgr2=0x55fdedfbf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.266+0000 7f54a3e00640 0 client.15831 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.267+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 shutdown_connections 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.267+0000 7f54c7dc4740 1 -- 172.21.15.7:0/683250335 wait complete. 2024-06-13T07:02:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.267+0000 7f54c7dc4740 20 client.15831 trim_cache size 0 max 16384 2024-06-13T07:02:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 Processor -- start 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 -- start start 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc55f0 0x55fdedfd6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01c45c0 con 0x55fdedfca760 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01c1e30 con 0x55fdedfc55f0 2024-06-13T07:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.317+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01bfc80 con 0x55fdedfc4c00 2024-06-13T07:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.318+0000 7f54be800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.318+0000 7f54be800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33992/0 (socket says 172.21.15.7:33992) 2024-06-13T07:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.318+0000 7f54be800640 1 -- 172.21.15.7:0/3735575249 learned_addr learned my addr 172.21.15.7:0/3735575249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.318+0000 7f54be800640 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 -- 172.21.15.7:0/3735575249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc55f0 msgr2=0x55fdedfd6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc55f0 0x55fdedfd6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 -- 172.21.15.7:0/3735575249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 msgr2=0x55fdedfce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 -- 172.21.15.7:0/3735575249 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdedfc59d0 con 0x55fdedfc4c00 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.319+0000 7f54be800640 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54b80095b0 tx=0x7f54b800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554fc9127cf5f4f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.320+0000 7f54aea00640 1 -- 172.21.15.7:0/3735575249 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800b920 con 0x55fdedfc4c00 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.321+0000 7f54aea00640 1 -- 172.21.15.7:0/3735575249 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8038070 con 0x55fdedfc4c00 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.321+0000 7f54aea00640 1 -- 172.21.15.7:0/3735575249 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80297f0 con 0x55fdedfc4c00 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 -- 172.21.15.7:0/3735575249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 msgr2=0x55fdedfc3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54b80095b0 tx=0x7f54b800b420 comp rx=0 tx=0).stop 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 -- 172.21.15.7:0/3735575249 shutdown_connections 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfca760 0x55fdedfce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfc55f0 0x55fdedfd6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/3735575249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc4c00 0x55fdedfc3ee0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 -- 172.21.15.7:0/3735575249 >> 172.21.15.7:0/3735575249 conn(0x55fdedf8cfb0 msgr2=0x55fdedfcdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 -- 172.21.15.7:0/3735575249 shutdown_connections 2024-06-13T07:02:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.322+0000 7f54c7dc4740 1 -- 172.21.15.7:0/3735575249 wait complete. 2024-06-13T07:02:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.323+0000 7f54c7dc4740 1 -- start start 2024-06-13T07:02:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 Processor -- start 2024-06-13T07:02:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfd3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdee043740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01c02f0 con 0x55fdedfc4c00 2024-06-13T07:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01c1760 con 0x55fdedfca760 2024-06-13T07:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01c09c0 con 0x55fdedfc55f0 2024-06-13T07:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.325+0000 7f54c7dc4740 10 client.0 _ll_register_callbacks cb 0x7f54b8009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.327+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfd3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.327+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfd3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34002/0 (socket says 172.21.15.7:34002) 2024-06-13T07:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.327+0000 7f54b7e00640 1 -- 172.21.15.7:0/1773038723 learned_addr learned my addr 172.21.15.7:0/1773038723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.327+0000 7f54be800640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdee043740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.327+0000 7f54be800640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.328+0000 7f54be800640 1 -- 172.21.15.7:0/1773038723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 msgr2=0x55fdedfd3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.328+0000 7f54be800640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfd3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.328+0000 7f54be800640 1 -- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 msgr2=0x55fdee043740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.328+0000 7f54be800640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdee043740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.328+0000 7f54be800640 1 -- 172.21.15.7:0/1773038723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdedfd3c70 con 0x55fdedfc4c00 2024-06-13T07:02:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.329+0000 7f54be800640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfd3140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f54b8033b60 tx=0x7f54b8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7bfd50ac7b328e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.329+0000 7f54a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.330+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8041070 con 0x55fdedfc4c00 2024-06-13T07:02:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.330+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54b800f050 con 0x55fdedfc4c00 2024-06-13T07:02:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.330+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b8038070 con 0x55fdedfc4c00 2024-06-13T07:02:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.331+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f54b804b050 con 0x55fdedfc4c00 2024-06-13T07:02:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.331+0000 7f54c7dc4740 10 client.15837 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.331+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fdee044d00 con 0x55fdedfc4c00 2024-06-13T07:02:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8038070 con 0x55fdedfc4c00 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f54b8011040 con 0x55fdedfc4c00 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 10 client.15837 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 20 client.15837 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdee0455c0 0x55fdee0479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 4 client.15837 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.332+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdedf2fea0 con 0x55fdee0455c0 2024-06-13T07:02:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.334+0000 7f54b7e00640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdee0455c0 0x55fdee0479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.335+0000 7f54b7e00640 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdee0455c0 0x55fdee0479e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19452bdbb3709d2b server_cookie=d53157aae023439a in_seq=0 out_seq=0 2024-06-13T07:02:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.335+0000 7f54a3e00640 10 client.15837 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 -- 172.21.15.7:0/3196100230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa281073c0 msgr2=0x7ffa2806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3196100230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa281073c0 0x7ffa2806a1a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009990 tx=0x7ffa14009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 -- 172.21.15.7:0/3196100230 shutdown_connections 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3196100230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa28104ed0 0x7ffa28105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3196100230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa281073c0 0x7ffa2806a1a0 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009990 tx=0x7ffa14009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3196100230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.416+0000 7ffa2d400700 1 -- 172.21.15.7:0/3196100230 >> 172.21.15.7:0/3196100230 conn(0x7ffa28078340 msgr2=0x7ffa28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.418+0000 7ffa2d400700 1 -- 172.21.15.7:0/3196100230 shutdown_connections 2024-06-13T07:02:16.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.419+0000 7ffa2d400700 1 -- 172.21.15.7:0/3196100230 wait complete. 2024-06-13T07:02:16.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.421+0000 7ffa2d400700 1 Processor -- start 2024-06-13T07:02:16.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.422+0000 7ffa2d400700 1 -- start start 2024-06-13T07:02:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa281a3df0 0x7ffa281a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28118e70 con 0x7ffa2806a6e0 2024-06-13T07:02:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28118cf0 con 0x7ffa28104ed0 2024-06-13T07:02:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa2d400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28118ff0 con 0x7ffa281a3df0 2024-06-13T07:02:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa25e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa25e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33044/0 (socket says 172.21.15.7:33044) 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa25e00700 1 -- 172.21.15.7:0/3654644105 learned_addr learned my addr 172.21.15.7:0/3654644105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.429+0000 7ffa26800700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 -- 172.21.15.7:0/3654644105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa281a3df0 msgr2=0x7ffa281a4270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa281a3df0 0x7ffa281a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 -- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 msgr2=0x7ffa2819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 -- 172.21.15.7:0/3654644105 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa281a4910 con 0x7ffa28104ed0 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa26800700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:16.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.430+0000 7ffa25e00700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007f10 tx=0x7ffa2000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.431+0000 7ffa1f400700 1 -- 172.21.15.7:0/3654644105 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000ddd0 con 0x7ffa28104ed0 2024-06-13T07:02:16.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.431+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa281a4c00 con 0x7ffa28104ed0 2024-06-13T07:02:16.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.431+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281ac430 con 0x7ffa28104ed0 2024-06-13T07:02:16.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.431+0000 7ffa1f400700 1 -- 172.21.15.7:0/3654644105 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20004bb0 con 0x7ffa28104ed0 2024-06-13T07:02:16.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.431+0000 7ffa1f400700 1 -- 172.21.15.7:0/3654644105 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa200047b0 con 0x7ffa28104ed0 2024-06-13T07:02:16.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.435+0000 7ffa1f400700 1 -- 172.21.15.7:0/3654644105 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ffa2001c020 con 0x7ffa28104ed0 2024-06-13T07:02:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.443+0000 7ffa1f400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 0x7ffa0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.443+0000 7ffa1f400700 1 -- 172.21.15.7:0/3654644105 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ffa200cec10 con 0x7ffa28104ed0 2024-06-13T07:02:16.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.449+0000 7ffa26800700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 0x7ffa0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.450+0000 7ffa37e5db80 1 Processor -- start 2024-06-13T07:02:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.450+0000 7ffa37e5db80 1 -- start start 2024-06-13T07:02:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.452+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.452+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aedcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa26800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56351adeba10 con 0x56351aee34b0 2024-06-13T07:02:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56351ae2eb00 con 0x56351aedd240 2024-06-13T07:02:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa25e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aedcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.453+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56351ade98f0 con 0x56351aee2b90 2024-06-13T07:02:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33058/0 (socket says 172.21.15.7:33058) 2024-06-13T07:02:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 -- 172.21.15.7:0/1808932998 learned_addr learned my addr 172.21.15.7:0/1808932998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 -- 172.21.15.7:0/1808932998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 msgr2=0x56351aee2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa27200700 1 -- 172.21.15.7:0/1808932998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 msgr2=0x56351aedcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.454+0000 7ffa26800700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 0x7ffa0c0a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa140098e0 tx=0x7ffa14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.455+0000 7ffa27200700 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aedcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.455+0000 7ffa27200700 1 -- 172.21.15.7:0/1808932998 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351add6f90 con 0x56351aedd240 2024-06-13T07:02:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.457+0000 7ffa27200700 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097b0 tx=0x7ffa18009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394dcf12e1755292 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.457+0000 7ffa13e00700 1 -- 172.21.15.7:0/1808932998 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18015020 con 0x56351aedd240 2024-06-13T07:02:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.457+0000 7ffa13e00700 1 -- 172.21.15.7:0/1808932998 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18011d10 con 0x56351aedd240 2024-06-13T07:02:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.457+0000 7ffa13e00700 1 -- 172.21.15.7:0/1808932998 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18010e50 con 0x56351aedd240 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.458+0000 7ffa37e5db80 1 -- 172.21.15.7:0/1808932998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 msgr2=0x56351aedd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.458+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097b0 tx=0x7ffa18009ac0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.459+0000 7ffa37e5db80 1 -- 172.21.15.7:0/1808932998 shutdown_connections 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.459+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedd680 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.459+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.459+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/1808932998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aedcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.459+0000 7ffa37e5db80 1 -- 172.21.15.7:0/1808932998 >> 172.21.15.7:0/1808932998 conn(0x56351ae09170 msgr2=0x56351aed7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.461+0000 7ffa37e5db80 1 -- 172.21.15.7:0/1808932998 shutdown_connections 2024-06-13T07:02:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.461+0000 7ffa37e5db80 1 -- 172.21.15.7:0/1808932998 wait complete. 2024-06-13T07:02:16.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.461+0000 7ffa37e5db80 1 -- start start 2024-06-13T07:02:16.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.462+0000 7ffa37e5db80 1 Processor -- start 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.463+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aedd240 0x56351aee9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.463+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.463+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 0x56351aee6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.463+0000 7ffa26800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aedd240 0x56351aee9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.463+0000 7ffa25e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa25e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34030/0 (socket says 172.21.15.7:34030) 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa27200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 0x56351aee6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa27200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 0x56351aee6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33070/0 (socket says 172.21.15.7:33070) 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa25e00700 1 -- 172.21.15.7:0/487401855 learned_addr learned my addr 172.21.15.7:0/487401855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56351adcaf60 con 0x56351aedd240 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56351ae2aac0 con 0x56351aee34b0 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56351aee9e90 con 0x56351aee2b90 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.464+0000 7ffa37e5db80 10 client.0 _ll_register_callbacks cb 0x56351adc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 -- 172.21.15.7:0/487401855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 msgr2=0x56351aee6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 0x56351aee6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 -- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aedd240 msgr2=0x56351aee9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aedd240 0x56351aee9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351ad7d8d0 con 0x56351aee2b90 2024-06-13T07:02:16.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa25e00700 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa200cc3d0 tx=0x7ffa2009b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cef3c12891d7f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.465+0000 7ffa10c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:16.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.466+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20019030 con 0x56351aee2b90 2024-06-13T07:02:16.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.466+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ffa2009f120 con 0x56351aee2b90 2024-06-13T07:02:16.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.466+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18011d10 con 0x56351aee2b90 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.467+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffa200a3da0 con 0x56351aee2b90 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.469+0000 7ffa37e5db80 10 client.26275 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.469+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56351aef4650 con 0x56351aee2b90 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18011d10 con 0x56351aee2b90 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ffa200a3910 con 0x56351aee2b90 2024-06-13T07:02:16.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 10 client.26275 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 10 client.26275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 20 client.26275 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351aef6be0 0x56351aef8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 4 client.26275 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.470+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56351ae3c000 con 0x56351aef6be0 2024-06-13T07:02:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.471+0000 7ffa26800700 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351aef6be0 0x56351aef8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.471+0000 7ffa26800700 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351aef6be0 0x56351aef8fd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0e6bca8b53f84da server_cookie=3fadd6235cd4d561 in_seq=0 out_seq=0 2024-06-13T07:02:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.471+0000 7ffa10c00700 10 client.26275 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:02:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa10c00700 1 -- 172.21.15.7:0/487401855 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56351ae3c000 con 0x56351aef6be0 2024-06-13T07:02:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa10c00700 10 client.26275 handle_command_reply: tid=0 2024-06-13T07:02:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa37e5db80 1 client.26275 shutdown 2024-06-13T07:02:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 msgr2=0x56351aee6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa200cc3d0 tx=0x7ffa2009b830 comp rx=0 tx=0).stop 2024-06-13T07:02:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 shutdown_connections 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.473+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351aef6be0 0x56351aef8fd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aee34b0 0x56351aee6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aedd240 0x56351aee9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/487401855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aee6500 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 >> 172.21.15.7:0/487401855 conn(0x56351ae09170 msgr2=0x56351aed1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa10c00700 0 client.26275 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 shutdown_connections 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 1 -- 172.21.15.7:0/487401855 wait complete. 2024-06-13T07:02:16.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.474+0000 7ffa37e5db80 20 client.26275 trim_cache size 0 max 16384 2024-06-13T07:02:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.507+0000 7ffa37e5db80 1 Processor -- start 2024-06-13T07:02:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.507+0000 7ffa37e5db80 1 -- start start 2024-06-13T07:02:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.507+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aed81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aed2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b79e0 con 0x56351aee34b0 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b5210 con 0x56351aedd240 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b3060 con 0x56351aee2b90 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa26800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa26800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33084/0 (socket says 172.21.15.7:33084) 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa26800700 1 -- 172.21.15.7:0/3720262462 learned_addr learned my addr 172.21.15.7:0/3720262462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa25e00700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aed81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.508+0000 7ffa27200700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aed2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.509+0000 7ffa26800700 1 -- 172.21.15.7:0/3720262462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 msgr2=0x56351aed81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.509+0000 7ffa26800700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aed81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.509+0000 7ffa26800700 1 -- 172.21.15.7:0/3720262462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 msgr2=0x56351aed2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.509+0000 7ffa26800700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aed2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.509+0000 7ffa26800700 1 -- 172.21.15.7:0/3720262462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351aee73e0 con 0x56351aedd240 2024-06-13T07:02:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.511+0000 7ffa27200700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aed2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.512+0000 7ffa25e00700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aed81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.512+0000 7ffa26800700 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa28079720 tx=0x7ffa14025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b53e34fd3ac6ca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.512+0000 7ffa13400700 1 -- 172.21.15.7:0/3720262462 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14025db0 con 0x56351aedd240 2024-06-13T07:02:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.512+0000 7ffa13400700 1 -- 172.21.15.7:0/3720262462 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1402c040 con 0x56351aedd240 2024-06-13T07:02:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.512+0000 7ffa13400700 1 -- 172.21.15.7:0/3720262462 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14031650 con 0x56351aedd240 2024-06-13T07:02:16.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.513+0000 7ffa37e5db80 1 -- 172.21.15.7:0/3720262462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 msgr2=0x56351aedcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.513+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa28079720 tx=0x7ffa14025000 comp rx=0 tx=0).stop 2024-06-13T07:02:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 -- 172.21.15.7:0/3720262462 shutdown_connections 2024-06-13T07:02:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee34b0 0x56351aed2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aedcd00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/3720262462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee2b90 0x56351aed81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 -- 172.21.15.7:0/3720262462 >> 172.21.15.7:0/3720262462 conn(0x56351ae09170 msgr2=0x56351aed7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 -- 172.21.15.7:0/3720262462 shutdown_connections 2024-06-13T07:02:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.514+0000 7ffa37e5db80 1 -- 172.21.15.7:0/3720262462 wait complete. 2024-06-13T07:02:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.515+0000 7ffa37e5db80 1 -- start start 2024-06-13T07:02:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.515+0000 7ffa37e5db80 1 Processor -- start 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee2b90 0x56351aeef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee34b0 0x56351aeef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b36d0 con 0x56351aee2b90 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b4b40 con 0x56351aedd240 2024-06-13T07:02:16.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b3da0 con 0x56351aee34b0 2024-06-13T07:02:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.516+0000 7ffa37e5db80 10 client.0 _ll_register_callbacks cb 0x56351adc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.517+0000 7ffa26800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.517+0000 7ffa26800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33086/0 (socket says 172.21.15.7:33086) 2024-06-13T07:02:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.517+0000 7ffa26800700 1 -- 172.21.15.7:0/4161692344 learned_addr learned my addr 172.21.15.7:0/4161692344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.517+0000 7ffa27200700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee34b0 0x56351aeef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.517+0000 7ffa25e00700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee2b90 0x56351aeef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 -- 172.21.15.7:0/4161692344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee34b0 msgr2=0x56351aeef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee34b0 0x56351aeef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 -- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee2b90 msgr2=0x56351aeef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee2b90 0x56351aeef150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 -- 172.21.15.7:0/4161692344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351aeefd00 con 0x56351aedd240 2024-06-13T07:02:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa26800700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009ce0 tx=0x7ffa140038c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945bf77df240bda server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.518+0000 7ffa10c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:02:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.519+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400f460 con 0x56351aedd240 2024-06-13T07:02:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.519+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ffa1400f600 con 0x56351aedd240 2024-06-13T07:02:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.519+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1402c040 con 0x56351aedd240 2024-06-13T07:02:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.520+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffa1404c050 con 0x56351aedd240 2024-06-13T07:02:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.520+0000 7ffa37e5db80 10 client.26318 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.520+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56351af60860 con 0x56351aedd240 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1402c040 con 0x56351aedd240 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7ffa1403a3f0 con 0x56351aedd240 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 10 client.26318 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 10 client.26318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 20 client.26318 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351af60e30 0x56351af63250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 4 client.26318 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:02:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.522+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56351ae36580 con 0x56351af60e30 2024-06-13T07:02:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.525+0000 7ffa25e00700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351af60e30 0x56351af63250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.527+0000 7ffa10c00700 10 client.26318 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:02:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.527+0000 7ffa25e00700 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351af60e30 0x56351af63250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fedac7421f387847 server_cookie=fa704c0a5b632d8f in_seq=0 out_seq=0 2024-06-13T07:02:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.527+0000 7ffa10c00700 1 -- 172.21.15.7:0/4161692344 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffa200bb0b0 con 0x56351af60e30 2024-06-13T07:02:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.527+0000 7ffa10c00700 10 client.26318 handle_command_reply: tid=0 2024-06-13T07:02:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.527+0000 7ffa37e5db80 1 client.26318 shutdown 2024-06-13T07:02:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 msgr2=0x56351aefeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009ce0 tx=0x7ffa140038c0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 shutdown_connections 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56351aee34b0 0x56351aeef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x56351af60e30 0x56351af63250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56351aedd240 0x56351aefeea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 --2- 172.21.15.7:0/4161692344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56351aee2b90 0x56351aeef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 >> 172.21.15.7:0/4161692344 conn(0x56351ae09170 msgr2=0x56351aee3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.528+0000 7ffa10c00700 0 client.26318 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:02:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.529+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 shutdown_connections 2024-06-13T07:02:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.529+0000 7ffa37e5db80 1 -- 172.21.15.7:0/4161692344 wait complete. 2024-06-13T07:02:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.529+0000 7ffa37e5db80 20 client.26318 trim_cache size 0 max 16384 2024-06-13T07:02:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:02:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.530+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 msgr2=0x7ffa0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.530+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 0x7ffa0c0a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa140098e0 tx=0x7ffa14005fb0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.530+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 msgr2=0x7ffa2819fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.530+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007f10 tx=0x7ffa2000d3b0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 shutdown_connections 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa281a3df0 0x7ffa281a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2806a6e0 0x7ffa2819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa0c0a2cf0 0x7ffa0c0a51b0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffa140098e0 tx=0x7ffa14005fb0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 --2- 172.21.15.7:0/3654644105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa28104ed0 0x7ffa2819fa60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 >> 172.21.15.7:0/3654644105 conn(0x7ffa28078340 msgr2=0x7ffa28100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.531+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 shutdown_connections 2024-06-13T07:02:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:16.532+0000 7ffa2d400700 1 -- 172.21.15.7:0/3654644105 wait complete. 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54a3e00640 1 -- 172.21.15.7:0/1773038723 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdedf2fea0 con 0x55fdee0455c0 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54a3e00640 10 client.15837 handle_command_reply: tid=0 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 client.15837 shutdown 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 msgr2=0x55fdedfd3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfd3140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f54b8033b60 tx=0x7f54b8000f40 comp rx=0 tx=0).stop 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 shutdown_connections 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55fdee0455c0 0x55fdee0479e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fdedfca760 0x55fdee043740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdedfc55f0 0x55fdedfd3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 --2- 172.21.15.7:0/1773038723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fdedfc4c00 0x55fdedfd3140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 >> 172.21.15.7:0/1773038723 conn(0x55fdedf8cfb0 msgr2=0x55fdedfbf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.577+0000 7f54a3e00640 0 client.15837 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.578+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 shutdown_connections 2024-06-13T07:02:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.578+0000 7f54c7dc4740 1 -- 172.21.15.7:0/1773038723 wait complete. 2024-06-13T07:02:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.578+0000 7f54c7dc4740 20 client.15837 trim_cache size 0 max 16384 2024-06-13T07:02:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:02:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:02:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:02:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 msgr2=0x7f54a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 0x7f54a40a5170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cc560 tx=0x7f54b00bb040 comp rx=0 tx=0).stop 2024-06-13T07:02:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 msgr2=0x7f54c006be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b000d770 tx=0x7f54b000dc30 comp rx=0 tx=0).stop 2024-06-13T07:02:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 shutdown_connections 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f54a40a2cb0 0x7f54a40a5170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c01057e0 0x7f54c00733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c0104e20 0x7f54c006c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 --2- 172.21.15.7:0/814974734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c0103c30 0x7f54c006be90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.579+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 >> 172.21.15.7:0/814974734 conn(0x7f54c00ff1c0 msgr2=0x7f54c0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.581+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 shutdown_connections 2024-06-13T07:02:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:16.581+0000 7f54c5000640 1 -- 172.21.15.7:0/814974734 wait complete. 2024-06-13T07:02:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:17 smithi189 ceph-mon[36616]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 500 op/s 2024-06-13T07:02:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:17 smithi175 ceph-mon[36886]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 500 op/s 2024-06-13T07:02:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:17 smithi007 ceph-mon[32077]: pgmap v696: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 500 op/s 2024-06-13T07:02:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 375.0 s, 601.3 tps, lat 165.608 ms stddev 245.508 2024-06-13T07:02:17.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:02:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:19.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:19 smithi007 ceph-mon[32077]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 645 op/s 2024-06-13T07:02:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:19 smithi175 ceph-mon[36886]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 645 op/s 2024-06-13T07:02:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:19 smithi189 ceph-mon[36616]: pgmap v697: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 MiB/s wr, 645 op/s 2024-06-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 -- 172.21.15.7:0/2778093148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 msgr2=0x7f06901115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 --2- 172.21.15.7:0/2778093148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 0x7f06901115a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0688009f90 tx=0x7f068800b330 comp rx=0 tx=0).stop 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 -- 172.21.15.7:0/2778093148 shutdown_connections 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 --2- 172.21.15.7:0/2778093148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 0x7f06901115a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 --2- 172.21.15.7:0/2778093148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 --2- 172.21.15.7:0/2778093148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 -- 172.21.15.7:0/2778093148 >> 172.21.15.7:0/2778093148 conn(0x7f06901001c0 msgr2=0x7f06901025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 -- 172.21.15.7:0/2778093148 shutdown_connections 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.021+0000 7f0697c00700 1 -- 172.21.15.7:0/2778093148 wait complete. 2024-06-13T07:02:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.023+0000 7f0697c00700 1 Processor -- start 2024-06-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.023+0000 7f0697c00700 1 -- start start 2024-06-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.023+0000 7f0697c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690111340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.023+0000 7f0697c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 0x7f069010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.023+0000 7f0697c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0697c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0690114a60 con 0x7f06901046d0 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0697c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06901148e0 con 0x7f0690105090 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0697c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0690114be0 con 0x7f069010a6d0 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34124/0 (socket says 172.21.15.7:34124) 2024-06-13T07:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 -- 172.21.15.7:0/1446299979 learned_addr learned my addr 172.21.15.7:0/1446299979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695200700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 -- 172.21.15.7:0/1446299979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 msgr2=0x7f069010e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 0x7f069010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 -- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 msgr2=0x7f0690111340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690111340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695c00700 1 -- 172.21.15.7:0/1446299979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069010f230 con 0x7f069010a6d0 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.024+0000 7f0695200700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.025+0000 7f0695c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0688009550 tx=0x7f0688009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.025+0000 7f068d600700 1 -- 172.21.15.7:0/1446299979 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688021070 con 0x7f069010a6d0 2024-06-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.025+0000 7f068d600700 1 -- 172.21.15.7:0/1446299979 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0688007b40 con 0x7f069010a6d0 2024-06-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.025+0000 7f068d600700 1 -- 172.21.15.7:0/1446299979 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880052a0 con 0x7f069010a6d0 2024-06-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.025+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901a0f40 con 0x7f069010a6d0 2024-06-13T07:02:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.026+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901a1400 con 0x7f069010a6d0 2024-06-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.033+0000 7f068d600700 1 -- 172.21.15.7:0/1446299979 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0688024070 con 0x7f069010a6d0 2024-06-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.034+0000 7f06a25e1b80 1 Processor -- start 2024-06-13T07:02:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.034+0000 7f06a25e1b80 1 -- start start 2024-06-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.034+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d748a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d657a10 con 0x55ee5d7485b0 2024-06-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d69ab00 con 0x55ee5d753300 2024-06-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d6558f0 con 0x55ee5d6c4e40 2024-06-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f068fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f068fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34134/0 (socket says 172.21.15.7:34134) 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f068fe00700 1 -- 172.21.15.7:0/2359399603 learned_addr learned my addr 172.21.15.7:0/2359399603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f068fe00700 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.035+0000 7f0695200700 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d748a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068d600700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 0x7f06780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068fe00700 1 -- 172.21.15.7:0/2359399603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 msgr2=0x55ee5d7556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068fe00700 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068fe00700 1 -- 172.21.15.7:0/2359399603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 msgr2=0x55ee5d748a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068fe00700 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d748a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f0695c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 0x7f06780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068fe00700 1 -- 172.21.15.7:0/2359399603 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5d642f90 con 0x55ee5d6c4e40 2024-06-13T07:02:21.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.036+0000 7f068d600700 1 -- 172.21.15.7:0/1446299979 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f06880d02b0 con 0x7f069010a6d0 2024-06-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.038+0000 7f068fe00700 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0680013860 tx=0x7f06800096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6683d69518ebf778 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.038+0000 7f067e000700 1 -- 172.21.15.7:0/2359399603 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800132d0 con 0x55ee5d6c4e40 2024-06-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.038+0000 7f067e000700 1 -- 172.21.15.7:0/2359399603 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0680004510 con 0x55ee5d6c4e40 2024-06-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.038+0000 7f067e000700 1 -- 172.21.15.7:0/2359399603 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680005020 con 0x55ee5d6c4e40 2024-06-13T07:02:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.038+0000 7f0695c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 0x7f06780a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0688009790 tx=0x7f06880674b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 -- 172.21.15.7:0/2359399603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 msgr2=0x55ee5d748070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0680013860 tx=0x7f06800096c0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 -- 172.21.15.7:0/2359399603 shutdown_connections 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d748070 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/2359399603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d748a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.039+0000 7f06a25e1b80 1 -- 172.21.15.7:0/2359399603 >> 172.21.15.7:0/2359399603 conn(0x55ee5d675170 msgr2=0x55ee5d7444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.040+0000 7f06a25e1b80 1 -- 172.21.15.7:0/2359399603 shutdown_connections 2024-06-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.040+0000 7f06a25e1b80 1 -- 172.21.15.7:0/2359399603 wait complete. 2024-06-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.040+0000 7f06a25e1b80 1 -- start start 2024-06-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.041+0000 7f06a25e1b80 1 Processor -- start 2024-06-13T07:02:21.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.041+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 0x55ee5d752aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.041+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d74fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.041+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.041+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d636f60 con 0x55ee5d6c4e40 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d696ac0 con 0x55ee5d753300 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee5d755f60 con 0x55ee5d7485b0 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 0x55ee5d752aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f06a25e1b80 10 client.0 _ll_register_callbacks cb 0x55ee5d634720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 0x55ee5d752aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34584/0 (socket says 172.21.15.7:34584) 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 -- 172.21.15.7:0/1932713690 learned_addr learned my addr 172.21.15.7:0/1932713690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:21.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f0695200700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d74fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 -- 172.21.15.7:0/1932713690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 msgr2=0x55ee5d74fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d74fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 -- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 msgr2=0x55ee5d752aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 0x55ee5d752aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.042+0000 7f068fe00700 1 -- 172.21.15.7:0/1932713690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5d5e98d0 con 0x55ee5d753300 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f068fe00700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7502e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f06800092a0 tx=0x7f0680009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbbfdb1972a7e17 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f0672a00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680005000 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f06800051a0 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0680004510 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.043+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0680032050 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 10 client.26327 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ee5d7606b0 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0680004510 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f068001ec70 con 0x55ee5d753300 2024-06-13T07:02:21.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 10 client.26327 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 10 client.26327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 20 client.26327 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d762be0 0x55ee5d764fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 4 client.26327 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.044+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee5d6a8000 con 0x55ee5d762be0 2024-06-13T07:02:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.045+0000 7f0695200700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d762be0 0x55ee5d764fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.046+0000 7f0695200700 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d762be0 0x55ee5d764fd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d85962524746057 server_cookie=6dc9d04701ec4262 in_seq=0 out_seq=0 2024-06-13T07:02:21.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.046+0000 7f0672a00700 10 client.26327 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f0672a00700 1 -- 172.21.15.7:0/1932713690 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee5d6a8000 con 0x55ee5d762be0 2024-06-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f0672a00700 10 client.26327 handle_command_reply: tid=0 2024-06-13T07:02:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 client.26327 shutdown 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 msgr2=0x55ee5d7502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7502e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f06800092a0 tx=0x7f0680009ac0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 shutdown_connections 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d7502e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.048+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d6c4e40 0x55ee5d752aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d762be0 0x55ee5d764fd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/1932713690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d74fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f0672a00700 0 client.26327 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:02:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 >> 172.21.15.7:0/1932713690 conn(0x55ee5d675170 msgr2=0x55ee5d743740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 shutdown_connections 2024-06-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.049+0000 7f06a25e1b80 1 -- 172.21.15.7:0/1932713690 wait complete. 2024-06-13T07:02:21.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.050+0000 7f06a25e1b80 20 client.26327 trim_cache size 0 max 16384 2024-06-13T07:02:21.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.078+0000 7f06a25e1b80 1 Processor -- start 2024-06-13T07:02:21.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.078+0000 7f06a25e1b80 1 -- start start 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d7525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d753300 0x55ee5d74afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06901bf750 con 0x55ee5d753300 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06901bcf80 con 0x55ee5d6c4e40 2024-06-13T07:02:21.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.079+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06901badd0 con 0x55ee5d7485b0 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f068fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f068fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48262/0 (socket says 172.21.15.7:48262) 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f068fe00700 1 -- 172.21.15.7:0/4954793 learned_addr learned my addr 172.21.15.7:0/4954793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f0695200700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d7525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f068fe00700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d753300 0x55ee5d74afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.080+0000 7f0695200700 1 -- 172.21.15.7:0/4954793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 msgr2=0x55ee5d7503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.081+0000 7f0695200700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.081+0000 7f0695200700 1 -- 172.21.15.7:0/4954793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d753300 msgr2=0x55ee5d74afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.081+0000 7f0695200700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d753300 0x55ee5d74afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.081+0000 7f0695200700 1 -- 172.21.15.7:0/4954793 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5d748fd0 con 0x55ee5d7485b0 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.082+0000 7f068fe00700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:21.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.082+0000 7f0695200700 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d7525e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ee5d765680 tx=0x7f0684014750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a750e302c3ec97 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.083+0000 7f067d600700 1 -- 172.21.15.7:0/4954793 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684014ba0 con 0x55ee5d7485b0 2024-06-13T07:02:21.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.083+0000 7f067d600700 1 -- 172.21.15.7:0/4954793 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0684004510 con 0x55ee5d7485b0 2024-06-13T07:02:21.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.083+0000 7f067d600700 1 -- 172.21.15.7:0/4954793 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684003db0 con 0x55ee5d7485b0 2024-06-13T07:02:21.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 -- 172.21.15.7:0/4954793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 msgr2=0x55ee5d7525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d7525e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ee5d765680 tx=0x7f0684014750 comp rx=0 tx=0).stop 2024-06-13T07:02:21.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 -- 172.21.15.7:0/4954793 shutdown_connections 2024-06-13T07:02:21.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d753300 0x55ee5d74afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d6c4e40 0x55ee5d7503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/4954793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d7485b0 0x55ee5d7525e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 -- 172.21.15.7:0/4954793 >> 172.21.15.7:0/4954793 conn(0x55ee5d675170 msgr2=0x55ee5d75e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 -- 172.21.15.7:0/4954793 shutdown_connections 2024-06-13T07:02:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.084+0000 7f06a25e1b80 1 -- 172.21.15.7:0/4954793 wait complete. 2024-06-13T07:02:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.085+0000 7f06a25e1b80 1 -- start start 2024-06-13T07:02:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.085+0000 7f06a25e1b80 1 Processor -- start 2024-06-13T07:02:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.086+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d75b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d74d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06901bb440 con 0x55ee5d7485b0 2024-06-13T07:02:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06901bc8b0 con 0x55ee5d753300 2024-06-13T07:02:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06901bbb10 con 0x55ee5d6c4e40 2024-06-13T07:02:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.087+0000 7f06a25e1b80 10 client.0 _ll_register_callbacks cb 0x55ee5d634720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.089+0000 7f068fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.089+0000 7f068fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34162/0 (socket says 172.21.15.7:34162) 2024-06-13T07:02:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.089+0000 7f068fe00700 1 -- 172.21.15.7:0/345158175 learned_addr learned my addr 172.21.15.7:0/345158175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.089+0000 7f0695200700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d75b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d74d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 -- 172.21.15.7:0/345158175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 msgr2=0x55ee5d74d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d74d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 -- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 msgr2=0x55ee5d75b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d75b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.090+0000 7f068fe00700 1 -- 172.21.15.7:0/345158175 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5d74d580 con 0x55ee5d6c4e40 2024-06-13T07:02:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.091+0000 7f068fe00700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0680011630 tx=0x7f0680009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec99cd191872bf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.091+0000 7f0672a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.094+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06800119d0 con 0x55ee5d6c4e40 2024-06-13T07:02:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.094+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0680011b70 con 0x55ee5d6c4e40 2024-06-13T07:02:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.094+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0684004510 con 0x55ee5d6c4e40 2024-06-13T07:02:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.095+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0680046050 con 0x55ee5d6c4e40 2024-06-13T07:02:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.095+0000 7f06a25e1b80 10 client.26296 fetch_fsmap learned FSMap version 36 2024-06-13T07:02:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.095+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ee5d769fb0 con 0x55ee5d6c4e40 2024-06-13T07:02:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0684004510 con 0x55ee5d6c4e40 2024-06-13T07:02:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f068001e490 con 0x55ee5d6c4e40 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 36 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 10 client.26296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 20 client.26296 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d7cd6e0 0x55ee5d7cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 4 client.26296 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:02:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.096+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee5d6a2580 con 0x55ee5d7cd6e0 2024-06-13T07:02:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.098+0000 7f0695200700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d7cd6e0 0x55ee5d7cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.099+0000 7f0695200700 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d7cd6e0 0x55ee5d7cfad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dc3df4bf5e428ce server_cookie=6f82574aee5a574e in_seq=0 out_seq=0 2024-06-13T07:02:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.099+0000 7f0672a00700 10 client.26296 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:02:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.100+0000 7f0672a00700 1 -- 172.21.15.7:0/345158175 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee5d6a2580 con 0x55ee5d7cd6e0 2024-06-13T07:02:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.100+0000 7f0672a00700 10 client.26296 handle_command_reply: tid=0 2024-06-13T07:02:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.100+0000 7f06a25e1b80 1 client.26296 shutdown 2024-06-13T07:02:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 msgr2=0x55ee5d75b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0680011630 tx=0x7f0680009c70 comp rx=0 tx=0).stop 2024-06-13T07:02:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 shutdown_connections 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee5d753300 0x55ee5d74d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55ee5d7cd6e0 0x55ee5d7cfad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee5d6c4e40 0x55ee5d75b1f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 --2- 172.21.15.7:0/345158175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee5d7485b0 0x55ee5d75b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.101+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 >> 172.21.15.7:0/345158175 conn(0x55ee5d675170 msgr2=0x55ee5d75e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.102+0000 7f0672a00700 0 client.26296 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:02:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.102+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 shutdown_connections 2024-06-13T07:02:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.102+0000 7f06a25e1b80 1 -- 172.21.15.7:0/345158175 wait complete. 2024-06-13T07:02:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.102+0000 7f06a25e1b80 20 client.26296 trim_cache size 0 max 16384 2024-06-13T07:02:21.102 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:02:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.103+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 msgr2=0x7f06780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.103+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 0x7f06780a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0688009790 tx=0x7f06880674b0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.103+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 msgr2=0x7f069010e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.103+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0688009550 tx=0x7f0688009ba0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 shutdown_connections 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f069010a6d0 0x7f069010e930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06901046d0 0x7f0690111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f06780a2d70 0x7f06780a5230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 --2- 172.21.15.7:0/1446299979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0690105090 0x7f069010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 >> 172.21.15.7:0/1446299979 conn(0x7f06901001c0 msgr2=0x7f0690100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 shutdown_connections 2024-06-13T07:02:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:21.104+0000 7f0697c00700 1 -- 172.21.15.7:0/1446299979 wait complete. 2024-06-13T07:02:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:21 smithi175 ceph-mon[36886]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.0 MiB/s wr, 489 op/s 2024-06-13T07:02:21.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:21 smithi007 ceph-mon[32077]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.0 MiB/s wr, 489 op/s 2024-06-13T07:02:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:21 smithi189 ceph-mon[36616]: pgmap v698: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.0 MiB/s wr, 489 op/s 2024-06-13T07:02:21.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:02:22.540 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 380.0 s, 479.9 tps, lat 209.469 ms stddev 256.892 2024-06-13T07:02:23.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:23 smithi175 ceph-mon[36886]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2024-06-13T07:02:23.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:23 smithi007 ceph-mon[32077]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2024-06-13T07:02:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:23 smithi189 ceph-mon[36616]: pgmap v699: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2024-06-13T07:02:25.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:25 smithi175 ceph-mon[36886]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 620 op/s 2024-06-13T07:02:25.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:25 smithi007 ceph-mon[32077]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 620 op/s 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.442+0000 7eff05200700 1 -- 172.21.15.7:0/2591116019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 msgr2=0x7eff0010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.442+0000 7eff05200700 1 --2- 172.21.15.7:0/2591116019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0010aa80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efef0009990 tx=0x7efef0009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 -- 172.21.15.7:0/2591116019 shutdown_connections 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 --2- 172.21.15.7:0/2591116019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 0x7eff00111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 --2- 172.21.15.7:0/2591116019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0010aa80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 --2- 172.21.15.7:0/2591116019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff00104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 -- 172.21.15.7:0/2591116019 >> 172.21.15.7:0/2591116019 conn(0x7eff00100170 msgr2=0x7eff00102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 -- 172.21.15.7:0/2591116019 shutdown_connections 2024-06-13T07:02:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.443+0000 7eff05200700 1 -- 172.21.15.7:0/2591116019 wait complete. 2024-06-13T07:02:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.444+0000 7eff05200700 1 Processor -- start 2024-06-13T07:02:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.444+0000 7eff05200700 1 -- start start 2024-06-13T07:02:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff0019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 0x7eff0019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff00114a40 con 0x7eff001046a0 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff001148c0 con 0x7eff00105060 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7eff05200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff00114bc0 con 0x7eff0010a6a0 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7efefde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff0019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7efefde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff0019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34620/0 (socket says 172.21.15.7:34620) 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7efefe800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7efefde00700 1 -- 172.21.15.7:0/2393136177 learned_addr learned my addr 172.21.15.7:0/2393136177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.445+0000 7efefd400700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 0x7eff0019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 -- 172.21.15.7:0/2393136177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 msgr2=0x7eff0019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 0x7eff0019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 -- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 msgr2=0x7eff0019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff0019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 -- 172.21.15.7:0/2393136177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0019fe50 con 0x7eff0010a6a0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efefe800700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0019f710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efef800efc0 tx=0x7efef800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.446+0000 7efef6a00700 1 -- 172.21.15.7:0/2393136177 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef80092e0 con 0x7eff0010a6a0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.447+0000 7efef6a00700 1 -- 172.21.15.7:0/2393136177 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef8004bb0 con 0x7eff0010a6a0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.447+0000 7efef6a00700 1 -- 172.21.15.7:0/2393136177 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8004280 con 0x7eff0010a6a0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.447+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff001a00e0 con 0x7eff0010a6a0 2024-06-13T07:02:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.447+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff001a79c0 con 0x7eff0010a6a0 2024-06-13T07:02:25.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.452+0000 7efef6a00700 1 -- 172.21.15.7:0/2393136177 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efef801b030 con 0x7eff0010a6a0 2024-06-13T07:02:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.453+0000 7efef6a00700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 0x7efee40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.453+0000 7efef6a00700 1 -- 172.21.15.7:0/2393136177 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efef80cee50 con 0x7eff0010a6a0 2024-06-13T07:02:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.454+0000 7efefde00700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 0x7efee40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.454+0000 7eff0fc08b80 1 Processor -- start 2024-06-13T07:02:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.454+0000 7eff0fc08b80 1 -- start start 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e537095e40 0x55e537119070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 0x55e53712ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e537028a10 con 0x55e537095e40 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e53706bb00 con 0x55e5371195b0 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e5370268f0 con 0x55e53712cb10 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7efefde00700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 0x7efee40a50e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efef0009750 tx=0x7efef000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7efefd400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7efefd400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48308/0 (socket says 172.21.15.7:48308) 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7efefd400700 1 -- 172.21.15.7:0/4212817533 learned_addr learned my addr 172.21.15.7:0/4212817533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.455+0000 7efefe800700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 0x55e53712ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefde00700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e537095e40 0x55e537119070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 -- 172.21.15.7:0/4212817533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 msgr2=0x55e53712ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 0x55e53712ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 -- 172.21.15.7:0/4212817533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e537095e40 msgr2=0x55e537119070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e537095e40 0x55e537119070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 -- 172.21.15.7:0/4212817533 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e537013f90 con 0x55e5371195b0 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefe800700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 0x55e53712ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.456+0000 7efefd400700 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d20 tx=0x7efeec011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8beb3f65d373e08d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.457+0000 7efeeb400700 1 -- 172.21.15.7:0/4212817533 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec01e070 con 0x55e5371195b0 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.457+0000 7efeeb400700 1 -- 172.21.15.7:0/4212817533 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeec019b10 con 0x55e5371195b0 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.457+0000 7efeeb400700 1 -- 172.21.15.7:0/4212817533 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec00ebc0 con 0x55e5371195b0 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.457+0000 7eff0fc08b80 1 -- 172.21.15.7:0/4212817533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 msgr2=0x55e537119a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.457+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d20 tx=0x7efeec011720 comp rx=0 tx=0).stop 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 -- 172.21.15.7:0/4212817533 shutdown_connections 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e53712cb10 0x55e53712ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e537095e40 0x55e537119070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/4212817533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e537119a30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 -- 172.21.15.7:0/4212817533 >> 172.21.15.7:0/4212817533 conn(0x55e537046170 msgr2=0x55e5371154b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 -- 172.21.15.7:0/4212817533 shutdown_connections 2024-06-13T07:02:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.458+0000 7eff0fc08b80 1 -- 172.21.15.7:0/4212817533 wait complete. 2024-06-13T07:02:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.461+0000 7eff0fc08b80 1 -- start start 2024-06-13T07:02:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 Processor -- start 2024-06-13T07:02:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537129540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537129a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e537007f60 con 0x55e53712cb10 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e537067ac0 con 0x55e537095e40 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.462+0000 7eff0fc08b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e53712f7d0 con 0x55e5371195b0 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7eff0fc08b80 10 client.0 _ll_register_callbacks cb 0x55e537005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7efefde00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7efefde00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48322/0 (socket says 172.21.15.7:48322) 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7efefde00700 1 -- 172.21.15.7:0/2275670137 learned_addr learned my addr 172.21.15.7:0/2275670137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7efefe800700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537129a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.463+0000 7efefd400700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537129540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 -- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 msgr2=0x55e53712c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 -- 172.21.15.7:0/2275670137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 msgr2=0x55e537129a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537129a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 -- 172.21.15.7:0/2275670137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e536fba8d0 con 0x55e5371195b0 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefde00700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefe800700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537129a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efefd400700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537129540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d20 tx=0x7efeec0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8e55572ed07eacd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.464+0000 7efedfe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec01e040 con 0x55e5371195b0 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efeec004030 con 0x55e5371195b0 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeec019b10 con 0x55e5371195b0 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeec032050 con 0x55e5371195b0 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7eff0fc08b80 10 client.26305 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:25.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.465+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e537139e20 con 0x55e5371195b0 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeec019b10 con 0x55e5371195b0 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7efeec00e7c0 con 0x55e5371195b0 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 10 client.26305 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 20 client.26305 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e53713c3a0 0x55e53713e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 4 client.26305 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e537079000 con 0x55e53713c3a0 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.466+0000 7efefde00700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e53713c3a0 0x55e53713e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.467+0000 7efedfe00700 10 client.26305 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.467+0000 7efefde00700 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e53713c3a0 0x55e53713e790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1393e83c58986cb9 server_cookie=2da48262e0470bbc in_seq=0 out_seq=0 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7efedfe00700 1 -- 172.21.15.7:0/2275670137 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e537079000 con 0x55e53713c3a0 2024-06-13T07:02:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7efedfe00700 10 client.26305 handle_command_reply: tid=0 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 client.26305 shutdown 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 msgr2=0x55e537129540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537129540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d20 tx=0x7efeec0058e0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 shutdown_connections 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e53713c3a0 0x55e53713e790 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.468+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537129a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e53712c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/2275670137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537129540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 >> 172.21.15.7:0/2275670137 conn(0x55e537046170 msgr2=0x55e537114740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7efedfe00700 0 client.26305 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 shutdown_connections 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.469+0000 7eff0fc08b80 1 -- 172.21.15.7:0/2275670137 wait complete. 2024-06-13T07:02:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.470+0000 7eff0fc08b80 20 client.26305 trim_cache size 0 max 16384 2024-06-13T07:02:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.494+0000 7eff0fc08b80 1 Processor -- start 2024-06-13T07:02:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.495+0000 7eff0fc08b80 1 -- start start 2024-06-13T07:02:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.496+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.496+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e5371286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.496+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e5371145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.496+0000 7efefde00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.496+0000 7efefde00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34196/0 (socket says 172.21.15.7:34196) 2024-06-13T07:02:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.497+0000 7efefd400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e5371286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.497+0000 7efefde00700 1 -- 172.21.15.7:0/1644632886 learned_addr learned my addr 172.21.15.7:0/1644632886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.497+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff001ae950 con 0x55e53712cb10 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.497+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff001ac1b0 con 0x55e5371195b0 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.497+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff001aa000 con 0x55e537095e40 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.498+0000 7efefde00700 1 -- 172.21.15.7:0/1644632886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 msgr2=0x55e5371286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.498+0000 7efefde00700 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e5371286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.498+0000 7efefde00700 1 -- 172.21.15.7:0/1644632886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 msgr2=0x55e5371145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.498+0000 7efefde00700 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e5371145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.498+0000 7efefde00700 1 -- 172.21.15.7:0/1644632886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53712f270 con 0x55e537095e40 2024-06-13T07:02:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.501+0000 7efefde00700 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55e53713ee40 tx=0x7efef0021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a108c8d58b520096 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.501+0000 7efeeaa00700 1 -- 172.21.15.7:0/1644632886 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0035020 con 0x55e537095e40 2024-06-13T07:02:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.505+0000 7efeeaa00700 1 -- 172.21.15.7:0/1644632886 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef0005670 con 0x55e537095e40 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.505+0000 7efeeaa00700 1 -- 172.21.15.7:0/1644632886 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef000fd40 con 0x55e537095e40 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 msgr2=0x55e537115620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55e53713ee40 tx=0x7efef0021000 comp rx=0 tx=0).stop 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 shutdown_connections 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e5371145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e537095e40 0x55e537115620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1644632886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e5371195b0 0x55e5371286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.506+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 >> 172.21.15.7:0/1644632886 conn(0x55e537046170 msgr2=0x55e537113eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.508+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 shutdown_connections 2024-06-13T07:02:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.509+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1644632886 wait complete. 2024-06-13T07:02:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.510+0000 7eff0fc08b80 1 -- start start 2024-06-13T07:02:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.510+0000 7eff0fc08b80 1 Processor -- start 2024-06-13T07:02:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.511+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.511+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537128770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.511+0000 7eff0fc08b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537128cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7efefde00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7efefde00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48348/0 (socket says 172.21.15.7:48348) 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7efefd400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537128770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7efefd400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537128770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34200/0 (socket says 172.21.15.7:34200) 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7efefde00700 1 -- 172.21.15.7:0/1659321346 learned_addr learned my addr 172.21.15.7:0/1659321346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff001aa670 con 0x55e53712cb10 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff001abae0 con 0x55e537095e40 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff001aad40 con 0x55e5371195b0 2024-06-13T07:02:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.512+0000 7eff0fc08b80 10 client.0 _ll_register_callbacks cb 0x55e537005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 -- 172.21.15.7:0/1659321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 msgr2=0x55e537128770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537128770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 -- 172.21.15.7:0/1659321346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 msgr2=0x55e537128cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537128cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e537133b10 con 0x55e537095e40 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efefde00700 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efef0005a60 tx=0x7efef000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac41ac7e664355f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.513+0000 7efedfe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:02:25.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.514+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef003e020 con 0x55e537095e40 2024-06-13T07:02:25.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.514+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efef0003c60 con 0x55e537095e40 2024-06-13T07:02:25.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.514+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef0005670 con 0x55e537095e40 2024-06-13T07:02:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.516+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efef0035040 con 0x55e537095e40 2024-06-13T07:02:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.516+0000 7eff0fc08b80 10 client.26348 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.516+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e537134e10 con 0x55e537095e40 2024-06-13T07:02:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.516+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef0005670 con 0x55e537095e40 2024-06-13T07:02:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7efef003aa20 con 0x55e537095e40 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 10 client.26348 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 20 client.26348 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e5371a6ea0 0x55e5371a9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 4 client.26348 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.517+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e537073580 con 0x55e5371a6ea0 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.519+0000 7efefd400700 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e5371a6ea0 0x55e5371a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.519+0000 7efefd400700 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e5371a6ea0 0x55e5371a9290 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e816295bed95c8cb server_cookie=10069763815fff5a in_seq=0 out_seq=0 2024-06-13T07:02:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.519+0000 7efedfe00700 10 client.26348 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.520+0000 7efedfe00700 1 -- 172.21.15.7:0/1659321346 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e537073580 con 0x55e5371a6ea0 2024-06-13T07:02:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.520+0000 7efedfe00700 10 client.26348 handle_command_reply: tid=0 2024-06-13T07:02:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.520+0000 7eff0fc08b80 1 client.26348 shutdown 2024-06-13T07:02:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.521+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 msgr2=0x55e537128230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.521+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efef0005a60 tx=0x7efef000b410 comp rx=0 tx=0).stop 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 shutdown_connections 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e5371a6ea0 0x55e5371a9290 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53712cb10 0x55e537128cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e537095e40 0x55e537128230 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 --2- 172.21.15.7:0/1659321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5371195b0 0x55e537128770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7efedfe00700 0 client.26348 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 >> 172.21.15.7:0/1659321346 conn(0x55e537046170 msgr2=0x55e53711b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.522+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 shutdown_connections 2024-06-13T07:02:25.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.523+0000 7eff0fc08b80 1 -- 172.21.15.7:0/1659321346 wait complete. 2024-06-13T07:02:25.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.523+0000 7eff0fc08b80 20 client.26348 trim_cache size 0 max 16384 2024-06-13T07:02:25.523 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:02:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 msgr2=0x7efee40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 0x7efee40a50e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efef0009750 tx=0x7efef000b280 comp rx=0 tx=0).stop 2024-06-13T07:02:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 msgr2=0x7eff0019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0019f710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efef800efc0 tx=0x7efef800c3a0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 shutdown_connections 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff0010a6a0 0x7eff0019f710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff001046a0 0x7eff0019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efee40a2c20 0x7efee40a50e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 --2- 172.21.15.7:0/2393136177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff00105060 0x7eff0019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 >> 172.21.15.7:0/2393136177 conn(0x7eff00100170 msgr2=0x7eff00100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 shutdown_connections 2024-06-13T07:02:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:25.525+0000 7eff05200700 1 -- 172.21.15.7:0/2393136177 wait complete. 2024-06-13T07:02:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:25 smithi189 ceph-mon[36616]: pgmap v700: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 620 op/s 2024-06-13T07:02:26.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:26 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:26.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:26 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:26.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:26 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:02:27.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:27 smithi007 ceph-mon[32077]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2024-06-13T07:02:27.518 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:27 smithi175 ceph-mon[36886]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2024-06-13T07:02:27.558 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:02:27.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:02:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:27 smithi189 ceph-mon[36616]: pgmap v701: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2024-06-13T07:02:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 385.0 s, 545.8 tps, lat 184.579 ms stddev 221.364 2024-06-13T07:02:28.181 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:29 smithi007 ceph-mon[32077]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-06-13T07:02:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:29 smithi189 ceph-mon[36616]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-06-13T07:02:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:29 smithi175 ceph-mon[36886]: pgmap v702: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-06-13T07:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.790+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4283705551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec100270 msgr2=0x7f6fec1026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4283705551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec100270 0x7f6fec1026a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc01a910 tx=0x7f6fdc03b470 comp rx=0 tx=0).stop 2024-06-13T07:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4283705551 shutdown_connections 2024-06-13T07:02:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4283705551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec105590 0x7f6fec105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4283705551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec102be0 0x7f6fec105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4283705551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec100270 0x7f6fec1026a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4283705551 >> 172.21.15.7:0/4283705551 conn(0x7f6fec0fbe70 msgr2=0x7f6fec0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4283705551 shutdown_connections 2024-06-13T07:02:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.791+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4283705551 wait complete. 2024-06-13T07:02:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.792+0000 7f6ff1e00640 1 Processor -- start 2024-06-13T07:02:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.793+0000 7f6ff1e00640 1 -- start start 2024-06-13T07:02:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.794+0000 7f6ff1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6ff1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec102be0 0x7f6fec196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6ff1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 0x7f6fec19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48370/0 (socket says 172.21.15.7:48370) 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1094f0 con 0x7f6fec105590 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6feaa00640 1 -- 172.21.15.7:0/4262204160 learned_addr learned my addr 172.21.15.7:0/4262204160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec109370 con 0x7f6fec100270 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.795+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec109670 con 0x7f6fec102be0 2024-06-13T07:02:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feb400640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 0x7f6fec19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 -- 172.21.15.7:0/4262204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec102be0 msgr2=0x7f6fec196010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec102be0 0x7f6fec196010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 -- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 msgr2=0x7f6fec19d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 0x7f6fec19d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec19d6f0 con 0x7f6fec100270 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feb400640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 0x7f6fec19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.796+0000 7f6feaa00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc004b90 tx=0x7f6fdc004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.797+0000 7f6ff0a00640 1 -- 172.21.15.7:0/4262204160 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc04f070 con 0x7f6fec100270 2024-06-13T07:02:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.797+0000 7f6ff0a00640 1 -- 172.21.15.7:0/4262204160 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc040c70 con 0x7f6fec100270 2024-06-13T07:02:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.797+0000 7f6ff0a00640 1 -- 172.21.15.7:0/4262204160 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc046030 con 0x7f6fec100270 2024-06-13T07:02:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.797+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec19d980 con 0x7f6fec100270 2024-06-13T07:02:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.797+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec19dea0 con 0x7f6fec100270 2024-06-13T07:02:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.803+0000 7f6ff0a00640 1 -- 172.21.15.7:0/4262204160 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6fdc052070 con 0x7f6fec100270 2024-06-13T07:02:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.806+0000 7f6ff4bee740 1 Processor -- start 2024-06-13T07:02:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.809+0000 7f6ff0a00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 0x7f6fd00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.809+0000 7f6ff4bee740 1 -- start start 2024-06-13T07:02:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bac00 0x563c297bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297c0760 0x563c297c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c2971e420 con 0x563c297bac00 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c296c5c50 con 0x563c297bb5f0 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c2971a3e0 con 0x563c297c0760 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6fea000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6fea000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48380/0 (socket says 172.21.15.7:48380) 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6fea000640 1 -- 172.21.15.7:0/3265122324 learned_addr learned my addr 172.21.15.7:0/3265122324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.810+0000 7f6ff0a00640 1 -- 172.21.15.7:0/4262204160 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6fdc1006e0 con 0x7f6fec100270 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 -- 172.21.15.7:0/3265122324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297c0760 msgr2=0x563c297c2bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297c0760 0x563c297c2bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 -- 172.21.15.7:0/3265122324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bac00 msgr2=0x563c297bb020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bac00 0x563c297bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 -- 172.21.15.7:0/3265122324 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c29432210 con 0x563c297bb5f0 2024-06-13T07:02:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6feb400640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 0x7f6fd00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.811+0000 7f6fea000640 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009770 tx=0x7f6fd8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e44778ac8f81679 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.812+0000 7f6fd7e00640 1 -- 172.21.15.7:0/3265122324 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8015020 con 0x563c297bb5f0 2024-06-13T07:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.813+0000 7f6fd7e00640 1 -- 172.21.15.7:0/3265122324 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd8011420 con 0x563c297bb5f0 2024-06-13T07:02:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.813+0000 7f6fd7e00640 1 -- 172.21.15.7:0/3265122324 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80105e0 con 0x563c297bb5f0 2024-06-13T07:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.814+0000 7f6feb400640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 0x7f6fd00a51c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400ad00 tx=0x7f6fe400a370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.814+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3265122324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 msgr2=0x563c297c00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.814+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009770 tx=0x7f6fd8009c30 comp rx=0 tx=0).stop 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3265122324 shutdown_connections 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297c0760 0x563c297c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297c00f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3265122324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bac00 0x563c297bb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3265122324 >> 172.21.15.7:0/3265122324 conn(0x563c29782fb0 msgr2=0x563c297b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.815+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3265122324 shutdown_connections 2024-06-13T07:02:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.816+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3265122324 wait complete. 2024-06-13T07:02:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.817+0000 7f6ff4bee740 1 -- start start 2024-06-13T07:02:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.817+0000 7f6ff4bee740 1 Processor -- start 2024-06-13T07:02:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.821+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.821+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bb5f0 0x563c297c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.821+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c297c3710 con 0x563c297c0760 2024-06-13T07:02:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563c296d90b0 con 0x563c297bac00 2024-06-13T07:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c297c3890 con 0x563c297bb5f0 2024-06-13T07:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6ff4bee740 10 client.0 _ll_register_callbacks cb 0x563c296c4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48386/0 (socket says 172.21.15.7:48386) 2024-06-13T07:02:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.822+0000 7f6feaa00640 1 -- 172.21.15.7:0/3408857765 learned_addr learned my addr 172.21.15.7:0/3408857765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6fea000640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bb5f0 0x563c297c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feb400640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feaa00640 1 -- 172.21.15.7:0/3408857765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bb5f0 msgr2=0x563c297c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bb5f0 0x563c297c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feaa00640 1 -- 172.21.15.7:0/3408857765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 msgr2=0x563c297cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.823+0000 7f6feaa00640 1 -- 172.21.15.7:0/3408857765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c294fe620 con 0x563c297bac00 2024-06-13T07:02:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.829+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c95d0 tx=0x7f6fdc0cd300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98482a4cf2840901 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.829+0000 7f6fd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:02:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.830+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0d0fb0 con 0x563c297bac00 2024-06-13T07:02:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.830+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6fdc01a500 con 0x563c297bac00 2024-06-13T07:02:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.831+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd8011420 con 0x563c297bac00 2024-06-13T07:02:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.833+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc0d5870 con 0x563c297bac00 2024-06-13T07:02:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6ff4bee740 10 client.26357 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563c297cf910 con 0x563c297bac00 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fd8011420 con 0x563c297bac00 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f6fdc0d53e0 con 0x563c297bac00 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6ff4bee740 10 client.26357 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.836+0000 7f6ff4bee740 10 client.26357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.837+0000 7f6ff4bee740 20 client.26357 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.837+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c297d1040 0x563c297d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.837+0000 7f6ff4bee740 4 client.26357 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.837+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2972bb50 con 0x563c297d1040 2024-06-13T07:02:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.839+0000 7f6fea000640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c297d1040 0x563c297d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.840+0000 7f6fea000640 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c297d1040 0x563c297d3430 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413d7ea25394ca58 server_cookie=87e5745b32e08a3e in_seq=0 out_seq=0 2024-06-13T07:02:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.840+0000 7f6fd4c00640 10 client.26357 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.841+0000 7f6fd4c00640 1 -- 172.21.15.7:0/3408857765 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c2972bb50 con 0x563c297d1040 2024-06-13T07:02:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.841+0000 7f6fd4c00640 10 client.26357 handle_command_reply: tid=0 2024-06-13T07:02:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.841+0000 7f6ff4bee740 1 client.26357 shutdown 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.841+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 msgr2=0x563c297c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c95d0 tx=0x7f6fdc0cd300 comp rx=0 tx=0).stop 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 shutdown_connections 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c297d1040 0x563c297d3430 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bb5f0 0x563c297c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3408857765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bac00 0x563c297c5020 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 >> 172.21.15.7:0/3408857765 conn(0x563c29782fb0 msgr2=0x563c297b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6fd4c00640 0 client.26357 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.842+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 shutdown_connections 2024-06-13T07:02:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.843+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3408857765 wait complete. 2024-06-13T07:02:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.843+0000 7f6ff4bee740 20 client.26357 trim_cache size 0 max 16384 2024-06-13T07:02:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 Processor -- start 2024-06-13T07:02:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 -- start start 2024-06-13T07:02:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1aab30 con 0x563c297c0760 2024-06-13T07:02:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1a83d0 con 0x563c297bb5f0 2024-06-13T07:02:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1a6200 con 0x563c297bac00 2024-06-13T07:02:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6feb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34242/0 (socket says 172.21.15.7:34242) 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.868+0000 7f6feaa00640 1 -- 172.21.15.7:0/3378098932 learned_addr learned my addr 172.21.15.7:0/3378098932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6fea000640 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 -- 172.21.15.7:0/3378098932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 msgr2=0x563c297cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 -- 172.21.15.7:0/3378098932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 msgr2=0x563c297b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 -- 172.21.15.7:0/3378098932 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c297b7330 con 0x563c297bac00 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6feaa00640 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c9060 tx=0x7f6fdc0cb110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d7a49cbb91d2945 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6fd7400640 1 -- 172.21.15.7:0/3378098932 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0d0fb0 con 0x563c297bac00 2024-06-13T07:02:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6fd7400640 1 -- 172.21.15.7:0/3378098932 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc01a500 con 0x563c297bac00 2024-06-13T07:02:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.869+0000 7f6fd7400640 1 -- 172.21.15.7:0/3378098932 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0c93d0 con 0x563c297bac00 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3378098932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 msgr2=0x563c297b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c9060 tx=0x7f6fdc0cb110 comp rx=0 tx=0).stop 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3378098932 shutdown_connections 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297c0760 0x563c297b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297bb5f0 0x563c297cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/3378098932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297b5fc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3378098932 >> 172.21.15.7:0/3378098932 conn(0x563c29782fb0 msgr2=0x563c297c4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3378098932 shutdown_connections 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.870+0000 7f6ff4bee740 1 -- 172.21.15.7:0/3378098932 wait complete. 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.871+0000 7f6ff4bee740 1 -- start start 2024-06-13T07:02:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.871+0000 7f6ff4bee740 1 Processor -- start 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 0x563c297c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1a6890 con 0x563c297bb5f0 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1a7d00 con 0x563c297c0760 2024-06-13T07:02:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1a6f60 con 0x563c297bac00 2024-06-13T07:02:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.872+0000 7f6ff4bee740 10 client.0 _ll_register_callbacks cb 0x7f6fdc0cc740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.873+0000 7f6feb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.873+0000 7f6feb400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:48404/0 (socket says 172.21.15.7:48404) 2024-06-13T07:02:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.873+0000 7f6feb400640 1 -- 172.21.15.7:0/1880583788 learned_addr learned my addr 172.21.15.7:0/1880583788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.874+0000 7f6fea000640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 0x563c297c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.874+0000 7f6feaa00640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.874+0000 7f6fea000640 1 -- 172.21.15.7:0/1880583788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 msgr2=0x563c297c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.874+0000 7f6fea000640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.875+0000 7f6fea000640 1 -- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 msgr2=0x563c29839cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.875+0000 7f6fea000640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.875+0000 7f6fea000640 1 -- 172.21.15.7:0/1880583788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2983a240 con 0x563c297bb5f0 2024-06-13T07:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.875+0000 7f6feaa00640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.876+0000 7f6feb400640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.877+0000 7f6fea000640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 0x563c297c99d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563c297b7530 tx=0x7f6fd8009090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f846c08ff91903c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.878+0000 7f6fd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.879+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8015040 con 0x563c297bb5f0 2024-06-13T07:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.879+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f6fd8011420 con 0x563c297bb5f0 2024-06-13T07:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.879+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fdc01a500 con 0x563c297bb5f0 2024-06-13T07:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.880+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6fd8011640 con 0x563c297bb5f0 2024-06-13T07:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.880+0000 7f6ff4bee740 10 client.15888 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.880+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563c2983b590 con 0x563c297bb5f0 2024-06-13T07:02:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fdc01a500 con 0x563c297bb5f0 2024-06-13T07:02:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f6fd8024040 con 0x563c297bb5f0 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 10 client.15888 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 20 client.15888 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c2983bde0 0x563c2983e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 4 client.15888 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.883+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c29725ea0 con 0x563c2983bde0 2024-06-13T07:02:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.884+0000 7f6feaa00640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c2983bde0 0x563c2983e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.887+0000 7f6feaa00640 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c2983bde0 0x563c2983e200 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca6df06ac624b7 server_cookie=e1ebbaea6ecd7f83 in_seq=0 out_seq=0 2024-06-13T07:02:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:29.887+0000 7f6fd4c00640 10 client.15888 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.013+0000 7f6fd4c00640 1 -- 172.21.15.7:0/1880583788 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563c29725ea0 con 0x563c2983bde0 2024-06-13T07:02:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.013+0000 7f6fd4c00640 10 client.15888 handle_command_reply: tid=0 2024-06-13T07:02:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.013+0000 7f6ff4bee740 1 client.15888 shutdown 2024-06-13T07:02:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 msgr2=0x563c297c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 0x563c297c99d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563c297b7530 tx=0x7f6fd8009090 comp rx=0 tx=0).stop 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 shutdown_connections 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563c2983bde0 0x563c2983e200 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563c297c0760 0x563c29839cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c297bb5f0 0x563c297c99d0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563c297b7530 tx=0x7f6fd8009090 comp rx=0 tx=0).stop 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 --2- 172.21.15.7:0/1880583788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c297bac00 0x563c297c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 >> 172.21.15.7:0/1880583788 conn(0x563c29782fb0 msgr2=0x563c297b9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.015+0000 7f6fd4c00640 0 client.15888 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.016+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 shutdown_connections 2024-06-13T07:02:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.016+0000 7f6ff4bee740 1 -- 172.21.15.7:0/1880583788 wait complete. 2024-06-13T07:02:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.016+0000 7f6ff4bee740 20 client.15888 trim_cache size 0 max 16384 2024-06-13T07:02:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:02:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:02:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:02:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 msgr2=0x7f6fd00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 0x7f6fd00a51c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400ad00 tx=0x7f6fe400a370 comp rx=0 tx=0).stop 2024-06-13T07:02:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 msgr2=0x7f6fec195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc004b90 tx=0x7f6fdc004bc0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 shutdown_connections 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f6fd00a2d00 0x7f6fd00a51c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec105590 0x7f6fec19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fec102be0 0x7f6fec196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 --2- 172.21.15.7:0/4262204160 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fec100270 0x7f6fec195ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.018+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 >> 172.21.15.7:0/4262204160 conn(0x7f6fec0fbe70 msgr2=0x7f6fec0fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.019+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 shutdown_connections 2024-06-13T07:02:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:30.019+0000 7f6ff1e00640 1 -- 172.21.15.7:0/4262204160 wait complete. 2024-06-13T07:02:31.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:31 smithi007 ceph-mon[32077]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 420 op/s 2024-06-13T07:02:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:31 smithi189 ceph-mon[36616]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 420 op/s 2024-06-13T07:02:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:31 smithi175 ceph-mon[36886]: pgmap v703: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 420 op/s 2024-06-13T07:02:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 -- 172.21.15.7:0/2692061867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 msgr2=0x7f057810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 --2- 172.21.15.7:0/2692061867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f057810e240 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f056800b0a0 tx=0x7f056800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 -- 172.21.15.7:0/2692061867 shutdown_connections 2024-06-13T07:02:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 --2- 172.21.15.7:0/2692061867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f057810e240 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 --2- 172.21.15.7:0/2692061867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 0x7f05781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 --2- 172.21.15.7:0/2692061867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 -- 172.21.15.7:0/2692061867 >> 172.21.15.7:0/2692061867 conn(0x7f0578078340 msgr2=0x7f0578078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.756+0000 7f057d600700 1 -- 172.21.15.7:0/2692061867 shutdown_connections 2024-06-13T07:02:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.757+0000 7f057d600700 1 -- 172.21.15.7:0/2692061867 wait complete. 2024-06-13T07:02:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.758+0000 7f057d600700 1 Processor -- start 2024-06-13T07:02:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 -- start start 2024-06-13T07:02:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f0578108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 0x7f0578108750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0578111670 con 0x7f0578107400 2024-06-13T07:02:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05781114f0 con 0x7f0578101390 2024-06-13T07:02:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.759+0000 7f057d600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05781117f0 con 0x7f0578101d50 2024-06-13T07:02:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.760+0000 7f0576000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f0578108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.760+0000 7f0576a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.760+0000 7f0576000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f0578108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38866/0 (socket says 172.21.15.7:38866) 2024-06-13T07:02:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.760+0000 7f0576000700 1 -- 172.21.15.7:0/2932378607 learned_addr learned my addr 172.21.15.7:0/2932378607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0577400700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 0x7f0578108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 -- 172.21.15.7:0/2932378607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 msgr2=0x7f0578108210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f0578108210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 -- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 msgr2=0x7f0578108750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 0x7f0578108750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 -- 172.21.15.7:0/2932378607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0578108c90 con 0x7f0578101390 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f0576a00700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578073e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f057000e9f0 tx=0x7f057000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.761+0000 7f056f400700 1 -- 172.21.15.7:0/2932378607 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057000cbd0 con 0x7f0578101390 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.762+0000 7f056f400700 1 -- 172.21.15.7:0/2932378607 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057000cd70 con 0x7f0578101390 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.762+0000 7f056f400700 1 -- 172.21.15.7:0/2932378607 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570018780 con 0x7f0578101390 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.763+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0578108f80 con 0x7f0578101390 2024-06-13T07:02:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.763+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781a7d20 con 0x7f0578101390 2024-06-13T07:02:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.769+0000 7f056f400700 1 -- 172.21.15.7:0/2932378607 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f057001f020 con 0x7f0578101390 2024-06-13T07:02:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.770+0000 7f0587ee8b80 1 Processor -- start 2024-06-13T07:02:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.770+0000 7f0587ee8b80 1 -- start start 2024-06-13T07:02:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.771+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.771+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b7325d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0577400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0577400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40864/0 (socket says 172.21.15.7:40864) 2024-06-13T07:02:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0577400700 1 -- 172.21.15.7:0/1207916497 learned_addr learned my addr 172.21.15.7:0/1207916497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b73262c0 0x55f5b7326700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7234a90 con 0x55f5b732c530 2024-06-13T07:02:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7277b80 con 0x55f5b732bc10 2024-06-13T07:02:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.772+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7232970 con 0x55f5b73262c0 2024-06-13T07:02:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0576000700 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b73262c0 0x55f5b7326700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0577400700 1 -- 172.21.15.7:0/1207916497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b73262c0 msgr2=0x55f5b7326700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0577400700 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b73262c0 0x55f5b7326700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0577400700 1 -- 172.21.15.7:0/1207916497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 msgr2=0x55f5b7325d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0577400700 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b7325d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.773+0000 7f0577400700 1 -- 172.21.15.7:0/1207916497 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b7220010 con 0x55f5b732bc10 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.776+0000 7f056f400700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 0x7f055c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.777+0000 7f056f400700 1 -- 172.21.15.7:0/2932378607 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05700cf380 con 0x7f0578101390 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.777+0000 7f0576a00700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 0x7f055c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.777+0000 7f0577400700 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0568009df0 tx=0x7f0568009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfa066b079000df server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.777+0000 7f0563e00700 1 -- 172.21.15.7:0/1207916497 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056800bcb0 con 0x55f5b732bc10 2024-06-13T07:02:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.778+0000 7f0563e00700 1 -- 172.21.15.7:0/1207916497 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568009490 con 0x55f5b732bc10 2024-06-13T07:02:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.779+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 msgr2=0x55f5b732bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.780+0000 7f0563e00700 1 -- 172.21.15.7:0/1207916497 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056801c6a0 con 0x55f5b732bc10 2024-06-13T07:02:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.780+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0568009df0 tx=0x7f0568009e20 comp rx=0 tx=0).stop 2024-06-13T07:02:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.780+0000 7f0576a00700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 0x7f055c0a5160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0570097ca0 tx=0x7f057000cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 shutdown_connections 2024-06-13T07:02:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b73262c0 0x55f5b7326700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732bff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/1207916497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b7325d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 >> 172.21.15.7:0/1207916497 conn(0x55f5b72521f0 msgr2=0x55f5b7321070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 shutdown_connections 2024-06-13T07:02:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.784+0000 7f0587ee8b80 1 -- 172.21.15.7:0/1207916497 wait complete. 2024-06-13T07:02:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.785+0000 7f0587ee8b80 1 -- start start 2024-06-13T07:02:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.785+0000 7f0587ee8b80 1 Processor -- start 2024-06-13T07:02:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.786+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.786+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b732f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7213fe0 con 0x55f5b73262c0 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7273b40 con 0x55f5b732bc10 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f5b7332f00 con 0x55f5b732c530 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0587ee8b80 10 client.0 _ll_register_callbacks cb 0x55f5b720d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0577400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0577400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56996/0 (socket says 172.21.15.7:56996) 2024-06-13T07:02:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0577400700 1 -- 172.21.15.7:0/166455106 learned_addr learned my addr 172.21.15.7:0/166455106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.787+0000 7f0576000700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0577400700 1 -- 172.21.15.7:0/166455106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 msgr2=0x55f5b732f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:02:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0577400700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b732f9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0577400700 1 -- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 msgr2=0x55f5b732f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0577400700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0577400700 1 -- 172.21.15.7:0/166455106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b71c6950 con 0x55f5b73262c0 2024-06-13T07:02:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.788+0000 7f0576000700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.790+0000 7f0577400700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0568007ce0 tx=0x7f05680075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f119f02af76936e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.790+0000 7f0560c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.794+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056802c070 con 0x55f5b73262c0 2024-06-13T07:02:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.794+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f056801c440 con 0x55f5b73262c0 2024-06-13T07:02:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.794+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0568009490 con 0x55f5b73262c0 2024-06-13T07:02:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.795+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0568036050 con 0x55f5b73262c0 2024-06-13T07:02:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.795+0000 7f0587ee8b80 10 client.15897 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.795+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f5b733d5e0 con 0x55f5b73262c0 2024-06-13T07:02:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.795+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568009490 con 0x55f5b73262c0 2024-06-13T07:02:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.796+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f0568011070 con 0x55f5b73262c0 2024-06-13T07:02:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 20 client.15897 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b733fab0 0x55f5b7341ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 4 client.15897 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.797+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5b7285080 con 0x55f5b733fab0 2024-06-13T07:02:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.798+0000 7f0576000700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b733fab0 0x55f5b7341ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.799+0000 7f0576000700 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b733fab0 0x55f5b7341ea0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6adac2aed878fa server_cookie=fe53db49ed781867 in_seq=0 out_seq=0 2024-06-13T07:02:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.799+0000 7f0560c00700 10 client.15897 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.800+0000 7f0560c00700 1 -- 172.21.15.7:0/166455106 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f5b7285080 con 0x55f5b733fab0 2024-06-13T07:02:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.800+0000 7f0560c00700 10 client.15897 handle_command_reply: tid=0 2024-06-13T07:02:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.800+0000 7f0587ee8b80 1 client.15897 shutdown 2024-06-13T07:02:31.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 msgr2=0x55f5b73321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0568007ce0 tx=0x7f05680075a0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 shutdown_connections 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b733fab0 0x55f5b7341ea0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b732f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b73321a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/166455106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b732f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0560c00700 0 client.15897 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 >> 172.21.15.7:0/166455106 conn(0x55f5b72521f0 msgr2=0x55f5b731a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 shutdown_connections 2024-06-13T07:02:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.803+0000 7f0587ee8b80 1 -- 172.21.15.7:0/166455106 wait complete. 2024-06-13T07:02:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.804+0000 7f0587ee8b80 20 client.15897 trim_cache size 0 max 16384 2024-06-13T07:02:31.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 Processor -- start 2024-06-13T07:02:31.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 -- start start 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b732e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05781c8060 con 0x55f5b732c530 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05781c58c0 con 0x55f5b73262c0 2024-06-13T07:02:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.853+0000 7f0587ee8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05781ae6e0 con 0x55f5b732bc10 2024-06-13T07:02:31.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.854+0000 7f0576000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.854+0000 7f0576000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38892/0 (socket says 172.21.15.7:38892) 2024-06-13T07:02:31.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.854+0000 7f0577400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.854+0000 7f0577400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40892/0 (socket says 172.21.15.7:40892) 2024-06-13T07:02:31.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.854+0000 7f0577400700 1 -- 172.21.15.7:0/4104346608 learned_addr learned my addr 172.21.15.7:0/4104346608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.856+0000 7f0577400700 1 -- 172.21.15.7:0/4104346608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 msgr2=0x55f5b731b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.856+0000 7f0577400700 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.856+0000 7f0577400700 1 -- 172.21.15.7:0/4104346608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 msgr2=0x55f5b732e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.856+0000 7f0577400700 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b732e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.856+0000 7f0577400700 1 -- 172.21.15.7:0/4104346608 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b7322850 con 0x55f5b73262c0 2024-06-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.858+0000 7f0576000700 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.858+0000 7f0577400700 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f056800b3f0 tx=0x7f056800b020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ba88bc85e55b8c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.859+0000 7f0563400700 1 -- 172.21.15.7:0/4104346608 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056802c040 con 0x55f5b73262c0 2024-06-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.859+0000 7f0563400700 1 -- 172.21.15.7:0/4104346608 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056800be50 con 0x55f5b73262c0 2024-06-13T07:02:31.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.859+0000 7f0563400700 1 -- 172.21.15.7:0/4104346608 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568005d50 con 0x55f5b73262c0 2024-06-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.860+0000 7f0587ee8b80 1 -- 172.21.15.7:0/4104346608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 msgr2=0x55f5b7325d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.860+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f056800b3f0 tx=0x7f056800b020 comp rx=0 tx=0).stop 2024-06-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 -- 172.21.15.7:0/4104346608 shutdown_connections 2024-06-13T07:02:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b732c530 0x55f5b732e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b73262c0 0x55f5b7325d80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/4104346608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732bc10 0x55f5b731b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 -- 172.21.15.7:0/4104346608 >> 172.21.15.7:0/4104346608 conn(0x55f5b72521f0 msgr2=0x55f5b7330cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 -- 172.21.15.7:0/4104346608 shutdown_connections 2024-06-13T07:02:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.861+0000 7f0587ee8b80 1 -- 172.21.15.7:0/4104346608 wait complete. 2024-06-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.862+0000 7f0587ee8b80 1 -- start start 2024-06-13T07:02:31.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.862+0000 7f0587ee8b80 1 Processor -- start 2024-06-13T07:02:31.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.864+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b7342920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0587ee8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b7342e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0577400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0577400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57006/0 (socket says 172.21.15.7:57006) 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0577400700 1 -- 172.21.15.7:0/497945784 learned_addr learned my addr 172.21.15.7:0/497945784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05781aed50 con 0x55f5b73262c0 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05781b01c0 con 0x55f5b732bc10 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05781af420 con 0x55f5b732c530 2024-06-13T07:02:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.866+0000 7f0576000700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b7342920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b7342e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0587ee8b80 10 client.0 _ll_register_callbacks cb 0x55f5b720d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 -- 172.21.15.7:0/497945784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 msgr2=0x55f5b7342e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b7342e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 -- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 msgr2=0x55f5b7342920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b7342920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.867+0000 7f0577400700 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b73436a0 con 0x55f5b73262c0 2024-06-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.868+0000 7f0577400700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f056800d010 tx=0x7f056800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a180b5b98e1991 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.868+0000 7f0560c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.869+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568043060 con 0x55f5b73262c0 2024-06-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.869+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0568031790 con 0x55f5b73262c0 2024-06-13T07:02:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.869+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056800be50 con 0x55f5b73262c0 2024-06-13T07:02:31.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.872+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f056802c070 con 0x55f5b73262c0 2024-06-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.872+0000 7f0587ee8b80 10 client.15900 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.872+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f5b73aa3a0 con 0x55f5b73262c0 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056800be50 con 0x55f5b73262c0 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f0568011070 con 0x55f5b73262c0 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 20 client.15900 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b73aa840 0x55f5b73acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 4 client.15900 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:02:31.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.873+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5b727f600 con 0x55f5b73aa840 2024-06-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.874+0000 7f0576000700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b73aa840 0x55f5b73acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.874+0000 7f0560c00700 10 client.15900 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:31.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.874+0000 7f0576000700 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b73aa840 0x55f5b73acc60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3626fedd9be3c0e server_cookie=826ea82fd40f3d90 in_seq=0 out_seq=0 2024-06-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.876+0000 7f0560c00700 1 -- 172.21.15.7:0/497945784 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5b727f600 con 0x55f5b73aa840 2024-06-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.876+0000 7f0560c00700 10 client.15900 handle_command_reply: tid=0 2024-06-13T07:02:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.876+0000 7f0587ee8b80 1 client.15900 shutdown 2024-06-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.877+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 msgr2=0x55f5b7345580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.877+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f056800d010 tx=0x7f056800b700 comp rx=0 tx=0).stop 2024-06-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 shutdown_connections 2024-06-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f5b73aa840 0x55f5b73acc60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5b732c530 0x55f5b7342e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5b73262c0 0x55f5b7345580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 --2- 172.21.15.7:0/497945784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f5b732bc10 0x55f5b7342920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 >> 172.21.15.7:0/497945784 conn(0x55f5b72521f0 msgr2=0x55f5b733b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.879+0000 7f0560c00700 0 client.15900 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.880+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 shutdown_connections 2024-06-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.880+0000 7f0587ee8b80 1 -- 172.21.15.7:0/497945784 wait complete. 2024-06-13T07:02:31.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.880+0000 7f0587ee8b80 20 client.15900 trim_cache size 0 max 16384 2024-06-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "59e2c6b6-1302-4dba-b816-6ac8d8831ea2", 2024-06-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:02:31.881 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:02:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.881+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 msgr2=0x7f055c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.881+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 0x7f055c0a5160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0570097ca0 tx=0x7f057000cf20 comp rx=0 tx=0).stop 2024-06-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.881+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 msgr2=0x7f0578073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.881+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578073e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f057000e9f0 tx=0x7f057000ed00 comp rx=0 tx=0).stop 2024-06-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 shutdown_connections 2024-06-13T07:02:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0578107400 0x7f0578108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f055c0a2ca0 0x7f055c0a5160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0578101390 0x7f0578073e00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 --2- 172.21.15.7:0/2932378607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0578101d50 0x7f0578108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 >> 172.21.15.7:0/2932378607 conn(0x7f0578078340 msgr2=0x7f05780febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.882+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 shutdown_connections 2024-06-13T07:02:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:31.883+0000 7f057d600700 1 -- 172.21.15.7:0/2932378607 wait complete. 2024-06-13T07:02:32.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:02:32.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 390.0 s, 505.6 tps, lat 201.597 ms stddev 238.322 2024-06-13T07:02:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:02:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 443 op/s 2024-06-13T07:02:33.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:33 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:02:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 443 op/s 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:33 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:02:33.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: pgmap v704: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 443 op/s 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:33.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:33 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:02:35.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:35 smithi007 ceph-mon[32077]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 636 op/s 2024-06-13T07:02:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:35 smithi189 ceph-mon[36616]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 636 op/s 2024-06-13T07:02:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:35 smithi175 ceph-mon[36886]: pgmap v705: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 636 op/s 2024-06-13T07:02:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:37 smithi189 ceph-mon[36616]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-06-13T07:02:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:37 smithi175 ceph-mon[36886]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-06-13T07:02:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:37 smithi007 ceph-mon[32077]: pgmap v706: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2024-06-13T07:02:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.679+0000 7fabae800700 1 --2- 172.21.15.7:0/223631724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb006ab30 0x7fabb010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.680+0000 7fabb5200700 1 -- 172.21.15.7:0/223631724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb006a170 msgr2=0x7fabb006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.680+0000 7fabb5200700 1 --2- 172.21.15.7:0/223631724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb006a170 0x7fabb006a5f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009990 tx=0x7fab9c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.681+0000 7fabb5200700 1 -- 172.21.15.7:0/223631724 shutdown_connections 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.681+0000 7fabb5200700 1 --2- 172.21.15.7:0/223631724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb006ab30 0x7fabb010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.681+0000 7fabb5200700 1 --2- 172.21.15.7:0/223631724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb0108500 0x7fabb01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.681+0000 7fabb5200700 1 --2- 172.21.15.7:0/223631724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb006a170 0x7fabb006a5f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.681+0000 7fabb5200700 1 -- 172.21.15.7:0/223631724 >> 172.21.15.7:0/223631724 conn(0x7fabb0078340 msgr2=0x7fabb0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.682+0000 7fabb5200700 1 -- 172.21.15.7:0/223631724 shutdown_connections 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.682+0000 7fabb5200700 1 -- 172.21.15.7:0/223631724 wait complete. 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.683+0000 7fabb5200700 1 Processor -- start 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.683+0000 7fabb5200700 1 -- start start 2024-06-13T07:02:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.685+0000 7fabb5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb006ab30 0x7fabb019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.685+0000 7fabb5200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.685+0000 7fabb5200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.686+0000 7fabad400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.686+0000 7fabad400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40916/0 (socket says 172.21.15.7:40916) 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabae800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabae800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38908/0 (socket says 172.21.15.7:38908) 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabb5200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112850 con 0x7fabb006ab30 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabb5200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01126d0 con 0x7fabb0108500 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabb5200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01129d0 con 0x7fabb010e770 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.687+0000 7fabad400700 1 -- 172.21.15.7:0/441441773 learned_addr learned my addr 172.21.15.7:0/441441773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.688+0000 7fabad400700 1 -- 172.21.15.7:0/441441773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 msgr2=0x7fabb010ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.689+0000 7fabad400700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.689+0000 7fabad400700 1 -- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb006ab30 msgr2=0x7fabb019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.689+0000 7fabad400700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb006ab30 0x7fabb019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.689+0000 7fabad400700 1 -- 172.21.15.7:0/441441773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb019b4c0 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.690+0000 7fabae800700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.691+0000 7fabad400700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0098e0 tx=0x7fab9c005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.692+0000 7faba6a00700 1 -- 172.21.15.7:0/441441773 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00fa10 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.692+0000 7faba6a00700 1 -- 172.21.15.7:0/441441773 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9c004d10 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.692+0000 7faba6a00700 1 -- 172.21.15.7:0/441441773 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0055c0 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.693+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb019b750 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.693+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb019bb80 con 0x7fabb0108500 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.699+0000 7fabbfc4eb80 1 Processor -- start 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.701+0000 7fabbfc4eb80 1 -- start start 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5566d8958a10 con 0x5566d8a4a240 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566d899bb00 con 0x5566d8a4fb90 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.702+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566d89568f0 con 0x5566d8a504b0 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.704+0000 7fabae800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.704+0000 7fabae800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40922/0 (socket says 172.21.15.7:40922) 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.704+0000 7fabae800700 1 -- 172.21.15.7:0/3917541665 learned_addr learned my addr 172.21.15.7:0/3917541665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.704+0000 7fabade00700 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.705+0000 7fabae800700 1 -- 172.21.15.7:0/3917541665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 msgr2=0x5566d8a49d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.705+0000 7fabae800700 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a49d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.705+0000 7fabae800700 1 -- 172.21.15.7:0/3917541665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 msgr2=0x5566d8a4a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.705+0000 7fabae800700 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a4a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.706+0000 7fabae800700 1 -- 172.21.15.7:0/3917541665 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d8943f90 con 0x5566d8a4fb90 2024-06-13T07:02:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.706+0000 7fabade00700 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.706+0000 7fabae800700 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faba8009510 tx=0x7faba8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc92a3440c14ef server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.706+0000 7fab9b400700 1 -- 172.21.15.7:0/3917541665 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba801b020 con 0x5566d8a4fb90 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.707+0000 7fab9b400700 1 -- 172.21.15.7:0/3917541665 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba8007e20 con 0x5566d8a4fb90 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.707+0000 7fab9b400700 1 -- 172.21.15.7:0/3917541665 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8015410 con 0x5566d8a4fb90 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.707+0000 7faba6a00700 1 -- 172.21.15.7:0/441441773 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fab9c009d10 con 0x7fabb0108500 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/3917541665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 msgr2=0x5566d8a4ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faba8009510 tx=0x7faba8009920 comp rx=0 tx=0).stop 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/3917541665 shutdown_connections 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4fb90 0x5566d8a4ff70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/3917541665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/3917541665 >> 172.21.15.7:0/3917541665 conn(0x5566d8976170 msgr2=0x5566d8a44ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/3917541665 shutdown_connections 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.708+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/3917541665 wait complete. 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.709+0000 7fabbfc4eb80 1 -- start start 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.710+0000 7fabbfc4eb80 1 Processor -- start 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a56280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 0x5566d8a53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5566d8937f60 con 0x5566d8a4fb90 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5566d8997ac0 con 0x5566d8a4a240 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566d8a67ea0 con 0x5566d8a504b0 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabbfc4eb80 10 client.0 _ll_register_callbacks cb 0x5566d8931380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.712+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.713+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38922/0 (socket says 172.21.15.7:38922) 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.713+0000 7fabad400700 1 -- 172.21.15.7:0/4091185642 learned_addr learned my addr 172.21.15.7:0/4091185642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.713+0000 7fabae800700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a56280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.713+0000 7faba6a00700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 0x7fab940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.714+0000 7faba6a00700 1 -- 172.21.15.7:0/441441773 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fab9c0cfa00 con 0x7fabb0108500 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabae800700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 0x7fab940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.714+0000 7fabade00700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 0x5566d8a53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabad400700 1 -- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 msgr2=0x5566d8a56280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabad400700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a56280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabad400700 1 -- 172.21.15.7:0/4091185642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 msgr2=0x5566d8a53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabad400700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 0x5566d8a53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.715+0000 7fabad400700 1 -- 172.21.15.7:0/4091185642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d88ea8d0 con 0x5566d8a504b0 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.716+0000 7fabad400700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab9c098920 tx=0x7fab9c00fe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe5351ba2712e10 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.716+0000 7fab8fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.716+0000 7fabade00700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 0x5566d8a53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c09bd20 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fab9c09a250 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba8007e20 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fab9c09b8b0 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fabbfc4eb80 10 client.26344 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.718+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5566d8a72440 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.719+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba8007e20 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.719+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fab9c09b3a0 con 0x5566d8a504b0 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 10 client.26344 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 20 client.26344 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8a74a50 0x5566d8a76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 4 client.26344 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566d89a9000 con 0x5566d8a74a50 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.720+0000 7fabade00700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8a74a50 0x5566d8a76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.721+0000 7fabae800700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 0x7fab940a51c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faba8019000 tx=0x7faba80179c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.722+0000 7fabade00700 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8a74a50 0x5566d8a76e70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5618797d696c12e server_cookie=19ae519ba026c717 in_seq=0 out_seq=0 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.722+0000 7fab8fe00700 10 client.26344 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.723+0000 7fab8fe00700 1 -- 172.21.15.7:0/4091185642 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566d89a9000 con 0x5566d8a74a50 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.723+0000 7fab8fe00700 10 client.26344 handle_command_reply: tid=0 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.724+0000 7fabbfc4eb80 1 client.26344 shutdown 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.724+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 msgr2=0x5566d8a53ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.724+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab9c098920 tx=0x7fab9c00fe90 comp rx=0 tx=0).stop 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 shutdown_connections 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8a74a50 0x5566d8a76e70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a504b0 0x5566d8a53ac0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a56280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4091185642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4fb90 0x5566d8a53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fab8fe00700 0 client.26344 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.725+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 >> 172.21.15.7:0/4091185642 conn(0x5566d8976170 msgr2=0x5566d8a3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.726+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 shutdown_connections 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.726+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4091185642 wait complete. 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.727+0000 7fabbfc4eb80 20 client.26344 trim_cache size 0 max 16384 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.773+0000 7fabbfc4eb80 1 Processor -- start 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 -- start start 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a4dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 0x5566d8a53af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab900112c0 con 0x5566d8a4a240 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab9000eb20 con 0x5566d8a504b0 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.774+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab9000c970 con 0x5566d8a4fb90 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.775+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a4dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.775+0000 7fabae800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 0x5566d8a53af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.775+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a4dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38926/0 (socket says 172.21.15.7:38926) 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.775+0000 7fabad400700 1 -- 172.21.15.7:0/4018898233 learned_addr learned my addr 172.21.15.7:0/4018898233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.776+0000 7fabade00700 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabade00700 1 -- 172.21.15.7:0/4018898233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 msgr2=0x5566d8a4dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabade00700 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a4dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabade00700 1 -- 172.21.15.7:0/4018898233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 msgr2=0x5566d8a53af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabade00700 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 0x5566d8a53af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabade00700 1 -- 172.21.15.7:0/4018898233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d8a544c0 con 0x5566d8a4a240 2024-06-13T07:02:37.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.777+0000 7fabae800700 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 0x5566d8a53af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.778+0000 7fabade00700 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a49d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5566d8a77680 tx=0x7faba00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670bd61f57ff73e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.781+0000 7fab9aa00700 1 -- 172.21.15.7:0/4018898233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0012b50 con 0x5566d8a4a240 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.781+0000 7fab9aa00700 1 -- 172.21.15.7:0/4018898233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0004510 con 0x5566d8a4a240 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.781+0000 7fab9aa00700 1 -- 172.21.15.7:0/4018898233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0004e60 con 0x5566d8a4a240 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.782+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4018898233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 msgr2=0x5566d8a49d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.782+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a49d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5566d8a77680 tx=0x7faba00092a0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.783+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4018898233 shutdown_connections 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.783+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a504b0 0x5566d8a53af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.783+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a4a240 0x5566d8a49d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.783+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/4018898233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a4dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.783+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4018898233 >> 172.21.15.7:0/4018898233 conn(0x5566d8976170 msgr2=0x5566d8a70040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.792+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4018898233 shutdown_connections 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.792+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/4018898233 wait complete. 2024-06-13T07:02:37.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.793+0000 7fabbfc4eb80 1 -- start start 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.793+0000 7fabbfc4eb80 1 Processor -- start 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.795+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a6d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.796+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabbfc4eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a504b0 0x5566d8a7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab9000cfe0 con 0x5566d8a504b0 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fab9000e450 con 0x5566d8a4a240 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabbfc4eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab9000d6b0 con 0x5566d8a4fb90 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabbfc4eb80 10 client.0 _ll_register_callbacks cb 0x5566d8931380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabad400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38932/0 (socket says 172.21.15.7:38932) 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabad400700 1 -- 172.21.15.7:0/581736577 learned_addr learned my addr 172.21.15.7:0/581736577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabade00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a6d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.797+0000 7fabad400700 1 -- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 msgr2=0x5566d8a6d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.799+0000 7fabad400700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a6d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.799+0000 7fabad400700 1 -- 172.21.15.7:0/581736577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a504b0 msgr2=0x5566d8a7c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.799+0000 7fabad400700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a504b0 0x5566d8a7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.800+0000 7fabad400700 1 -- 172.21.15.7:0/581736577 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d8a7cec0 con 0x5566d8a4fb90 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.801+0000 7fabade00700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a6d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.802+0000 7fabad400700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c0d52a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5df0e3a16c48219 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.802+0000 7fab8fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.803+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c09ba90 con 0x5566d8a4fb90 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.803+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fab9c004a30 con 0x5566d8a4fb90 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.804+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0004510 con 0x5566d8a4fb90 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.805+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fab9c0b7070 con 0x5566d8a4fb90 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.805+0000 7fabbfc4eb80 10 client.26350 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.805+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5566d8adf740 con 0x5566d8a4fb90 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.806+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0004510 con 0x5566d8a4fb90 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.806+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fab9c0ba070 con 0x5566d8a4fb90 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.806+0000 7fabbfc4eb80 10 client.26350 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.806+0000 7fabbfc4eb80 10 client.26350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.807+0000 7fabbfc4eb80 20 client.26350 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.807+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8adfcc0 0x5566d8ae20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.807+0000 7fabbfc4eb80 4 client.26350 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.807+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566d89a3580 con 0x5566d8adfcc0 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.809+0000 7fabade00700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8adfcc0 0x5566d8ae20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fabade00700 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8adfcc0 0x5566d8ae20e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb6430e510195dc server_cookie=fddf6abcb637682e in_seq=0 out_seq=0 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fab8fe00700 10 client.26350 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fab8fe00700 1 -- 172.21.15.7:0/581736577 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566d89a3580 con 0x5566d8adfcc0 2024-06-13T07:02:37.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fab8fe00700 10 client.26350 handle_command_reply: tid=0 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fabbfc4eb80 1 client.26350 shutdown 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 msgr2=0x5566d8a7c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.810+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c0d52a0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 shutdown_connections 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5566d8adfcc0 0x5566d8ae20e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5566d8a504b0 0x5566d8a7c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fab8fe00700 0 client.26350 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5566d8a4a240 0x5566d8a6d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 --2- 172.21.15.7:0/581736577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566d8a4fb90 0x5566d8a7c110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.775 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "59e2c6b6-1302-4dba-b816-6ac8d8831ea2": { 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "59e2c6b6-1302-4dba-b816-6ac8d8831ea2", 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.811+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 >> 172.21.15.7:0/581736577 conn(0x5566d8976170 msgr2=0x5566d8a53370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.812+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 shutdown_connections 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.812+0000 7fabbfc4eb80 1 -- 172.21.15.7:0/581736577 wait complete. 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.813+0000 7fabbfc4eb80 20 client.26350 trim_cache size 0 max 16384 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.816+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 msgr2=0x7fab940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.816+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 0x7fab940a51c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faba8019000 tx=0x7faba80179c0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.816+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 msgr2=0x7fabb019ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.816+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0098e0 tx=0x7fab9c005e50 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 shutdown_connections 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb010e770 0x7fabb010ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb006ab30 0x7fabb019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fab940a2d00 0x7fab940a51c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 --2- 172.21.15.7:0/441441773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fabb0108500 0x7fabb019ffe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 >> 172.21.15.7:0/441441773 conn(0x7fabb0078340 msgr2=0x7fabb00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 shutdown_connections 2024-06-13T07:02:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:02:36.818+0000 7fabb5200700 1 -- 172.21.15.7:0/441441773 wait complete. 2024-06-13T07:02:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 395.0 s, 749.1 tps, lat 132.029 ms stddev 151.114 2024-06-13T07:02:38.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:38 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:02:38.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:38 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:02:38.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:38 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:02:38.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:38 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:02:38.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:38 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:02:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:38 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:02:39.133 INFO:tasks.cephfs.filesystem:scrub status for tag:59e2c6b6-1302-4dba-b816-6ac8d8831ea2 - {'path': '/', 'tag': '59e2c6b6-1302-4dba-b816-6ac8d8831ea2', 'options': 'recursive,force'} 2024-06-13T07:02:39.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:39 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:02:39.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:39 smithi007 ceph-mon[32077]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-06-13T07:02:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:39 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:02:39.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:39 smithi189 ceph-mon[36616]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-06-13T07:02:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:39 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:02:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:39 smithi175 ceph-mon[36886]: pgmap v707: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.9 MiB/s wr, 669 op/s 2024-06-13T07:02:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:41 smithi189 ceph-mon[36616]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.4 MiB/s wr, 562 op/s 2024-06-13T07:02:41.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:41 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:41 smithi175 ceph-mon[36886]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.4 MiB/s wr, 562 op/s 2024-06-13T07:02:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:41 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:41.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:41 smithi007 ceph-mon[32077]: pgmap v708: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.4 MiB/s wr, 562 op/s 2024-06-13T07:02:41.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:41 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:02:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 400.0 s, 336.2 tps, lat 293.306 ms stddev 382.207 2024-06-13T07:02:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 -- 172.21.15.7:0/172635331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc807a310 msgr2=0x7f7fc807c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 --2- 172.21.15.7:0/172635331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc807a310 0x7f7fc807c7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009900 tx=0x7f7fb002a410 comp rx=0 tx=0).stop 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 -- 172.21.15.7:0/172635331 shutdown_connections 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 --2- 172.21.15.7:0/172635331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc807a310 0x7f7fc807c7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 --2- 172.21.15.7:0/172635331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc8074100 0x7f7fc8079dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 --2- 172.21.15.7:0/172635331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc8077880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 -- 172.21.15.7:0/172635331 >> 172.21.15.7:0/172635331 conn(0x7f7fc806ea10 msgr2=0x7f7fc8070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 -- 172.21.15.7:0/172635331 shutdown_connections 2024-06-13T07:02:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.229+0000 7f7fcda00640 1 -- 172.21.15.7:0/172635331 wait complete. 2024-06-13T07:02:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.231+0000 7f7fcda00640 1 Processor -- start 2024-06-13T07:02:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.231+0000 7f7fcda00640 1 -- start start 2024-06-13T07:02:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.231+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 0x7f7fc81b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc807a310 0x7f7fc81bad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fcda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc80800d0 con 0x7f7fc8074100 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53436/0 (socket says 172.21.15.7:53436) 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fc6a00640 1 -- 172.21.15.7:0/2462940169 learned_addr learned my addr 172.21.15.7:0/2462940169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc807ff50 con 0x7f7fc8073ce0 2024-06-13T07:02:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.232+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8080250 con 0x7f7fc807a310 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.233+0000 7f7fc6a00640 1 -- 172.21.15.7:0/2462940169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc807a310 msgr2=0x7f7fc81bad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 0x7f7fc81b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc807a310 0x7f7fc81bad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6a00640 1 -- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 msgr2=0x7f7fc81b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 0x7f7fc81b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6a00640 1 -- 172.21.15.7:0/2462940169 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc81bb3e0 con 0x7f7fc8073ce0 2024-06-13T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.234+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 0x7f7fc81b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.235+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ba50 tx=0x7f7fc000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.235+0000 7f7fbf400640 1 -- 172.21.15.7:0/2462940169 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0019070 con 0x7f7fc8073ce0 2024-06-13T07:02:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.236+0000 7f7fbf400640 1 -- 172.21.15.7:0/2462940169 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc000cc10 con 0x7f7fc8073ce0 2024-06-13T07:02:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.236+0000 7f7fbf400640 1 -- 172.21.15.7:0/2462940169 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00076a0 con 0x7f7fc8073ce0 2024-06-13T07:02:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.238+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc81bb6d0 con 0x7f7fc8073ce0 2024-06-13T07:02:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.238+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81bbc20 con 0x7f7fc8073ce0 2024-06-13T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.243+0000 7f7fbf400640 1 -- 172.21.15.7:0/2462940169 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7fc0007840 con 0x7f7fc8073ce0 2024-06-13T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.245+0000 7f7fbf400640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 0x7f7fac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.245+0000 7f7fbf400640 1 -- 172.21.15.7:0/2462940169 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7fc00ce9b0 con 0x7f7fc8073ce0 2024-06-13T07:02:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.247+0000 7f7fd0834740 1 Processor -- start 2024-06-13T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 -- start start 2024-06-13T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 0x565043089830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x56504308e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 0x7f7fac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043091400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565042fe4420 con 0x565043089410 2024-06-13T07:02:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565042f8bc50 con 0x56504308ef70 2024-06-13T07:02:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.248+0000 7f7fd0834740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565042fe03e0 con 0x565043089e00 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.249+0000 7f7fc7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043091400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.249+0000 7f7fc7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043091400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53452/0 (socket says 172.21.15.7:53452) 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.249+0000 7f7fc7400640 1 -- 172.21.15.7:0/3967133548 learned_addr learned my addr 172.21.15.7:0/3967133548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.249+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 0x565043089830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.250+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x56504308e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.250+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3967133548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 msgr2=0x56504308e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.251+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x56504308e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.251+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3967133548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 msgr2=0x565043091400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.251+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043091400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.251+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3967133548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565042cf8210 con 0x565043089410 2024-06-13T07:02:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.252+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x56504308e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.252+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 0x565043089830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009ae00 tx=0x7f7fc009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62554e8d13dc88de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.256+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 0x7f7fac0a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56504308a200 tx=0x7f7fb8006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.259+0000 7f7fb7e00640 1 -- 172.21.15.7:0/3967133548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0099170 con 0x565043089410 2024-06-13T07:02:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.260+0000 7f7fb7e00640 1 -- 172.21.15.7:0/3967133548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc00cc220 con 0x565043089410 2024-06-13T07:02:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.260+0000 7f7fb7e00640 1 -- 172.21.15.7:0/3967133548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009a2c0 con 0x565043089410 2024-06-13T07:02:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 -- 172.21.15.7:0/3967133548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 msgr2=0x565043089830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 0x565043089830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009ae00 tx=0x7f7fc009ae70 comp rx=0 tx=0).stop 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 -- 172.21.15.7:0/3967133548 shutdown_connections 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043091400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x56504308e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3967133548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043089410 0x565043089830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.261+0000 7f7fd0834740 1 -- 172.21.15.7:0/3967133548 >> 172.21.15.7:0/3967133548 conn(0x565043048fb0 msgr2=0x56504307c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.262+0000 7f7fd0834740 1 -- 172.21.15.7:0/3967133548 shutdown_connections 2024-06-13T07:02:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.262+0000 7f7fd0834740 1 -- 172.21.15.7:0/3967133548 wait complete. 2024-06-13T07:02:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.263+0000 7f7fd0834740 1 -- start start 2024-06-13T07:02:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.263+0000 7f7fd0834740 1 Processor -- start 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.263+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043093fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043098f70 0x56504309b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565043091f20 con 0x565043098f70 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565042f9f0b0 con 0x56504308ef70 2024-06-13T07:02:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650430920a0 con 0x565043089e00 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fd0834740 10 client.0 _ll_register_callbacks cb 0x565042f8af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56318/0 (socket says 172.21.15.7:56318) 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3661124678 learned_addr learned my addr 172.21.15.7:0/3661124678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc7400640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043098f70 0x56504309b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043093fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.264+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 msgr2=0x565043093fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043093fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3661124678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043098f70 msgr2=0x56504309b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043098f70 0x56504309b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3661124678 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565042dc4620 con 0x565043089e00 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00a35b0 tx=0x7f7fc009aab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1670805d8e354c32 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043093fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.265+0000 7f7fb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.269+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009a2c0 con 0x565043089e00 2024-06-13T07:02:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.269+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7fc009a460 con 0x565043089e00 2024-06-13T07:02:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.269+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc00cc220 con 0x565043089e00 2024-06-13T07:02:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.271+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7fc0007b30 con 0x565043089e00 2024-06-13T07:02:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.271+0000 7f7fd0834740 10 client.26353 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.271+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565043091f20 con 0x565043089e00 2024-06-13T07:02:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.272+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc00cc220 con 0x565043089e00 2024-06-13T07:02:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.272+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f7fc0079ce0 con 0x565043089e00 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.272+0000 7f7fd0834740 10 client.26353 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.275+0000 7f7fd0834740 10 client.26353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.275+0000 7f7fd0834740 20 client.26353 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.275+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56504309f7c0 0x5650430a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.275+0000 7f7fd0834740 4 client.26353 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.275+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565042ff1b50 con 0x56504309f7c0 2024-06-13T07:02:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.281+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56504309f7c0 0x5650430a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.282+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56504309f7c0 0x5650430a1bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c981be40a3df5f54 server_cookie=8ef9892a8554c3fc in_seq=0 out_seq=0 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fb4c00640 10 client.26353 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3661124678 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565042ff1b50 con 0x56504309f7c0 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fb4c00640 10 client.26353 handle_command_reply: tid=0 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 client.26353 shutdown 2024-06-13T07:02:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 msgr2=0x565043093a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00a35b0 tx=0x7f7fc009aab0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 shutdown_connections 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56504309f7c0 0x5650430a1bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565043098f70 0x56504309b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56504308ef70 0x565043093fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3661124678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565043089e00 0x565043093a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 >> 172.21.15.7:0/3661124678 conn(0x565043048fb0 msgr2=0x56504307b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.284+0000 7f7fb4c00640 0 client.26353 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.285+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 shutdown_connections 2024-06-13T07:02:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.285+0000 7f7fd0834740 1 -- 172.21.15.7:0/3661124678 wait complete. 2024-06-13T07:02:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.285+0000 7f7fd0834740 20 client.26353 trim_cache size 0 max 16384 2024-06-13T07:02:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.333+0000 7f7fd0834740 1 Processor -- start 2024-06-13T07:02:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.334+0000 7f7fd0834740 1 -- start start 2024-06-13T07:02:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 0x565043093f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c4500 con 0x56504308ef70 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c1da0 con 0x565043098f70 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fd0834740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81bfc10 con 0x56504307c990 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fc7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56328/0 (socket says 172.21.15.7:56328) 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fc7400640 1 -- 172.21.15.7:0/2418862763 learned_addr learned my addr 172.21.15.7:0/2418862763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.335+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53462/0 (socket says 172.21.15.7:53462) 2024-06-13T07:02:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.336+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 0x565043093f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 -- 172.21.15.7:0/2418862763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 msgr2=0x56504307b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 -- 172.21.15.7:0/2418862763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 msgr2=0x565043093f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 0x565043093f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 -- 172.21.15.7:0/2418862763 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650430776d0 con 0x565043098f70 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 0x565043093f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc7400640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.338+0000 7f7fc6000640 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5650430a2260 tx=0x7f7fb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52401672922d35b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.339+0000 7f7fb7400640 1 -- 172.21.15.7:0/2418862763 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800f040 con 0x565043098f70 2024-06-13T07:02:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.339+0000 7f7fb7400640 1 -- 172.21.15.7:0/2418862763 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb8010d40 con 0x565043098f70 2024-06-13T07:02:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.339+0000 7f7fb7400640 1 -- 172.21.15.7:0/2418862763 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb802b610 con 0x565043098f70 2024-06-13T07:02:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 -- 172.21.15.7:0/2418862763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 msgr2=0x56504307afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5650430a2260 tx=0x7f7fb80046c0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 -- 172.21.15.7:0/2418862763 shutdown_connections 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504307c990 0x56504307b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x56504307afb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 --2- 172.21.15.7:0/2418862763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504308ef70 0x565043093f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.340+0000 7f7fd0834740 1 -- 172.21.15.7:0/2418862763 >> 172.21.15.7:0/2418862763 conn(0x565043048fb0 msgr2=0x565043093820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.341+0000 7f7fd0834740 1 -- 172.21.15.7:0/2418862763 shutdown_connections 2024-06-13T07:02:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.341+0000 7f7fd0834740 1 -- 172.21.15.7:0/2418862763 wait complete. 2024-06-13T07:02:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.341+0000 7f7fd0834740 1 -- start start 2024-06-13T07:02:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.342+0000 7f7fd0834740 1 Processor -- start 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 0x5650430b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x5650430b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c0260 con 0x56504307c990 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c16d0 con 0x565043098f70 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81c0930 con 0x56504308ef70 2024-06-13T07:02:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.343+0000 7f7fd0834740 10 client.0 _ll_register_callbacks cb 0x7f7fc009a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 0x5650430b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56338/0 (socket says 172.21.15.7:56338) 2024-06-13T07:02:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc7400640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x5650430b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6000640 1 -- 172.21.15.7:0/3873435125 learned_addr learned my addr 172.21.15.7:0/3873435125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3873435125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 msgr2=0x5650430b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 msgr2=0x5650430b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x5650430b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.344+0000 7f7fc6a00640 1 -- 172.21.15.7:0/3873435125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650430b25c0 con 0x56504307c990 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fc7400640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x5650430b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fc6a00640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 0x5650430b0a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0099120 tx=0x7f7fc00b9ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e452251e08c0d102 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc009f220 con 0x56504307c990 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7fc000cdc0 con 0x56504307c990 2024-06-13T07:02:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.345+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb8010d40 con 0x56504307c990 2024-06-13T07:02:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.346+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7fc00a3650 con 0x56504307c990 2024-06-13T07:02:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.346+0000 7f7fd0834740 10 client.15918 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.346+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5650430b3840 con 0x56504307c990 2024-06-13T07:02:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb8010d40 con 0x56504307c990 2024-06-13T07:02:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f7fc0079ce0 con 0x56504307c990 2024-06-13T07:02:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 10 client.15918 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 20 client.15918 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x565043086950 0x565043115120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 4 client.15918 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.348+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565042febea0 con 0x565043086950 2024-06-13T07:02:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.349+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x565043086950 0x565043115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.350+0000 7f7fc6000640 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x565043086950 0x565043115120 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbe113bf1c404a9 server_cookie=159a80b44e85e97d in_seq=0 out_seq=0 2024-06-13T07:02:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.350+0000 7f7fb4c00640 10 client.15918 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.407+0000 7f7fb4c00640 1 -- 172.21.15.7:0/3873435125 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565042febea0 con 0x565043086950 2024-06-13T07:02:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.407+0000 7f7fb4c00640 10 client.15918 handle_command_reply: tid=0 2024-06-13T07:02:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.407+0000 7f7fd0834740 1 client.15918 shutdown 2024-06-13T07:02:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 msgr2=0x5650430b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 0x5650430b0a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0099120 tx=0x7f7fc00b9ea0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 shutdown_connections 2024-06-13T07:02:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x565043086950 0x565043115120 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565043098f70 0x5650430b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56504308ef70 0x5650430b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 --2- 172.21.15.7:0/3873435125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56504307c990 0x5650430b0a80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fb4c00640 0 client.15918 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.408+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 >> 172.21.15.7:0/3873435125 conn(0x565043048fb0 msgr2=0x56504307afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.409+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 shutdown_connections 2024-06-13T07:02:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.409+0000 7f7fd0834740 1 -- 172.21.15.7:0/3873435125 wait complete. 2024-06-13T07:02:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.410+0000 7f7fd0834740 20 client.15918 trim_cache size 0 max 16384 2024-06-13T07:02:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.411+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 msgr2=0x7f7fac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.411+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 0x7f7fac0a51f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56504308a200 tx=0x7f7fb8006c90 comp rx=0 tx=0).stop 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.411+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 msgr2=0x7f7fc81b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.411+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ba50 tx=0x7f7fc000bf10 comp rx=0 tx=0).stop 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 shutdown_connections 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7fac0a2d30 0x7f7fac0a51f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fc807a310 0x7f7fc81bad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8074100 0x7f7fc81b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 --2- 172.21.15.7:0/2462940169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8073ce0 0x7f7fc81b37c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 >> 172.21.15.7:0/2462940169 conn(0x7f7fc806ea10 msgr2=0x7f7fc8071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 shutdown_connections 2024-06-13T07:02:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:43.412+0000 7f7fcda00640 1 -- 172.21.15.7:0/2462940169 wait complete. 2024-06-13T07:02:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:43 smithi189 ceph-mon[36616]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.0 MiB/s wr, 694 op/s 2024-06-13T07:02:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:43 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:43 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:43 smithi175 ceph-mon[36886]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.0 MiB/s wr, 694 op/s 2024-06-13T07:02:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:43 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:43 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:43 smithi007 ceph-mon[32077]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.0 MiB/s wr, 694 op/s 2024-06-13T07:02:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:43 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:43 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:44 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:44 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:44.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:44 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:45.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:45 smithi189 ceph-mon[36616]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-06-13T07:02:45.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:45 smithi175 ceph-mon[36886]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-06-13T07:02:45.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:45 smithi007 ceph-mon[32077]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.24k op/s 2024-06-13T07:02:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:47 smithi189 ceph-mon[36616]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-06-13T07:02:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:47 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:47 smithi175 ceph-mon[36886]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-06-13T07:02:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:47 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:47 smithi007 ceph-mon[32077]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-06-13T07:02:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:47 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 405.0 s, 572.8 tps, lat 178.822 ms stddev 209.062 2024-06-13T07:02:48.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:48 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:48.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:48 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:48.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:48 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:48.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:48 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:48.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:48 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:48.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:48 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:48.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:02:48.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:02:48.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:02:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:49 smithi189 ceph-mon[36616]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-06-13T07:02:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:49 smithi175 ceph-mon[36886]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-06-13T07:02:49.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:49 smithi007 ceph-mon[32077]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-06-13T07:02:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:51 smithi189 ceph-mon[36616]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-06-13T07:02:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:51 smithi175 ceph-mon[36886]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-06-13T07:02:51.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:51 smithi007 ceph-mon[32077]: pgmap v713: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 1.02k op/s 2024-06-13T07:02:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 410.0 s, 550.2 tps, lat 178.763 ms stddev 246.399 2024-06-13T07:02:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:53 smithi189 ceph-mon[36616]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-06-13T07:02:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:53 smithi175 ceph-mon[36886]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-06-13T07:02:53.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:53 smithi007 ceph-mon[32077]: pgmap v714: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.9 MiB/s wr, 1.01k op/s 2024-06-13T07:02:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:55 smithi189 ceph-mon[36616]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-06-13T07:02:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:55 smithi175 ceph-mon[36886]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-06-13T07:02:55.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:55 smithi007 ceph-mon[32077]: pgmap v715: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-06-13T07:02:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.620+0000 7f5f8c000640 1 -- 172.21.15.7:0/3407186582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f841120e0 msgr2=0x7f5f84114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.620+0000 7f5f8c000640 1 --2- 172.21.15.7:0/3407186582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f841120e0 0x7f5f84114530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009be0 tx=0x7f5f7c02a6f0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.621+0000 7f5f8c000640 1 -- 172.21.15.7:0/3407186582 shutdown_connections 2024-06-13T07:02:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.621+0000 7f5f8c000640 1 --2- 172.21.15.7:0/3407186582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f841120e0 0x7f5f84114530 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.621+0000 7f5f8c000640 1 --2- 172.21.15.7:0/3407186582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f8410b310 0x7f5f8410b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.621+0000 7f5f8c000640 1 --2- 172.21.15.7:0/3407186582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841089a0 0x7f5f8410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.621+0000 7f5f8c000640 1 -- 172.21.15.7:0/3407186582 >> 172.21.15.7:0/3407186582 conn(0x7f5f8406ea10 msgr2=0x7f5f84070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.622+0000 7f5f8c000640 1 -- 172.21.15.7:0/3407186582 shutdown_connections 2024-06-13T07:02:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.622+0000 7f5f8c000640 1 -- 172.21.15.7:0/3407186582 wait complete. 2024-06-13T07:02:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.623+0000 7f5f8c000640 1 Processor -- start 2024-06-13T07:02:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.623+0000 7f5f8c000640 1 -- start start 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f841089a0 0x7f5f8410f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 0x7f5f8410fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f84118090 con 0x7f5f841089a0 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f84117f10 con 0x7f5f841120e0 2024-06-13T07:02:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8c000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f84118210 con 0x7f5f8410b310 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f89600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f841089a0 0x7f5f8410f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 0x7f5f8410fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57222/0 (socket says 172.21.15.7:57222) 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f8a000640 1 -- 172.21.15.7:0/4254205575 learned_addr learned my addr 172.21.15.7:0/4254205575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.624+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 0x7f5f8410fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36104/0 (socket says 172.21.15.7:36104) 2024-06-13T07:02:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 -- 172.21.15.7:0/4254205575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 msgr2=0x7f5f8410fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 0x7f5f8410fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 -- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f841089a0 msgr2=0x7f5f8410f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f841089a0 0x7f5f8410f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 -- 172.21.15.7:0/4254205575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8410ca10 con 0x7f5f841120e0 2024-06-13T07:02:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.625+0000 7f5f8a000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.626+0000 7f5f82000640 1 -- 172.21.15.7:0/4254205575 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c041020 con 0x7f5f841120e0 2024-06-13T07:02:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.626+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8410cca0 con 0x7f5f841120e0 2024-06-13T07:02:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.626+0000 7f5f82000640 1 -- 172.21.15.7:0/4254205575 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c030740 con 0x7f5f841120e0 2024-06-13T07:02:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.626+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f841a45e0 con 0x7f5f841120e0 2024-06-13T07:02:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.626+0000 7f5f82000640 1 -- 172.21.15.7:0/4254205575 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c036040 con 0x7f5f841120e0 2024-06-13T07:02:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.639+0000 7f5f8efc0740 1 Processor -- start 2024-06-13T07:02:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.639+0000 7f5f82000640 1 -- 172.21.15.7:0/4254205575 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5f7c044020 con 0x7f5f841120e0 2024-06-13T07:02:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.642+0000 7f5f82000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 0x7f5f6c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.642+0000 7f5f82000640 1 -- 172.21.15.7:0/4254205575 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5f7c0f0250 con 0x7f5f841120e0 2024-06-13T07:02:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.643+0000 7f5f88c00640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 0x7f5f6c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.643+0000 7f5f8efc0740 1 -- start start 2024-06-13T07:02:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.643+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 0x560f531de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.643+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.643+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 0x560f531e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f53141420 con 0x560f531ddc00 2024-06-13T07:02:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 0x560f531e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36112/0 (socket says 172.21.15.7:36112) 2024-06-13T07:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f88c00640 1 -- 172.21.15.7:0/1941890022 learned_addr learned my addr 172.21.15.7:0/1941890022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560f530e8c50 con 0x560f531e3760 2024-06-13T07:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f5313d3e0 con 0x560f531de5f0 2024-06-13T07:02:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f89600640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 0x560f531de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f88c00640 1 -- 172.21.15.7:0/1941890022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 msgr2=0x560f531e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.644+0000 7f5f88c00640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 0x560f531e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f88c00640 1 -- 172.21.15.7:0/1941890022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 msgr2=0x560f531de020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f88c00640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 0x560f531de020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f88c00640 1 -- 172.21.15.7:0/1941890022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f52e55210 con 0x560f531de5f0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f89600640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 0x560f531de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f8a000640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 0x560f531e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f88c00640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 0x7f5f6c0a51f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f74013570 tx=0x7f5f74013970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.645+0000 7f5f88c00640 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400f8d0 tx=0x7f5f74019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b9980d2878aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.646+0000 7f5f72600640 1 -- 172.21.15.7:0/1941890022 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74025070 con 0x560f531de5f0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.646+0000 7f5f72600640 1 -- 172.21.15.7:0/1941890022 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7400a7b0 con 0x560f531de5f0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.646+0000 7f5f72600640 1 -- 172.21.15.7:0/1941890022 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74020990 con 0x560f531de5f0 2024-06-13T07:02:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.647+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 msgr2=0x560f531e30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.647+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400f8d0 tx=0x7f5f74019000 comp rx=0 tx=0).stop 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.648+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 shutdown_connections 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.648+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531e3760 0x560f531e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.648+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e30f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.648+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/1941890022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531ddc00 0x560f531de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.648+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 >> 172.21.15.7:0/1941890022 conn(0x560f531a5fb0 msgr2=0x560f531d9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.649+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 shutdown_connections 2024-06-13T07:02:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.649+0000 7f5f8efc0740 1 -- 172.21.15.7:0/1941890022 wait complete. 2024-06-13T07:02:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.651+0000 7f5f8efc0740 1 -- start start 2024-06-13T07:02:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.651+0000 7f5f8efc0740 1 Processor -- start 2024-06-13T07:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f531e66b0 con 0x560f531e3760 2024-06-13T07:02:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560f530fc0b0 con 0x560f531ddc00 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f531e6830 con 0x560f531de5f0 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.652+0000 7f5f8efc0740 10 client.0 _ll_register_callbacks cb 0x560f530e7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.653+0000 7f5f89600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.653+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.653+0000 7f5f88c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36114/0 (socket says 172.21.15.7:36114) 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.653+0000 7f5f88c00640 1 -- 172.21.15.7:0/921209537 learned_addr learned my addr 172.21.15.7:0/921209537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.653+0000 7f5f8a000640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.654+0000 7f5f88c00640 1 -- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 msgr2=0x560f531e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.654+0000 7f5f88c00640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.654+0000 7f5f88c00640 1 -- 172.21.15.7:0/921209537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 msgr2=0x560f531ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.654+0000 7f5f88c00640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.654+0000 7f5f88c00640 1 -- 172.21.15.7:0/921209537 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f52f21620 con 0x560f531de5f0 2024-06-13T07:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.655+0000 7f5f88c00640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5f740058c0 tx=0x7f5f7400df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9ef59e3ba6f3a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.655+0000 7f5f66a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.655+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402d020 con 0x560f531de5f0 2024-06-13T07:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.655+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f5314eb50 con 0x560f531de5f0 2024-06-13T07:02:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.655+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5f74020710 con 0x560f531de5f0 2024-06-13T07:02:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.657+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5f74038050 con 0x560f531de5f0 2024-06-13T07:02:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.658+0000 7f5f8efc0740 10 client.26368 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.658+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560f531f2790 con 0x560f531de5f0 2024-06-13T07:02:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560f5314eb50 con 0x560f531de5f0 2024-06-13T07:02:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f5f74028070 con 0x560f531de5f0 2024-06-13T07:02:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 10 client.26368 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 20 client.26368 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f531f3c00 0x560f531f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 4 client.26368 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f53148ea0 con 0x560f531f3c00 2024-06-13T07:02:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.659+0000 7f5f89600640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f531f3c00 0x560f531f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.661+0000 7f5f89600640 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f531f3c00 0x560f531f5ff0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141cceaa0e15116c server_cookie=1026960469ebfd05 in_seq=0 out_seq=0 2024-06-13T07:02:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.661+0000 7f5f66a00640 10 client.26368 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.662+0000 7f5f66a00640 1 -- 172.21.15.7:0/921209537 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560f53148ea0 con 0x560f531f3c00 2024-06-13T07:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.662+0000 7f5f66a00640 10 client.26368 handle_command_reply: tid=0 2024-06-13T07:02:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.662+0000 7f5f8efc0740 1 client.26368 shutdown 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 msgr2=0x560f531e8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5f740058c0 tx=0x7f5f7400df80 comp rx=0 tx=0).stop 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 shutdown_connections 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f531f3c00 0x560f531f5ff0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531de5f0 0x560f531e8440 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5f740058c0 tx=0x7f5f7400df80 comp rx=0 tx=0).stop 2024-06-13T07:02:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f66a00640 0 client.26368 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/921209537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.663+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 >> 172.21.15.7:0/921209537 conn(0x560f531a5fb0 msgr2=0x560f531d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.665+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 shutdown_connections 2024-06-13T07:02:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.665+0000 7f5f8efc0740 1 -- 172.21.15.7:0/921209537 wait complete. 2024-06-13T07:02:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.665+0000 7f5f8efc0740 20 client.26368 trim_cache size 0 max 16384 2024-06-13T07:02:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.704+0000 7f5f8efc0740 1 Processor -- start 2024-06-13T07:02:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.704+0000 7f5f8efc0740 1 -- start start 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531ddc00 0x560f531dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f68012340 con 0x560f531e3760 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6800fbe0 con 0x560f531d9990 2024-06-13T07:02:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6800da30 con 0x560f531ddc00 2024-06-13T07:02:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f89600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531ddc00 0x560f531dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57254/0 (socket says 172.21.15.7:57254) 2024-06-13T07:02:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.705+0000 7f5f8a000640 1 -- 172.21.15.7:0/3752097214 learned_addr learned my addr 172.21.15.7:0/3752097214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.706+0000 7f5f88c00640 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.706+0000 7f5f8a000640 1 -- 172.21.15.7:0/3752097214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531ddc00 msgr2=0x560f531dcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.706+0000 7f5f8a000640 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531ddc00 0x560f531dcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.706+0000 7f5f8a000640 1 -- 172.21.15.7:0/3752097214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 msgr2=0x560f531ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.707+0000 7f5f8a000640 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.708+0000 7f5f8a000640 1 -- 172.21.15.7:0/3752097214 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f531f6360 con 0x560f531d9990 2024-06-13T07:02:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.708+0000 7f5f88c00640 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:02:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.709+0000 7f5f8a000640 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031040 tx=0x7f5f7c02f4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b33ec316ee0fef5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.709+0000 7f5f71c00640 1 -- 172.21.15.7:0/3752097214 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0d5070 con 0x560f531d9990 2024-06-13T07:02:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.709+0000 7f5f71c00640 1 -- 172.21.15.7:0/3752097214 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c0c0ac0 con 0x560f531d9990 2024-06-13T07:02:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.709+0000 7f5f71c00640 1 -- 172.21.15.7:0/3752097214 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0bcbd0 con 0x560f531d9990 2024-06-13T07:02:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.711+0000 7f5f8efc0740 1 -- 172.21.15.7:0/3752097214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 msgr2=0x560f531fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.711+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031040 tx=0x7f5f7c02f4b0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 -- 172.21.15.7:0/3752097214 shutdown_connections 2024-06-13T07:02:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531d9990 0x560f531fa8d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/3752097214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531ddc00 0x560f531dcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 -- 172.21.15.7:0/3752097214 >> 172.21.15.7:0/3752097214 conn(0x560f531a5fb0 msgr2=0x560f531dc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 -- 172.21.15.7:0/3752097214 shutdown_connections 2024-06-13T07:02:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.712+0000 7f5f8efc0740 1 -- 172.21.15.7:0/3752097214 wait complete. 2024-06-13T07:02:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.713+0000 7f5f8efc0740 1 -- start start 2024-06-13T07:02:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.713+0000 7f5f8efc0740 1 Processor -- start 2024-06-13T07:02:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6800e0a0 con 0x560f531e3760 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6800f510 con 0x560f531ddc00 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6800e770 con 0x560f531d9990 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8efc0740 10 client.0 _ll_register_callbacks cb 0x7f5f74023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f89600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f89600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:36140/0 (socket says 172.21.15.7:36140) 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f89600640 1 -- 172.21.15.7:0/4980627 learned_addr learned my addr 172.21.15.7:0/4980627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f8a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.714+0000 7f5f88c00640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 -- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 msgr2=0x560f531ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 -- 172.21.15.7:0/4980627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 msgr2=0x560f531ed420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ed420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 -- 172.21.15.7:0/4980627 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f531edb90 con 0x560f531d9990 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f89600640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800dc10 tx=0x7f5f7801b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfc75113ae4edb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.715+0000 7f5f66a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78011850 con 0x560f531d9990 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5f780119f0 con 0x560f531d9990 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c0c0ac0 con 0x560f531d9990 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5f78011c10 con 0x560f531d9990 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f8efc0740 10 client.26374 fetch_fsmap learned FSMap version 37 2024-06-13T07:02:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560f5325f340 con 0x560f531d9990 2024-06-13T07:02:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c0c0ac0 con 0x560f531d9990 2024-06-13T07:02:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.716+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f5f78022070 con 0x560f531d9990 2024-06-13T07:02:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 10 client.26374 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:02:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:02:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 20 client.26374 populate_metadata read hostname 'smithi007' 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f5325fd20 0x560f53262140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 4 client.26374 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f530c67a0 con 0x560f5325fd20 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.717+0000 7f5f88c00640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f5325fd20 0x560f53262140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.718+0000 7f5f66a00640 10 client.26374 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.718+0000 7f5f88c00640 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f5325fd20 0x560f53262140 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2e0636287d75da server_cookie=c747a3b4437dcaee in_seq=0 out_seq=0 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f66a00640 1 -- 172.21.15.7:0/4980627 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560f530c67a0 con 0x560f5325fd20 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f66a00640 10 client.26374 handle_command_reply: tid=0 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 client.26374 shutdown 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 msgr2=0x560f531eed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800dc10 tx=0x7f5f7801b6d0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 shutdown_connections 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560f5325fd20 0x560f53262140 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f531e3760 0x560f531ed420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560f531ddc00 0x560f531ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 --2- 172.21.15.7:0/4980627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f531d9990 0x560f531eed30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f66a00640 0 client.26374 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.734+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 >> 172.21.15.7:0/4980627 conn(0x560f531a5fb0 msgr2=0x560f531dcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 shutdown_connections 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8efc0740 1 -- 172.21.15.7:0/4980627 wait complete. 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8efc0740 20 client.26374 trim_cache size 0 max 16384 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:02:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:02:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 msgr2=0x7f5f6c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 0x7f5f6c0a51f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f74013570 tx=0x7f5f74013970 comp rx=0 tx=0).stop 2024-06-13T07:02:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 msgr2=0x7f5f8410c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:02:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.735+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c0305d0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 shutdown_connections 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5f6c0a2d30 0x7f5f6c0a51f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f841120e0 0x7f5f8410c150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f8410b310 0x7f5f8410fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 --2- 172.21.15.7:0/4254205575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f841089a0 0x7f5f8410f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 >> 172.21.15.7:0/4254205575 conn(0x7f5f8406ea10 msgr2=0x7f5f8410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 shutdown_connections 2024-06-13T07:02:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:02:56.736+0000 7f5f8c000640 1 -- 172.21.15.7:0/4254205575 wait complete. 2024-06-13T07:02:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:57 smithi189 ceph-mon[36616]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-06-13T07:02:57.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:57 smithi175 ceph-mon[36886]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-06-13T07:02:57.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:57 smithi007 ceph-mon[32077]: pgmap v716: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-06-13T07:02:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 415.0 s, 812.6 tps, lat 121.348 ms stddev 154.853 2024-06-13T07:02:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:02:59 smithi189 ceph-mon[36616]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 597 op/s 2024-06-13T07:02:59.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:02:59 smithi175 ceph-mon[36886]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 597 op/s 2024-06-13T07:02:59.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:02:59 smithi007 ceph-mon[32077]: pgmap v717: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 597 op/s 2024-06-13T07:03:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:01 smithi175 ceph-mon[36886]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-06-13T07:03:01.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:01 smithi007 ceph-mon[32077]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-06-13T07:03:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:01 smithi189 ceph-mon[36616]: pgmap v718: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-06-13T07:03:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 420.0 s, 580.4 tps, lat 176.972 ms stddev 203.519 2024-06-13T07:03:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:03 smithi175 ceph-mon[36886]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-06-13T07:03:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:03 smithi007 ceph-mon[32077]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-06-13T07:03:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:03 smithi189 ceph-mon[36616]: pgmap v719: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-06-13T07:03:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:05 smithi175 ceph-mon[36886]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2024-06-13T07:03:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:05 smithi007 ceph-mon[32077]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2024-06-13T07:03:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:05 smithi189 ceph-mon[36616]: pgmap v720: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2024-06-13T07:03:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:07 smithi175 ceph-mon[36886]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 451 op/s 2024-06-13T07:03:07.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:07 smithi007 ceph-mon[32077]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 451 op/s 2024-06-13T07:03:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:07 smithi189 ceph-mon[36616]: pgmap v721: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 451 op/s 2024-06-13T07:03:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 425.0 s, 702.8 tps, lat 142.802 ms stddev 154.984 2024-06-13T07:03:09.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:03:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:09 smithi175 ceph-mon[36886]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 601 op/s 2024-06-13T07:03:09.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:09 smithi007 ceph-mon[32077]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 601 op/s 2024-06-13T07:03:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:09 smithi189 ceph-mon[36616]: pgmap v722: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 601 op/s 2024-06-13T07:03:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 -- 172.21.15.7:0/1972528918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990103410 msgr2=0x7fd990109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 --2- 172.21.15.7:0/1972528918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990103410 0x7fd990109ca0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c02a410 comp rx=0 tx=0).stop 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 -- 172.21.15.7:0/1972528918 shutdown_connections 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 --2- 172.21.15.7:0/1972528918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990103410 0x7fd990109ca0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 --2- 172.21.15.7:0/1972528918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990102a50 0x7fd990102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 --2- 172.21.15.7:0/1972528918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd990101860 0x7fd990101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 -- 172.21.15.7:0/1972528918 >> 172.21.15.7:0/1972528918 conn(0x7fd9900fcdd0 msgr2=0x7fd9900ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 -- 172.21.15.7:0/1972528918 shutdown_connections 2024-06-13T07:03:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.980+0000 7fd98f200640 1 -- 172.21.15.7:0/1972528918 wait complete. 2024-06-13T07:03:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.983+0000 7fd98f200640 1 Processor -- start 2024-06-13T07:03:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 -- start start 2024-06-13T07:03:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990101860 0x7fd990197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd990103410 0x7fd99019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d550 con 0x7fd990103410 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d3d0 con 0x7fd990101860 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.985+0000 7fd98f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd99010d6d0 con 0x7fd990102a50 2024-06-13T07:03:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.986+0000 7fd98d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.986+0000 7fd98d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:59606/0 (socket says 172.21.15.7:59606) 2024-06-13T07:03:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.986+0000 7fd98d400640 1 -- 172.21.15.7:0/194945017 learned_addr learned my addr 172.21.15.7:0/194945017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98de00640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990101860 0x7fd990197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98d400640 1 -- 172.21.15.7:0/194945017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990101860 msgr2=0x7fd990197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98d400640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990101860 0x7fd990197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98d400640 1 -- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd990103410 msgr2=0x7fd99019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98d400640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd990103410 0x7fd99019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.988+0000 7fd98d400640 1 -- 172.21.15.7:0/194945017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99019f780 con 0x7fd990102a50 2024-06-13T07:03:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.990+0000 7fd98d400640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd98000bea0 tx=0x7fd98000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.994+0000 7fd986a00640 1 -- 172.21.15.7:0/194945017 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980018070 con 0x7fd990102a50 2024-06-13T07:03:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.994+0000 7fd986a00640 1 -- 172.21.15.7:0/194945017 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd980012c40 con 0x7fd990102a50 2024-06-13T07:03:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.995+0000 7fd986a00640 1 -- 172.21.15.7:0/194945017 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98001b8f0 con 0x7fd990102a50 2024-06-13T07:03:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.995+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd990073c30 con 0x7fd990102a50 2024-06-13T07:03:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:09.996+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9900741b0 con 0x7fd990102a50 2024-06-13T07:03:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.002+0000 7fd9979a0740 1 Processor -- start 2024-06-13T07:03:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.005+0000 7fd986a00640 1 -- 172.21.15.7:0/194945017 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd980012760 con 0x7fd990102a50 2024-06-13T07:03:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.008+0000 7fd9979a0740 1 -- start start 2024-06-13T07:03:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.008+0000 7fd986a00640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 0x7fd9740a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.009+0000 7fd986a00640 1 -- 172.21.15.7:0/194945017 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd9800d3390 con 0x7fd990102a50 2024-06-13T07:03:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d8688db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607d85e4420 con 0x5607d8686920 2024-06-13T07:03:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607d858bc50 con 0x5607d8680dc0 2024-06-13T07:03:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.010+0000 7fd9979a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607d85e03e0 con 0x5607d86817b0 2024-06-13T07:03:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d8688db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d8688db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53794/0 (socket says 172.21.15.7:53794) 2024-06-13T07:03:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98e800640 1 -- 172.21.15.7:0/4102085115 learned_addr learned my addr 172.21.15.7:0/4102085115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98d400640 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98de00640 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98d400640 1 -- 172.21.15.7:0/4102085115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 msgr2=0x5607d86811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.011+0000 7fd98d400640 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd98d400640 1 -- 172.21.15.7:0/4102085115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 msgr2=0x5607d8688db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd98d400640 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d8688db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd98d400640 1 -- 172.21.15.7:0/4102085115 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d82f8210 con 0x5607d86817b0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd98e800640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 0x7fd9740a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd98d400640 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86862b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd98009f550 tx=0x7fd98009f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca2833fb0267f2f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd97b000640 1 -- 172.21.15.7:0/4102085115 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800d3170 con 0x5607d86817b0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd97b000640 1 -- 172.21.15.7:0/4102085115 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98009f850 con 0x5607d86817b0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.012+0000 7fd97b000640 1 -- 172.21.15.7:0/4102085115 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98009bec0 con 0x5607d86817b0 2024-06-13T07:03:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.013+0000 7fd9979a0740 1 -- 172.21.15.7:0/4102085115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 msgr2=0x5607d86862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.013+0000 7fd98e800640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 0x7fd9740a5180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd97c031610 tx=0x7fd97c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.013+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86862b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd98009f550 tx=0x7fd98009f480 comp rx=0 tx=0).stop 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.014+0000 7fd9979a0740 1 -- 172.21.15.7:0/4102085115 shutdown_connections 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.014+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d8688db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.014+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86862b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.014+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4102085115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.014+0000 7fd9979a0740 1 -- 172.21.15.7:0/4102085115 >> 172.21.15.7:0/4102085115 conn(0x5607d8648fb0 msgr2=0x5607d867c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.015+0000 7fd9979a0740 1 -- 172.21.15.7:0/4102085115 shutdown_connections 2024-06-13T07:03:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.015+0000 7fd9979a0740 1 -- 172.21.15.7:0/4102085115 wait complete. 2024-06-13T07:03:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.018+0000 7fd9979a0740 1 -- start start 2024-06-13T07:03:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.018+0000 7fd9979a0740 1 Processor -- start 2024-06-13T07:03:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d86817b0 0x5607d869c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d8686920 0x5607d86a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607d869a880 con 0x5607d86817b0 2024-06-13T07:03:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5607d859f0b0 con 0x5607d8680dc0 2024-06-13T07:03:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607d869aa00 con 0x5607d8686920 2024-06-13T07:03:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.019+0000 7fd9979a0740 10 client.0 _ll_register_callbacks cb 0x5607d858af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55840/0 (socket says 172.21.15.7:55840) 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 -- 172.21.15.7:0/765839753 learned_addr learned my addr 172.21.15.7:0/765839753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98d400640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d86817b0 0x5607d869c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 -- 172.21.15.7:0/765839753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d8686920 msgr2=0x5607d86a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d8686920 0x5607d86a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 -- 172.21.15.7:0/765839753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d86817b0 msgr2=0x5607d869c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d86817b0 0x5607d869c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.020+0000 7fd98de00640 1 -- 172.21.15.7:0/765839753 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d83c4620 con 0x5607d8680dc0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.021+0000 7fd98de00640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd98800aae0 tx=0x7fd9880072b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99338dd32a7ca254 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.021+0000 7fd96f400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.023+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988016410 con 0x5607d8680dc0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.023+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd9880165b0 con 0x5607d8680dc0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.023+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98009f850 con 0x5607d8680dc0 2024-06-13T07:03:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.024+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd9880167b0 con 0x5607d8680dc0 2024-06-13T07:03:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.025+0000 7fd9979a0740 10 client.26438 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.025+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607d86a6ac0 con 0x5607d8680dc0 2024-06-13T07:03:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98009f850 con 0x5607d8680dc0 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd988015ab0 con 0x5607d8680dc0 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd9979a0740 10 client.26438 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd9979a0740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd9979a0740 20 client.26438 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.029+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d86a81c0 0x5607d86aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.030+0000 7fd9979a0740 4 client.26438 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.030+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607d85f1b50 con 0x5607d86a81c0 2024-06-13T07:03:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.030+0000 7fd98d400640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d86a81c0 0x5607d86aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.031+0000 7fd98d400640 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d86a81c0 0x5607d86aa650 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8adfdee6b3c8bc1f server_cookie=8d8cc41e730ea90e in_seq=0 out_seq=0 2024-06-13T07:03:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.031+0000 7fd96f400640 10 client.26438 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.033+0000 7fd96f400640 1 -- 172.21.15.7:0/765839753 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607d85f1b50 con 0x5607d86a81c0 2024-06-13T07:03:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.033+0000 7fd96f400640 10 client.26438 handle_command_reply: tid=0 2024-06-13T07:03:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.033+0000 7fd9979a0740 1 client.26438 shutdown 2024-06-13T07:03:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.034+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 msgr2=0x5607d869c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.034+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd98800aae0 tx=0x7fd9880072b0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 shutdown_connections 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d86a81c0 0x5607d86aa650 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d8686920 0x5607d86a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d86817b0 0x5607d869c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd96f400640 0 client.26438 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 --2- 172.21.15.7:0/765839753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d869c160 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.035+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 >> 172.21.15.7:0/765839753 conn(0x5607d8648fb0 msgr2=0x5607d867b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.036+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 shutdown_connections 2024-06-13T07:03:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.037+0000 7fd9979a0740 1 -- 172.21.15.7:0/765839753 wait complete. 2024-06-13T07:03:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.037+0000 7fd9979a0740 20 client.26438 trim_cache size 0 max 16384 2024-06-13T07:03:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.078+0000 7fd9979a0740 1 Processor -- start 2024-06-13T07:03:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 -- start start 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8686920 0x5607d8682c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9901a2360 con 0x5607d8680dc0 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd99019fc00 con 0x5607d8686920 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd9979a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd990071d20 con 0x5607d86817b0 2024-06-13T07:03:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.079+0000 7fd98de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.080+0000 7fd98de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53816/0 (socket says 172.21.15.7:53816) 2024-06-13T07:03:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.080+0000 7fd98d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.081+0000 7fd98de00640 1 -- 172.21.15.7:0/3252835307 learned_addr learned my addr 172.21.15.7:0/3252835307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.080+0000 7fd98e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8686920 0x5607d8682c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 -- 172.21.15.7:0/3252835307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 msgr2=0x5607d86a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 -- 172.21.15.7:0/3252835307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8686920 msgr2=0x5607d8682c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8686920 0x5607d8682c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 -- 172.21.15.7:0/3252835307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d8685e60 con 0x5607d8680dc0 2024-06-13T07:03:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98d400640 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.082+0000 7fd98de00640 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd988004b90 tx=0x7fd988007080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4770fc9d78258d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.083+0000 7fd97a600640 1 -- 172.21.15.7:0/3252835307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98802e070 con 0x5607d8680dc0 2024-06-13T07:03:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.084+0000 7fd97a600640 1 -- 172.21.15.7:0/3252835307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98801b070 con 0x5607d8680dc0 2024-06-13T07:03:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.084+0000 7fd97a600640 1 -- 172.21.15.7:0/3252835307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988015fa0 con 0x5607d8680dc0 2024-06-13T07:03:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.085+0000 7fd9979a0740 1 -- 172.21.15.7:0/3252835307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 msgr2=0x5607d8677af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.085+0000 7fd9979a0740 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd988004b90 tx=0x7fd988007080 comp rx=0 tx=0).stop 2024-06-13T07:03:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 -- 172.21.15.7:0/3252835307 shutdown_connections 2024-06-13T07:03:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8686920 0x5607d8682c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 --2- 172.21.15.7:0/3252835307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8680dc0 0x5607d8677af0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 -- 172.21.15.7:0/3252835307 >> 172.21.15.7:0/3252835307 conn(0x5607d8648fb0 msgr2=0x5607d867b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 -- 172.21.15.7:0/3252835307 shutdown_connections 2024-06-13T07:03:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.086+0000 7fd9979a0740 1 -- 172.21.15.7:0/3252835307 wait complete. 2024-06-13T07:03:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.087+0000 7fd9979a0740 1 -- start start 2024-06-13T07:03:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.088+0000 7fd9979a0740 1 Processor -- start 2024-06-13T07:03:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.091+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.091+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.091+0000 7fd9979a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d86a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd9979a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd990072390 con 0x5607d8686920 2024-06-13T07:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd9979a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd990073800 con 0x5607d8680dc0 2024-06-13T07:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd9979a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd990072a60 con 0x5607d86817b0 2024-06-13T07:03:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd9979a0740 10 client.0 _ll_register_callbacks cb 0x7fd988009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd98d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd98de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd98de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55860/0 (socket says 172.21.15.7:55860) 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.092+0000 7fd98de00640 1 -- 172.21.15.7:0/4088545946 learned_addr learned my addr 172.21.15.7:0/4088545946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 -- 172.21.15.7:0/4088545946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 msgr2=0x5607d86a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98e800640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d86a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 -- 172.21.15.7:0/4088545946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 msgr2=0x5607d86a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d86a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 -- 172.21.15.7:0/4088545946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d86a2210 con 0x5607d8680dc0 2024-06-13T07:03:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98e800640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d86a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd98de00640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd98802c040 tx=0x7fd9880077b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c172298b5650c2a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.093+0000 7fd96f400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.098+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98802e070 con 0x5607d8680dc0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.098+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd988016b50 con 0x5607d8680dc0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.098+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98801b070 con 0x5607d8680dc0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.099+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd988007ea0 con 0x5607d8680dc0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.099+0000 7fd9979a0740 10 client.26444 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.099+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607d86a3520 con 0x5607d8680dc0 2024-06-13T07:03:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.099+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98801b070 con 0x5607d8680dc0 2024-06-13T07:03:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd988015530 con 0x5607d8680dc0 2024-06-13T07:03:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 10 client.26444 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 10 client.26444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 20 client.26444 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d871b970 0x5607d871dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 4 client.26444 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607d85ebea0 con 0x5607d871b970 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.100+0000 7fd98d400640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d871b970 0x5607d871dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.101+0000 7fd98d400640 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d871b970 0x5607d871dd90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c282bb539fa938a5 server_cookie=6f5bca4ea91060a9 in_seq=0 out_seq=0 2024-06-13T07:03:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.101+0000 7fd96f400640 10 client.26444 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.151+0000 7fd96f400640 1 -- 172.21.15.7:0/4088545946 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607d85ebea0 con 0x5607d871b970 2024-06-13T07:03:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.151+0000 7fd96f400640 10 client.26444 handle_command_reply: tid=0 2024-06-13T07:03:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.151+0000 7fd9979a0740 1 client.26444 shutdown 2024-06-13T07:03:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 msgr2=0x5607d86a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd98802c040 tx=0x7fd9880077b0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 shutdown_connections 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5607d871b970 0x5607d871dd90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607d8686920 0x5607d86a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607d86817b0 0x5607d86a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 --2- 172.21.15.7:0/4088545946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607d8680dc0 0x5607d86a06d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd96f400640 0 client.26444 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.152+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 >> 172.21.15.7:0/4088545946 conn(0x5607d8648fb0 msgr2=0x5607d8682c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.153+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 shutdown_connections 2024-06-13T07:03:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.153+0000 7fd9979a0740 1 -- 172.21.15.7:0/4088545946 wait complete. 2024-06-13T07:03:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.153+0000 7fd9979a0740 20 client.26444 trim_cache size 0 max 16384 2024-06-13T07:03:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:03:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:03:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:03:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 msgr2=0x7fd9740a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 0x7fd9740a5180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd97c031610 tx=0x7fd97c005fd0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 msgr2=0x7fd9901980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd98000bea0 tx=0x7fd98000bf90 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 shutdown_connections 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd9740a2cc0 0x7fd9740a5180 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd990103410 0x7fd99019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd990102a50 0x7fd9901980a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 --2- 172.21.15.7:0/194945017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd990101860 0x7fd990197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 >> 172.21.15.7:0/194945017 conn(0x7fd9900fcdd0 msgr2=0x7fd990105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 shutdown_connections 2024-06-13T07:03:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:10.155+0000 7fd98f200640 1 -- 172.21.15.7:0/194945017 wait complete. 2024-06-13T07:03:11.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:11 smithi007 ceph-mon[32077]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 509 op/s 2024-06-13T07:03:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:11 smithi189 ceph-mon[36616]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 509 op/s 2024-06-13T07:03:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:11 smithi175 ceph-mon[36886]: pgmap v723: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.0 MiB/s wr, 509 op/s 2024-06-13T07:03:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 430.0 s, 511.2 tps, lat 193.963 ms stddev 229.613 2024-06-13T07:03:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 -- 172.21.15.7:0/1638028111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa244104ed0 msgr2=0x7fa244105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 --2- 172.21.15.7:0/1638028111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa244104ed0 0x7fa244105360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009990 tx=0x7fa23c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 -- 172.21.15.7:0/1638028111 shutdown_connections 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 --2- 172.21.15.7:0/1638028111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa244104ed0 0x7fa244105360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 --2- 172.21.15.7:0/1638028111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2441073c0 0x7fa24406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 --2- 172.21.15.7:0/1638028111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa24406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.963+0000 7fa249e00700 1 -- 172.21.15.7:0/1638028111 >> 172.21.15.7:0/1638028111 conn(0x7fa244078340 msgr2=0x7fa244078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.964+0000 7fa249e00700 1 -- 172.21.15.7:0/1638028111 shutdown_connections 2024-06-13T07:03:12.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.964+0000 7fa249e00700 1 -- 172.21.15.7:0/1638028111 wait complete. 2024-06-13T07:03:12.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 Processor -- start 2024-06-13T07:03:12.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 -- start start 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa244115710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 0x7fa244115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 0x7fa24410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa244118da0 con 0x7fa244104ed0 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa244118c20 con 0x7fa24406a6e0 2024-06-13T07:03:12.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.966+0000 7fa249e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa244118f20 con 0x7fa2441073c0 2024-06-13T07:03:12.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.967+0000 7fa23a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 0x7fa244115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:12.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.968+0000 7fa23a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 0x7fa244115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39636/0 (socket says 172.21.15.7:39636) 2024-06-13T07:03:12.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.968+0000 7fa23a000700 1 -- 172.21.15.7:0/1938409719 learned_addr learned my addr 172.21.15.7:0/1938409719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:12.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.968+0000 7fa242a00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa244115710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:12.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.968+0000 7fa243400700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 0x7fa24410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa242a00700 1 -- 172.21.15.7:0/1938409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 msgr2=0x7fa24410e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa242a00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 0x7fa24410e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa242a00700 1 -- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 msgr2=0x7fa244115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa242a00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 0x7fa244115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa242a00700 1 -- 172.21.15.7:0/1938409719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24410f2f0 con 0x7fa24406a6e0 2024-06-13T07:03:12.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.970+0000 7fa243400700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 0x7fa24410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:12.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.973+0000 7fa242a00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa244115710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e9f0 tx=0x7fa22c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:12.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.976+0000 7fa248a00700 1 -- 172.21.15.7:0/1938409719 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00c9c0 con 0x7fa24406a6e0 2024-06-13T07:03:12.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.976+0000 7fa248a00700 1 -- 172.21.15.7:0/1938409719 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c00cb60 con 0x7fa24406a6e0 2024-06-13T07:03:12.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.976+0000 7fa248a00700 1 -- 172.21.15.7:0/1938409719 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c018680 con 0x7fa24406a6e0 2024-06-13T07:03:12.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.976+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24410f580 con 0x7fa24406a6e0 2024-06-13T07:03:12.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.977+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2441134d0 con 0x7fa24406a6e0 2024-06-13T07:03:12.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.984+0000 7fa25485bb80 1 Processor -- start 2024-06-13T07:03:12.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.985+0000 7fa25485bb80 1 -- start start 2024-06-13T07:03:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.985+0000 7fa248a00700 1 -- 172.21.15.7:0/1938409719 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa22c010430 con 0x7fa24406a6e0 2024-06-13T07:03:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa248a00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 0x7fa2280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b913070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b9206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b822a10 con 0x55dc5b91e300 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b865b00 con 0x55dc5b9135b0 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa25485bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b8208f0 con 0x55dc5b88fe40 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa23a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa23a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53174/0 (socket says 172.21.15.7:53174) 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.988+0000 7fa23a000700 1 -- 172.21.15.7:0/3318726235 learned_addr learned my addr 172.21.15.7:0/3318726235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa23a000700 1 -- 172.21.15.7:0/3318726235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 msgr2=0x55dc5b913070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa23a000700 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b913070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa23a000700 1 -- 172.21.15.7:0/3318726235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 msgr2=0x55dc5b9206f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa23a000700 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b9206f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa23a000700 1 -- 172.21.15.7:0/3318726235 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5b80df90 con 0x55dc5b9135b0 2024-06-13T07:03:12.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.989+0000 7fa248a00700 1 -- 172.21.15.7:0/1938409719 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa22c0998a0 con 0x7fa24406a6e0 2024-06-13T07:03:12.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.990+0000 7fa243400700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 0x7fa2280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:12.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.991+0000 7fa23a000700 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa234009f60 tx=0x7fa2340092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8975f16a16a35f2f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:12.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.991+0000 7fa238c00700 1 -- 172.21.15.7:0/3318726235 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23401a070 con 0x55dc5b9135b0 2024-06-13T07:03:12.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.992+0000 7fa238c00700 1 -- 172.21.15.7:0/3318726235 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234003c00 con 0x55dc5b9135b0 2024-06-13T07:03:12.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.992+0000 7fa238c00700 1 -- 172.21.15.7:0/3318726235 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23401a070 con 0x55dc5b9135b0 2024-06-13T07:03:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.994+0000 7fa243400700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 0x7fa2280a5230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa23c006010 tx=0x7fa23c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.994+0000 7fa25485bb80 1 -- 172.21.15.7:0/3318726235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 msgr2=0x55dc5b913a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.994+0000 7fa25485bb80 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa234009f60 tx=0x7fa2340092b0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.995+0000 7fa25485bb80 1 -- 172.21.15.7:0/3318726235 shutdown_connections 2024-06-13T07:03:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.995+0000 7fa25485bb80 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b9206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.995+0000 7fa25485bb80 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b913070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.995+0000 7fa25485bb80 1 --2- 172.21.15.7:0/3318726235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b913a30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.995+0000 7fa25485bb80 1 -- 172.21.15.7:0/3318726235 >> 172.21.15.7:0/3318726235 conn(0x55dc5b840170 msgr2=0x55dc5b90f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.996+0000 7fa25485bb80 1 -- 172.21.15.7:0/3318726235 shutdown_connections 2024-06-13T07:03:12.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.996+0000 7fa25485bb80 1 -- 172.21.15.7:0/3318726235 wait complete. 2024-06-13T07:03:12.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.998+0000 7fa25485bb80 1 -- start start 2024-06-13T07:03:12.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:12.998+0000 7fa25485bb80 1 Processor -- start 2024-06-13T07:03:12.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.000+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b91dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.001+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.001+0000 7fa242a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b91dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.001+0000 7fa242a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b91dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44400/0 (socket says 172.21.15.7:44400) 2024-06-13T07:03:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.001+0000 7fa242a00700 1 -- 172.21.15.7:0/1112893321 learned_addr learned my addr 172.21.15.7:0/1112893321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.001+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b91b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.002+0000 7fa23a000700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.002+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b801f60 con 0x55dc5b91e300 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.002+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b861ac0 con 0x55dc5b9135b0 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.002+0000 7fa243400700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b91b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc5b920fc0 con 0x55dc5b88fe40 2024-06-13T07:03:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa25485bb80 10 client.0 _ll_register_callbacks cb 0x55dc5b7ff720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa23a000700 1 -- 172.21.15.7:0/1112893321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 msgr2=0x55dc5b91dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa23a000700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b91dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa23a000700 1 -- 172.21.15.7:0/1112893321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 msgr2=0x55dc5b91b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa23a000700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b91b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.003+0000 7fa23a000700 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5b7b48d0 con 0x55dc5b9135b0 2024-06-13T07:03:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.005+0000 7fa23a000700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91adc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa234009f60 tx=0x7fa234004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46605d09a7aca20 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.005+0000 7fa231600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.006+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23401a040 con 0x55dc5b9135b0 2024-06-13T07:03:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.006+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa234003a00 con 0x55dc5b9135b0 2024-06-13T07:03:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.009+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa234003c00 con 0x55dc5b9135b0 2024-06-13T07:03:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.010+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa2340145f0 con 0x55dc5b9135b0 2024-06-13T07:03:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.010+0000 7fa25485bb80 10 client.26456 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.010+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dc5b92b770 con 0x55dc5b9135b0 2024-06-13T07:03:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.010+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234003c00 con 0x55dc5b9135b0 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa23401d070 con 0x55dc5b9135b0 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 10 client.26456 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 20 client.26456 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b92dcf0 0x55dc5b9300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 4 client.26456 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc5b873000 con 0x55dc5b92dcf0 2024-06-13T07:03:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.012+0000 7fa242a00700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b92dcf0 0x55dc5b9300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.013+0000 7fa242a00700 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b92dcf0 0x55dc5b9300e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd7ffd1685af6fb server_cookie=a4c88dfcc2a3f6b4 in_seq=0 out_seq=0 2024-06-13T07:03:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.013+0000 7fa231600700 10 client.26456 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.016+0000 7fa231600700 1 -- 172.21.15.7:0/1112893321 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc5b873000 con 0x55dc5b92dcf0 2024-06-13T07:03:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.016+0000 7fa231600700 10 client.26456 handle_command_reply: tid=0 2024-06-13T07:03:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 client.26456 shutdown 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 msgr2=0x55dc5b91adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91adc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa234009f60 tx=0x7fa234004530 comp rx=0 tx=0).stop 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 shutdown_connections 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b92dcf0 0x55dc5b9300e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b91e300 0x55dc5b91b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b88fe40 0x55dc5b91dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 --2- 172.21.15.7:0/1112893321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91adc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 >> 172.21.15.7:0/1112893321 conn(0x55dc5b840170 msgr2=0x55dc5b90e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.017+0000 7fa231600700 0 client.26456 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.018+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 shutdown_connections 2024-06-13T07:03:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.019+0000 7fa25485bb80 1 -- 172.21.15.7:0/1112893321 wait complete. 2024-06-13T07:03:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.019+0000 7fa25485bb80 20 client.26456 trim_cache size 0 max 16384 2024-06-13T07:03:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.056+0000 7fa25485bb80 1 Processor -- start 2024-06-13T07:03:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 -- start start 2024-06-13T07:03:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b91d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b91e300 0x55dc5b91fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bfa00 con 0x55dc5b88fe40 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bd260 con 0x55dc5b9135b0 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa25485bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bb0b0 con 0x55dc5b91e300 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.057+0000 7fa23a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53190/0 (socket says 172.21.15.7:53190) 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 -- 172.21.15.7:0/210646627 learned_addr learned my addr 172.21.15.7:0/210646627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa243400700 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b91e300 0x55dc5b91fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 -- 172.21.15.7:0/210646627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b91e300 msgr2=0x55dc5b91fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b91e300 0x55dc5b91fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 -- 172.21.15.7:0/210646627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 msgr2=0x55dc5b91d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.058+0000 7fa23a000700 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b91d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.059+0000 7fa23a000700 1 -- 172.21.15.7:0/210646627 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5b91ab90 con 0x55dc5b9135b0 2024-06-13T07:03:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.059+0000 7fa23a000700 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa2340045f0 tx=0x7fa2340041f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2f0ee17a39794f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.059+0000 7fa233e00700 1 -- 172.21.15.7:0/210646627 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23401a040 con 0x55dc5b9135b0 2024-06-13T07:03:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.060+0000 7fa233e00700 1 -- 172.21.15.7:0/210646627 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234003a00 con 0x55dc5b9135b0 2024-06-13T07:03:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.060+0000 7fa233e00700 1 -- 172.21.15.7:0/210646627 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234004df0 con 0x55dc5b9135b0 2024-06-13T07:03:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 -- 172.21.15.7:0/210646627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 msgr2=0x55dc5b91f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa2340045f0 tx=0x7fa2340041f0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 -- 172.21.15.7:0/210646627 shutdown_connections 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b91e300 0x55dc5b91fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b91d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 --2- 172.21.15.7:0/210646627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b9135b0 0x55dc5b91f6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.061+0000 7fa25485bb80 1 -- 172.21.15.7:0/210646627 >> 172.21.15.7:0/210646627 conn(0x55dc5b840170 msgr2=0x55dc5b9196b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.062+0000 7fa25485bb80 1 -- 172.21.15.7:0/210646627 shutdown_connections 2024-06-13T07:03:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.062+0000 7fa25485bb80 1 -- 172.21.15.7:0/210646627 wait complete. 2024-06-13T07:03:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.063+0000 7fa25485bb80 1 -- start start 2024-06-13T07:03:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.063+0000 7fa25485bb80 1 Processor -- start 2024-06-13T07:03:13.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.065+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b933110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.065+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.066+0000 7fa25485bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b91e300 0x55dc5b917190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.066+0000 7fa25485bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bb720 con 0x55dc5b88fe40 2024-06-13T07:03:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa25485bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bcb90 con 0x55dc5b91e300 2024-06-13T07:03:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa23a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa23a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44422/0 (socket says 172.21.15.7:44422) 2024-06-13T07:03:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa23a000700 1 -- 172.21.15.7:0/4096494207 learned_addr learned my addr 172.21.15.7:0/4096494207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa243400700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b91e300 0x55dc5b917190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441bbdf0 con 0x55dc5b9135b0 2024-06-13T07:03:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.067+0000 7fa242a00700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b933110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa23a000700 1 -- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b91e300 msgr2=0x55dc5b917190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa23a000700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b91e300 0x55dc5b917190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa23a000700 1 -- 172.21.15.7:0/4096494207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 msgr2=0x55dc5b933110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa23a000700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b933110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa25485bb80 10 client.0 _ll_register_callbacks cb 0x55dc5b7ff720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa23a000700 1 -- 172.21.15.7:0/4096494207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5b9176d0 con 0x55dc5b9135b0 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.068+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa234003a00 con 0x55dc5b9135b0 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.069+0000 7fa23a000700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa234009b60 tx=0x7fa234004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbfabbd3249a53a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.069+0000 7fa231600700 10 client.26404 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.069+0000 7fa231600700 1 -- 172.21.15.7:0/4096494207 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23401a040 con 0x55dc5b9135b0 2024-06-13T07:03:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.069+0000 7fa231600700 1 -- 172.21.15.7:0/4096494207 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa234003bb0 con 0x55dc5b9135b0 2024-06-13T07:03:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa231600700 1 -- 172.21.15.7:0/4096494207 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa234007b70 con 0x55dc5b9135b0 2024-06-13T07:03:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa25485bb80 10 client.26404 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55dc5b997b80 con 0x55dc5b9135b0 2024-06-13T07:03:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa231600700 1 -- 172.21.15.7:0/4096494207 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa23401d070 con 0x55dc5b9135b0 2024-06-13T07:03:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa25485bb80 10 client.26404 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa25485bb80 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.070+0000 7fa25485bb80 20 client.26404 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.071+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b9980a0 0x55dc5b99a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.074+0000 7fa25485bb80 4 client.26404 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:03:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.074+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc5b86d580 con 0x55dc5b9980a0 2024-06-13T07:03:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.074+0000 7fa242a00700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b9980a0 0x55dc5b99a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.075+0000 7fa242a00700 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b9980a0 0x55dc5b99a4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29385ab7c0e1a07 server_cookie=9c4714290bdf3ab2 in_seq=0 out_seq=0 2024-06-13T07:03:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.075+0000 7fa231600700 10 client.26404 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.075+0000 7fa231600700 1 -- 172.21.15.7:0/4096494207 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc5b86d580 con 0x55dc5b9980a0 2024-06-13T07:03:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.075+0000 7fa231600700 10 client.26404 handle_command_reply: tid=0 2024-06-13T07:03:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.075+0000 7fa25485bb80 1 client.26404 shutdown 2024-06-13T07:03:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 msgr2=0x55dc5b933650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa234009b60 tx=0x7fa234004980 comp rx=0 tx=0).stop 2024-06-13T07:03:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 shutdown_connections 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc5b9980a0 0x55dc5b99a4c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc5b91e300 0x55dc5b917190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc5b88fe40 0x55dc5b933110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 --2- 172.21.15.7:0/4096494207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc5b9135b0 0x55dc5b933650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 >> 172.21.15.7:0/4096494207 conn(0x55dc5b840170 msgr2=0x55dc5b90deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa231600700 0 client.26404 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 shutdown_connections 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 1 -- 172.21.15.7:0/4096494207 wait complete. 2024-06-13T07:03:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.076+0000 7fa25485bb80 20 client.26404 trim_cache size 0 max 16384 2024-06-13T07:03:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:03:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:03:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:03:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 msgr2=0x7fa2280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 0x7fa2280a5230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa23c006010 tx=0x7fa23c00b260 comp rx=0 tx=0).stop 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 msgr2=0x7fa244115710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa244115710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e9f0 tx=0x7fa22c00ed00 comp rx=0 tx=0).stop 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 shutdown_connections 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2441073c0 0x7fa24410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2280a2d70 0x7fa2280a5230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24406a6e0 0x7fa244115710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 --2- 172.21.15.7:0/1938409719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa244104ed0 0x7fa244115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 >> 172.21.15.7:0/1938409719 conn(0x7fa244078340 msgr2=0x7fa244103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.077+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 shutdown_connections 2024-06-13T07:03:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:13.078+0000 7fa249e00700 1 -- 172.21.15.7:0/1938409719 wait complete. 2024-06-13T07:03:13.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:13 smithi189 ceph-mon[36616]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-06-13T07:03:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:13 smithi007 ceph-mon[32077]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-06-13T07:03:13.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:13 smithi175 ceph-mon[36886]: pgmap v724: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2024-06-13T07:03:13.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:03:13.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:03:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:15.450 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:15 smithi007 ceph-mon[32077]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-06-13T07:03:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:15 smithi189 ceph-mon[36616]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-06-13T07:03:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:15 smithi175 ceph-mon[36886]: pgmap v725: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 MiB/s wr, 634 op/s 2024-06-13T07:03:17.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:17 smithi007 ceph-mon[32077]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 MiB/s wr, 474 op/s 2024-06-13T07:03:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:17 smithi189 ceph-mon[36616]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 MiB/s wr, 474 op/s 2024-06-13T07:03:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:17 smithi175 ceph-mon[36886]: pgmap v726: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.7 MiB/s wr, 474 op/s 2024-06-13T07:03:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 435.0 s, 402.0 tps, lat 237.167 ms stddev 292.066 2024-06-13T07:03:19.610 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:19 smithi007 ceph-mon[32077]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-06-13T07:03:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:19 smithi189 ceph-mon[36616]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-06-13T07:03:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:19 smithi175 ceph-mon[36886]: pgmap v727: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 -- 172.21.15.7:0/958298756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 msgr2=0x7ff320109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 --2- 172.21.15.7:0/958298756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff320109490 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b0a0 tx=0x7ff30800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 -- 172.21.15.7:0/958298756 shutdown_connections 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 --2- 172.21.15.7:0/958298756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 0x7ff32011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 --2- 172.21.15.7:0/958298756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff320109490 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 --2- 172.21.15.7:0/958298756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff320109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.988+0000 7ff325e00700 1 -- 172.21.15.7:0/958298756 >> 172.21.15.7:0/958298756 conn(0x7ff32006d600 msgr2=0x7ff32006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:19.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.989+0000 7ff325e00700 1 -- 172.21.15.7:0/958298756 shutdown_connections 2024-06-13T07:03:19.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.990+0000 7ff325e00700 1 -- 172.21.15.7:0/958298756 wait complete. 2024-06-13T07:03:19.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.992+0000 7ff325e00700 1 Processor -- start 2024-06-13T07:03:19.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.992+0000 7ff325e00700 1 -- start start 2024-06-13T07:03:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.993+0000 7ff325e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.993+0000 7ff325e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff3201add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.994+0000 7ff325e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 0x7ff3201a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.994+0000 7ff31ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.994+0000 7ff31f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 0x7ff3201a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.994+0000 7ff31ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:44444/0 (socket says 172.21.15.7:44444) 2024-06-13T07:03:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.995+0000 7ff31ea00700 1 -- 172.21.15.7:0/3191228638 learned_addr learned my addr 172.21.15.7:0/3191228638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.995+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff320121680 con 0x7ff32011bc70 2024-06-13T07:03:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.995+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff320121500 con 0x7ff320110c70 2024-06-13T07:03:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.995+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff320121800 con 0x7ff3201099d0 2024-06-13T07:03:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31e000700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff3201add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31ea00700 1 -- 172.21.15.7:0/3191228638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 msgr2=0x7ff3201add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31ea00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff3201add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31ea00700 1 -- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 msgr2=0x7ff3201a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31ea00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 0x7ff3201a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.996+0000 7ff31ea00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3201a8140 con 0x7ff3201099d0 2024-06-13T07:03:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.997+0000 7ff31ea00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b070 tx=0x7ff308004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.997+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308021070 con 0x7ff3201099d0 2024-06-13T07:03:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.997+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3201a83d0 con 0x7ff3201099d0 2024-06-13T07:03:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.997+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3201b46a0 con 0x7ff3201099d0 2024-06-13T07:03:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.998+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308007c40 con 0x7ff3201099d0 2024-06-13T07:03:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:19.998+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080048a0 con 0x7ff3201099d0 2024-06-13T07:03:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.001+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff30801d470 con 0x7ff3201099d0 2024-06-13T07:03:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.003+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc005330 con 0x7ff3201099d0 2024-06-13T07:03:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.005+0000 7ff324a00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 0x7ff3040a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:20.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.007+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff30800f020 con 0x7ff3201099d0 2024-06-13T07:03:20.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.011+0000 7ff31e000700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 0x7ff3040a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:20.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.013+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff30802a020 con 0x7ff3201099d0 2024-06-13T07:03:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.020+0000 7ff31e000700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 0x7ff3040a5380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff32011e3b0 tx=0x7ff310009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.544+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2fc005620 con 0x7ff3201099d0 2024-06-13T07:03:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.546+0000 7ff324a00700 1 -- 172.21.15.7:0/3191228638 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11869 (secure 0 0 0) 0x7ff30809d3d0 con 0x7ff3201099d0 2024-06-13T07:03:20.546 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:03:20.546 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:02:25.094486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T07:03:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 msgr2=0x7ff3040a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 0x7ff3040a5380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff32011e3b0 tx=0x7ff310009260 comp rx=0 tx=0).stop 2024-06-13T07:03:20.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 msgr2=0x7ff3201ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b070 tx=0x7ff308004cf0 comp rx=0 tx=0).stop 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 shutdown_connections 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32011bc70 0x7ff3201a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3201099d0 0x7ff3201ad7d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff3040a2ec0 0x7ff3040a5380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 --2- 172.21.15.7:0/3191228638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff320110c70 0x7ff3201add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.552+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 >> 172.21.15.7:0/3191228638 conn(0x7ff32006d600 msgr2=0x7ff320119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.553+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 shutdown_connections 2024-06-13T07:03:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:20.553+0000 7ff325e00700 1 -- 172.21.15.7:0/3191228638 wait complete. 2024-06-13T07:03:20.555 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-06-13T07:03:21.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:21 smithi007 ceph-mon[32077]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 MiB/s wr, 631 op/s 2024-06-13T07:03:21.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:21 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3191228638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:03:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:21 smithi189 ceph-mon[36616]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 MiB/s wr, 631 op/s 2024-06-13T07:03:21.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:21 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3191228638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:03:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:21 smithi175 ceph-mon[36886]: pgmap v728: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 MiB/s wr, 631 op/s 2024-06-13T07:03:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:21 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3191228638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:03:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 440.0 s, 541.6 tps, lat 195.244 ms stddev 258.768 2024-06-13T07:03:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:23 smithi189 ceph-mon[36616]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 635 op/s 2024-06-13T07:03:23.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:23 smithi007 ceph-mon[32077]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 635 op/s 2024-06-13T07:03:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:23 smithi175 ceph-mon[36886]: pgmap v729: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 635 op/s 2024-06-13T07:03:23.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:03:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 -- 172.21.15.7:0/3780170274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 msgr2=0x7fa6b0108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3780170274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b0108dc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009900 tx=0x7fa6a002a410 comp rx=0 tx=0).stop 2024-06-13T07:03:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 -- 172.21.15.7:0/3780170274 shutdown_connections 2024-06-13T07:03:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3780170274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0110220 0x7fa6b0112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3780170274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0109300 0x7fa6b010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3780170274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b0108dc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.032+0000 7fa6b8200640 1 -- 172.21.15.7:0/3780170274 >> 172.21.15.7:0/3780170274 conn(0x7fa6b006ea10 msgr2=0x7fa6b0070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.034+0000 7fa6b8200640 1 -- 172.21.15.7:0/3780170274 shutdown_connections 2024-06-13T07:03:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.034+0000 7fa6b8200640 1 -- 172.21.15.7:0/3780170274 wait complete. 2024-06-13T07:03:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.036+0000 7fa6b8200640 1 Processor -- start 2024-06-13T07:03:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.037+0000 7fa6b8200640 1 -- start start 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 0x7fa6b01a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0110220 0x7fa6b01ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0115ee0 con 0x7fa6b0110220 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0115d60 con 0x7fa6b0109300 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b8200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0116060 con 0x7fa6b01089a0 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43804/0 (socket says 172.21.15.7:43804) 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.038+0000 7fa6b5800640 1 -- 172.21.15.7:0/3373785008 learned_addr learned my addr 172.21.15.7:0/3373785008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 0x7fa6b01a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b6200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0110220 0x7fa6b01ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b5800640 1 -- 172.21.15.7:0/3373785008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 msgr2=0x7fa6b01a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 0x7fa6b01a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b5800640 1 -- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0110220 msgr2=0x7fa6b01ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0110220 0x7fa6b01ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b5800640 1 -- 172.21.15.7:0/3373785008 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b01ae6b0 con 0x7fa6b01089a0 2024-06-13T07:03:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.039+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 0x7fa6b01a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.043+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006010 tx=0x7fa6a00052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.044+0000 7fa6ae000640 1 -- 172.21.15.7:0/3373785008 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0040030 con 0x7fa6b01089a0 2024-06-13T07:03:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.045+0000 7fa6ae000640 1 -- 172.21.15.7:0/3373785008 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0003e30 con 0x7fa6b01089a0 2024-06-13T07:03:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.045+0000 7fa6ae000640 1 -- 172.21.15.7:0/3373785008 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a003cb10 con 0x7fa6b01089a0 2024-06-13T07:03:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.045+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b01ae940 con 0x7fa6b01089a0 2024-06-13T07:03:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.045+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01aede0 con 0x7fa6b01089a0 2024-06-13T07:03:24.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.049+0000 7fa6ae000640 1 -- 172.21.15.7:0/3373785008 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa6a0043030 con 0x7fa6b01089a0 2024-06-13T07:03:24.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.052+0000 7fa6ae000640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 0x7fa6980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.054+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 0x7fa6980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.060+0000 7fa6bafb7740 1 Processor -- start 2024-06-13T07:03:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.061+0000 7fa6ae000640 1 -- 172.21.15.7:0/3373785008 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa6a00efc00 con 0x7fa6b01089a0 2024-06-13T07:03:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.061+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 0x7fa6980a5260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6b01a7fa0 tx=0x7fa6a4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.061+0000 7fa6bafb7740 1 -- start start 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d80b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d812bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b47d76e420 con 0x55b47d810760 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b47d715c50 con 0x55b47d80ac00 2024-06-13T07:03:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b47d76a3e0 con 0x55b47d80b5f0 2024-06-13T07:03:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43808/0 (socket says 172.21.15.7:43808) 2024-06-13T07:03:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.062+0000 7fa6b4e00640 1 -- 172.21.15.7:0/2629179488 learned_addr learned my addr 172.21.15.7:0/2629179488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.063+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d80b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.063+0000 7fa6b6200640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d812bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.063+0000 7fa6b5800640 1 -- 172.21.15.7:0/2629179488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 msgr2=0x55b47d8100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.063+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.063+0000 7fa6b5800640 1 -- 172.21.15.7:0/2629179488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 msgr2=0x55b47d812bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.064+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d812bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.064+0000 7fa6b5800640 1 -- 172.21.15.7:0/2629179488 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47d482210 con 0x55b47d80ac00 2024-06-13T07:03:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.064+0000 7fa6b6200640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d812bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.065+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d80b020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6a004b000 tx=0x7fa6a00056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80f8401050b64de server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.067+0000 7fa69ea00640 1 -- 172.21.15.7:0/2629179488 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a004a040 con 0x55b47d80ac00 2024-06-13T07:03:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.067+0000 7fa69ea00640 1 -- 172.21.15.7:0/2629179488 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a00bc120 con 0x55b47d80ac00 2024-06-13T07:03:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.067+0000 7fa69ea00640 1 -- 172.21.15.7:0/2629179488 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00b8790 con 0x55b47d80ac00 2024-06-13T07:03:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.068+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2629179488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 msgr2=0x55b47d80b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.068+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d80b020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa6a004b000 tx=0x7fa6a00056b0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2629179488 shutdown_connections 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d812bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2629179488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d80b020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2629179488 >> 172.21.15.7:0/2629179488 conn(0x55b47d7d2fb0 msgr2=0x55b47d806680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.069+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2629179488 shutdown_connections 2024-06-13T07:03:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.070+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2629179488 wait complete. 2024-06-13T07:03:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.070+0000 7fa6bafb7740 1 -- start start 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.071+0000 7fa6bafb7740 1 Processor -- start 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.072+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d825c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.072+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.072+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d82d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.072+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.072+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43812/0 (socket says 172.21.15.7:43812) 2024-06-13T07:03:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.074+0000 7fa6b4e00640 1 -- 172.21.15.7:0/2397431864 learned_addr learned my addr 172.21.15.7:0/2397431864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.074+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b47d715c50 con 0x55b47d810760 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.074+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b47d7290b0 con 0x55b47d80ac00 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.074+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b47d8134d0 con 0x55b47d80b5f0 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.074+0000 7fa6bafb7740 10 client.0 _ll_register_callbacks cb 0x55b47d714f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b6200640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d82d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b4e00640 1 -- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 msgr2=0x55b47d825c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d825c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b4e00640 1 -- 172.21.15.7:0/2397431864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 msgr2=0x55b47d82d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d82d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.075+0000 7fa6b4e00640 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47d54e620 con 0x55b47d80b5f0 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.076+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400bfd0 tx=0x7fa6a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e810ee29c05136 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.076+0000 7fa693400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.077+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401ce10 con 0x55b47d80b5f0 2024-06-13T07:03:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.077+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a00bc120 con 0x55b47d80b5f0 2024-06-13T07:03:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.077+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa6a401cc10 con 0x55b47d80b5f0 2024-06-13T07:03:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.077+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa6a401ce10 con 0x55b47d80b5f0 2024-06-13T07:03:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.078+0000 7fa6bafb7740 10 client.26422 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.078+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b47d8305a0 con 0x55b47d80b5f0 2024-06-13T07:03:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.078+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a00bc120 con 0x55b47d80b5f0 2024-06-13T07:03:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa6a401d430 con 0x55b47d80b5f0 2024-06-13T07:03:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 10 client.26422 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 10 client.26422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 20 client.26422 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d831d50 0x55b47d834210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 4 client.26422 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47d77bb50 con 0x55b47d831d50 2024-06-13T07:03:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.079+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d831d50 0x55b47d834210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.083+0000 7fa6b5800640 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d831d50 0x55b47d834210 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bf228b8fef0b55 server_cookie=6d4b040ffac514fa in_seq=0 out_seq=0 2024-06-13T07:03:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.083+0000 7fa693400640 10 client.26422 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.084+0000 7fa693400640 1 -- 172.21.15.7:0/2397431864 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b47d77bb50 con 0x55b47d831d50 2024-06-13T07:03:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.084+0000 7fa693400640 10 client.26422 handle_command_reply: tid=0 2024-06-13T07:03:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.084+0000 7fa6bafb7740 1 client.26422 shutdown 2024-06-13T07:03:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.085+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 msgr2=0x55b47d8261c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.085+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400bfd0 tx=0x7fa6a40046c0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 shutdown_connections 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d831d50 0x55b47d834210 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d82d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80b5f0 0x55b47d8261c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/2397431864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d825c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa693400640 0 client.26422 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.086+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 >> 172.21.15.7:0/2397431864 conn(0x55b47d7d2fb0 msgr2=0x55b47d805840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.087+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 shutdown_connections 2024-06-13T07:03:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.087+0000 7fa6bafb7740 1 -- 172.21.15.7:0/2397431864 wait complete. 2024-06-13T07:03:24.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.087+0000 7fa6bafb7740 20 client.26422 trim_cache size 0 max 16384 2024-06-13T07:03:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 Processor -- start 2024-06-13T07:03:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 -- start start 2024-06-13T07:03:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80b5f0 0x55b47d82d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d809650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01b3340 con 0x55b47d810760 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01b0be0 con 0x55b47d80b5f0 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.133+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01a37e0 con 0x55b47d80ac00 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43826/0 (socket says 172.21.15.7:43826) 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 -- 172.21.15.7:0/1132042650 learned_addr learned my addr 172.21.15.7:0/1132042650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80b5f0 0x55b47d82d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 -- 172.21.15.7:0/1132042650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80b5f0 msgr2=0x55b47d82d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80b5f0 0x55b47d82d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b5800640 1 -- 172.21.15.7:0/1132042650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 msgr2=0x55b47d809650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.134+0000 7fa6b6200640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d809650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.135+0000 7fa6b5800640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d809650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.135+0000 7fa6b5800640 1 -- 172.21.15.7:0/1132042650 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47d807330 con 0x55b47d80ac00 2024-06-13T07:03:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.135+0000 7fa6b6200640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d809650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.135+0000 7fa6b5800640 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00bbe20 tx=0x7fa6a002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7af68451b265e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.136+0000 7fa69e000640 1 -- 172.21.15.7:0/1132042650 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a004a040 con 0x55b47d80ac00 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.136+0000 7fa69e000640 1 -- 172.21.15.7:0/1132042650 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a00c04d0 con 0x55b47d80ac00 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.136+0000 7fa69e000640 1 -- 172.21.15.7:0/1132042650 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00b8790 con 0x55b47d80ac00 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 -- 172.21.15.7:0/1132042650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 msgr2=0x55b47d806870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00bbe20 tx=0x7fa6a002a410 comp rx=0 tx=0).stop 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 -- 172.21.15.7:0/1132042650 shutdown_connections 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d810760 0x55b47d809650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80b5f0 0x55b47d82d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/1132042650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d80ac00 0x55b47d806870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.137+0000 7fa6bafb7740 1 -- 172.21.15.7:0/1132042650 >> 172.21.15.7:0/1132042650 conn(0x55b47d7d2fb0 msgr2=0x55b47d806160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.139+0000 7fa6bafb7740 1 -- 172.21.15.7:0/1132042650 shutdown_connections 2024-06-13T07:03:24.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.139+0000 7fa6bafb7740 1 -- 172.21.15.7:0/1132042650 wait complete. 2024-06-13T07:03:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.140+0000 7fa6bafb7740 1 -- start start 2024-06-13T07:03:24.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.140+0000 7fa6bafb7740 1 Processor -- start 2024-06-13T07:03:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.141+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.141+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 0x55b47d829c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.141+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6bafb7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 0x55b47d82a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49702/0 (socket says 172.21.15.7:49702) 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6bafb7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01af0a0 con 0x55b47d80b5f0 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6b5800640 1 -- 172.21.15.7:0/3823720460 learned_addr learned my addr 172.21.15.7:0/3823720460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6b6200640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 0x55b47d82a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 0x55b47d829c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01b0510 con 0x55b47d80ac00 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01af770 con 0x55b47d810760 2024-06-13T07:03:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.142+0000 7fa6bafb7740 10 client.0 _ll_register_callbacks cb 0x7fa6a400f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.143+0000 7fa6b6200640 1 -- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 msgr2=0x55b47d82d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.143+0000 7fa6b6200640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.143+0000 7fa6b6200640 1 -- 172.21.15.7:0/3823720460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 msgr2=0x55b47d829c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:24.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.143+0000 7fa6b6200640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 0x55b47d829c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa6b6200640 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47d8189c0 con 0x55b47d810760 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa6b4e00640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 0x55b47d829c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa6b6200640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 0x55b47d82a250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005fd0 tx=0x7fa6a800e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6dea6812b92dca5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa693400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.144+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800ee50 con 0x55b47d810760 2024-06-13T07:03:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.145+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa6a8009be0 con 0x55b47d810760 2024-06-13T07:03:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.145+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b47d775ea0 con 0x55b47d810760 2024-06-13T07:03:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.146+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa6a8009de0 con 0x55b47d810760 2024-06-13T07:03:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.146+0000 7fa6bafb7740 10 client.26428 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.146+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b47d819d10 con 0x55b47d810760 2024-06-13T07:03:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.146+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b47d775ea0 con 0x55b47d810760 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.147+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa6a801b680 con 0x55b47d810760 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 10 client.26428 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 10 client.26428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 20 client.26428 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d81a640 0x55b47d81ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 4 client.26428 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b47d6f37a0 con 0x55b47d81a640 2024-06-13T07:03:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.148+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d81a640 0x55b47d81ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.151+0000 7fa6b5800640 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d81a640 0x55b47d81ca60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8e3d51888028bb server_cookie=3721799190e35468 in_seq=0 out_seq=0 2024-06-13T07:03:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.151+0000 7fa693400640 10 client.26428 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.209+0000 7fa693400640 1 -- 172.21.15.7:0/3823720460 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b47d6f37a0 con 0x55b47d81a640 2024-06-13T07:03:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.209+0000 7fa693400640 10 client.26428 handle_command_reply: tid=0 2024-06-13T07:03:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.209+0000 7fa6bafb7740 1 client.26428 shutdown 2024-06-13T07:03:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 msgr2=0x55b47d82a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 0x55b47d82a250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005fd0 tx=0x7fa6a800e780 comp rx=0 tx=0).stop 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 shutdown_connections 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b47d81a640 0x55b47d81ca60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b47d810760 0x55b47d82a250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b47d80b5f0 0x55b47d829c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 --2- 172.21.15.7:0/3823720460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b47d80ac00 0x55b47d82d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 >> 172.21.15.7:0/3823720460 conn(0x55b47d7d2fb0 msgr2=0x55b47d809650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.210+0000 7fa693400640 0 client.26428 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.213+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 shutdown_connections 2024-06-13T07:03:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.213+0000 7fa6bafb7740 1 -- 172.21.15.7:0/3823720460 wait complete. 2024-06-13T07:03:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.213+0000 7fa6bafb7740 20 client.26428 trim_cache size 0 max 16384 2024-06-13T07:03:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:03:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:03:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 msgr2=0x7fa6980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 0x7fa6980a5260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa6b01a7fa0 tx=0x7fa6a4005d30 comp rx=0 tx=0).stop 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 msgr2=0x7fa6b01a6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006010 tx=0x7fa6a00052b0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 shutdown_connections 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa6980a2da0 0x7fa6980a5260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b0110220 0x7fa6b01ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6b0109300 0x7fa6b01a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 --2- 172.21.15.7:0/3373785008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b01089a0 0x7fa6b01a6a90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.214+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 >> 172.21.15.7:0/3373785008 conn(0x7fa6b006ea10 msgr2=0x7fa6b0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.215+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 shutdown_connections 2024-06-13T07:03:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:24.215+0000 7fa6b8200640 1 -- 172.21.15.7:0/3373785008 wait complete. 2024-06-13T07:03:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:25 smithi189 ceph-mon[36616]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 758 op/s 2024-06-13T07:03:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:25 smithi175 ceph-mon[36886]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 758 op/s 2024-06-13T07:03:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:25 smithi007 ceph-mon[32077]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 758 op/s 2024-06-13T07:03:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:27 smithi189 ceph-mon[36616]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-06-13T07:03:27.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:27 smithi007 ceph-mon[32077]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-06-13T07:03:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:27 smithi175 ceph-mon[36886]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-06-13T07:03:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 445.0 s, 581.6 tps, lat 169.634 ms stddev 184.162 2024-06-13T07:03:28.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.525+0000 7fd80c200700 1 -- 172.21.15.7:0/2733795425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd80406a6e0 msgr2=0x7fd80406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.525+0000 7fd80c200700 1 --2- 172.21.15.7:0/2733795425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd80406a6e0 0x7fd80406ab40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009990 tx=0x7fd7f4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 -- 172.21.15.7:0/2733795425 shutdown_connections 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 --2- 172.21.15.7:0/2733795425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd804104ed0 0x7fd804105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 --2- 172.21.15.7:0/2733795425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd80406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 --2- 172.21.15.7:0/2733795425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd80406a6e0 0x7fd80406ab40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 -- 172.21.15.7:0/2733795425 >> 172.21.15.7:0/2733795425 conn(0x7fd804078340 msgr2=0x7fd804078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 -- 172.21.15.7:0/2733795425 shutdown_connections 2024-06-13T07:03:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.526+0000 7fd80c200700 1 -- 172.21.15.7:0/2733795425 wait complete. 2024-06-13T07:03:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.528+0000 7fd80c200700 1 Processor -- start 2024-06-13T07:03:28.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.528+0000 7fd80c200700 1 -- start start 2024-06-13T07:03:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80406a6e0 0x7fd80419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd8041a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd804118db0 con 0x7fd80406a6e0 2024-06-13T07:03:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd804118c30 con 0x7fd8041073c0 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd80c200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd804118f30 con 0x7fd804104ed0 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd808e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd808e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43854/0 (socket says 172.21.15.7:43854) 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.529+0000 7fd808e00700 1 -- 172.21.15.7:0/2213354772 learned_addr learned my addr 172.21.15.7:0/2213354772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd809800700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80406a6e0 0x7fd80419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd8041a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 -- 172.21.15.7:0/2213354772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 msgr2=0x7fd80419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 -- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80406a6e0 msgr2=0x7fd80419f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80406a6e0 0x7fd80419f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.530+0000 7fd80a200700 1 -- 172.21.15.7:0/2213354772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8041a4200 con 0x7fd8041073c0 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.531+0000 7fd808e00700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.532+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd8041a3ac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00cc50 tx=0x7fd7fc00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.532+0000 7fd802000700 1 -- 172.21.15.7:0/2213354772 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc016020 con 0x7fd8041073c0 2024-06-13T07:03:28.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.532+0000 7fd802000700 1 -- 172.21.15.7:0/2213354772 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7fc012420 con 0x7fd8041073c0 2024-06-13T07:03:28.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.532+0000 7fd802000700 1 -- 172.21.15.7:0/2213354772 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0115c0 con 0x7fd8041073c0 2024-06-13T07:03:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.534+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8041a44f0 con 0x7fd8041073c0 2024-06-13T07:03:28.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.534+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8041abd70 con 0x7fd8041073c0 2024-06-13T07:03:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.540+0000 7fd802000700 1 -- 172.21.15.7:0/2213354772 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd7fc019020 con 0x7fd8041073c0 2024-06-13T07:03:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.541+0000 7fd816b13b80 1 Processor -- start 2024-06-13T07:03:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 -- start start 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d53a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a187c62a10 con 0x55a187d5e300 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a187ca5b00 con 0x55a187ccfe40 2024-06-13T07:03:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.544+0000 7fd816b13b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a187c608f0 con 0x55a187d535b0 2024-06-13T07:03:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.545+0000 7fd808e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.545+0000 7fd808e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49730/0 (socket says 172.21.15.7:49730) 2024-06-13T07:03:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.545+0000 7fd808e00700 1 -- 172.21.15.7:0/2011593375 learned_addr learned my addr 172.21.15.7:0/2011593375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.545+0000 7fd802000700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 0x7fd7ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd809800700 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d53a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd802000700 1 -- 172.21.15.7:0/2213354772 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd7fc0cf190 con 0x7fd8041073c0 2024-06-13T07:03:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 -- 172.21.15.7:0/2011593375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 msgr2=0x55a187d53a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d53a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 -- 172.21.15.7:0/2011593375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 msgr2=0x55a187d606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 -- 172.21.15.7:0/2011593375 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a187c4df90 con 0x55a187ccfe40 2024-06-13T07:03:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.546+0000 7fd808e00700 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400b7f0 tx=0x7fd7f401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c87db2de46f9cde server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.547+0000 7fd7f2a00700 1 -- 172.21.15.7:0/2011593375 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400b900 con 0x55a187ccfe40 2024-06-13T07:03:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.547+0000 7fd7f2a00700 1 -- 172.21.15.7:0/2011593375 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f4005040 con 0x55a187ccfe40 2024-06-13T07:03:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.547+0000 7fd7f2a00700 1 -- 172.21.15.7:0/2011593375 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004b20 con 0x55a187ccfe40 2024-06-13T07:03:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.548+0000 7fd816b13b80 1 -- 172.21.15.7:0/2011593375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 msgr2=0x55a187d53070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.548+0000 7fd816b13b80 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400b7f0 tx=0x7fd7f401a040 comp rx=0 tx=0).stop 2024-06-13T07:03:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 0x7fd7ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd816b13b80 1 -- 172.21.15.7:0/2011593375 shutdown_connections 2024-06-13T07:03:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd816b13b80 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd816b13b80 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187ccfe40 0x55a187d53070 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400b7f0 tx=0x7fd7f401a040 comp rx=0 tx=0).stop 2024-06-13T07:03:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd816b13b80 1 --2- 172.21.15.7:0/2011593375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d53a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.549+0000 7fd816b13b80 1 -- 172.21.15.7:0/2011593375 >> 172.21.15.7:0/2011593375 conn(0x55a187c80170 msgr2=0x55a187d4f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.550+0000 7fd80a200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 0x7fd7ec0a5160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc09c3a0 tx=0x7fd7fc087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.551+0000 7fd816b13b80 1 -- 172.21.15.7:0/2011593375 shutdown_connections 2024-06-13T07:03:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.551+0000 7fd816b13b80 1 -- 172.21.15.7:0/2011593375 wait complete. 2024-06-13T07:03:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.553+0000 7fd816b13b80 1 -- start start 2024-06-13T07:03:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.553+0000 7fd816b13b80 1 Processor -- start 2024-06-13T07:03:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.556+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5b4e0 0x55a187d5b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a187c41f60 con 0x55a187d5e300 2024-06-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a187ca1ac0 con 0x55a187d535b0 2024-06-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a187d60f60 con 0x55a187d5b4e0 2024-06-13T07:03:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd808e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd808e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49740/0 (socket says 172.21.15.7:49740) 2024-06-13T07:03:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd808e00700 1 -- 172.21.15.7:0/3669270805 learned_addr learned my addr 172.21.15.7:0/3669270805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd816b13b80 10 client.0 _ll_register_callbacks cb 0x55a187c3f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.557+0000 7fd809800700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.559+0000 7fd808e00700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5b4e0 0x55a187d5b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 -- 172.21.15.7:0/3669270805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5b4e0 msgr2=0x55a187d5b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5b4e0 0x55a187d5b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 -- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 msgr2=0x55a187d5afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d5afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 -- 172.21.15.7:0/3669270805 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a187bf48d0 con 0x55a187d535b0 2024-06-13T07:03:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd808e00700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000c00 tx=0x7fd7f4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b112b5adc28e67 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.560+0000 7fd7e7400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.561+0000 7fd809800700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.561+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030040 con 0x55a187d535b0 2024-06-13T07:03:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.561+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd7f400b900 con 0x55a187d535b0 2024-06-13T07:03:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.562+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f4005040 con 0x55a187d535b0 2024-06-13T07:03:28.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.562+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd7f403c050 con 0x55a187d535b0 2024-06-13T07:03:28.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.562+0000 7fd816b13b80 10 client.26489 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:28.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.562+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a187d6bd20 con 0x55a187d535b0 2024-06-13T07:03:28.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.563+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f4005040 con 0x55a187d535b0 2024-06-13T07:03:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd7f4024070 con 0x55a187d535b0 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 10 client.26489 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 10 client.26489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 20 client.26489 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187d6e1f0 0x55a187d705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 4 client.26489 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.567+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a187cb3000 con 0x55a187d6e1f0 2024-06-13T07:03:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.568+0000 7fd809800700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187d6e1f0 0x55a187d705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.571+0000 7fd7e7400700 10 client.26489 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:03:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.571+0000 7fd809800700 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187d6e1f0 0x55a187d705e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4aae07e2a4824ee server_cookie=d8f6d15304796a08 in_seq=0 out_seq=0 2024-06-13T07:03:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.572+0000 7fd7e7400700 1 -- 172.21.15.7:0/3669270805 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a187cb3000 con 0x55a187d6e1f0 2024-06-13T07:03:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.573+0000 7fd7e7400700 10 client.26489 handle_command_reply: tid=0 2024-06-13T07:03:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.573+0000 7fd816b13b80 1 client.26489 shutdown 2024-06-13T07:03:28.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.573+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 msgr2=0x55a187d5dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.573+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4000c00 tx=0x7fd7f4009650 comp rx=0 tx=0).stop 2024-06-13T07:03:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.573+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 shutdown_connections 2024-06-13T07:03:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5b4e0 0x55a187d5b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187d6e1f0 0x55a187d705e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5dca0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 --2- 172.21.15.7:0/3669270805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5e300 0x55a187d5afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd7e7400700 0 client.26489 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 >> 172.21.15.7:0/3669270805 conn(0x55a187c80170 msgr2=0x55a187d4e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 shutdown_connections 2024-06-13T07:03:28.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 1 -- 172.21.15.7:0/3669270805 wait complete. 2024-06-13T07:03:28.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.574+0000 7fd816b13b80 20 client.26489 trim_cache size 0 max 16384 2024-06-13T07:03:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.611+0000 7fd816b13b80 1 Processor -- start 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 -- start start 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 0x55a187d4efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b6fa0 con 0x55a187d5b4e0 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b4800 con 0x55a187d535b0 2024-06-13T07:03:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.615+0000 7fd816b13b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b2650 con 0x55a187d5e300 2024-06-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.616+0000 7fd809800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.616+0000 7fd809800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56468/0 (socket says 172.21.15.7:56468) 2024-06-13T07:03:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.616+0000 7fd809800700 1 -- 172.21.15.7:0/1933535974 learned_addr learned my addr 172.21.15.7:0/1933535974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 0x55a187d4efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 -- 172.21.15.7:0/1933535974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 msgr2=0x55a187d5f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 -- 172.21.15.7:0/1933535974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 msgr2=0x55a187d5fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.617+0000 7fd808e00700 1 -- 172.21.15.7:0/1933535974 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a187d6a940 con 0x55a187d5e300 2024-06-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.619+0000 7fd809800700 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.619+0000 7fd808e00700 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 0x55a187d4efb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ce0 tx=0x7fd7f4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855888dfd9a4436d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.619+0000 7fd7f2000700 1 -- 172.21.15.7:0/1933535974 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f400b310 con 0x55a187d5e300 2024-06-13T07:03:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.619+0000 7fd7f2000700 1 -- 172.21.15.7:0/1933535974 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f400bd60 con 0x55a187d5e300 2024-06-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.620+0000 7fd7f2000700 1 -- 172.21.15.7:0/1933535974 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004940 con 0x55a187d5e300 2024-06-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 -- 172.21.15.7:0/1933535974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 msgr2=0x55a187d4efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 0x55a187d4efb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ce0 tx=0x7fd7f4009550 comp rx=0 tx=0).stop 2024-06-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 -- 172.21.15.7:0/1933535974 shutdown_connections 2024-06-13T07:03:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d5e300 0x55a187d4efb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d535b0 0x55a187d5f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 --2- 172.21.15.7:0/1933535974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d5fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 -- 172.21.15.7:0/1933535974 >> 172.21.15.7:0/1933535974 conn(0x55a187c80170 msgr2=0x55a187d56250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.621+0000 7fd816b13b80 1 -- 172.21.15.7:0/1933535974 shutdown_connections 2024-06-13T07:03:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.622+0000 7fd816b13b80 1 -- 172.21.15.7:0/1933535974 wait complete. 2024-06-13T07:03:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.623+0000 7fd816b13b80 1 -- start start 2024-06-13T07:03:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.624+0000 7fd816b13b80 1 Processor -- start 2024-06-13T07:03:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.625+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d5e300 0x55a187d57270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b2cc0 con 0x55a187d5b4e0 2024-06-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b4130 con 0x55a187d5e300 2024-06-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8041b3390 con 0x55a187d535b0 2024-06-13T07:03:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd816b13b80 10 client.0 _ll_register_callbacks cb 0x55a187c3f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.626+0000 7fd808e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.627+0000 7fd808e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:43892/0 (socket says 172.21.15.7:43892) 2024-06-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.627+0000 7fd808e00700 1 -- 172.21.15.7:0/163907670 learned_addr learned my addr 172.21.15.7:0/163907670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.628+0000 7fd809800700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d5e300 0x55a187d57270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 -- 172.21.15.7:0/163907670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d5e300 msgr2=0x55a187d57270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d5e300 0x55a187d57270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 -- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 msgr2=0x55a187d73b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d73b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.629+0000 7fd808e00700 1 -- 172.21.15.7:0/163907670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a187d74220 con 0x55a187d535b0 2024-06-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.630+0000 7fd7e7400700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.631+0000 7fd808e00700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005410 tx=0x7fd7f400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3840c159e5a9af04 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.631+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004740 con 0x55a187d535b0 2024-06-13T07:03:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.631+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f400bd60 con 0x55a187d535b0 2024-06-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.631+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd7f4021040 con 0x55a187d535b0 2024-06-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.632+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd7f4004740 con 0x55a187d535b0 2024-06-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.632+0000 7fd816b13b80 10 client.26446 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.632+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a187dd8110 con 0x55a187d535b0 2024-06-13T07:03:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f400bd60 con 0x55a187d535b0 2024-06-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd7f4024070 con 0x55a187d535b0 2024-06-13T07:03:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 10 client.26446 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 10 client.26446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 20 client.26446 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187dd8510 0x55a187dda900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 4 client.26446 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a187cad580 con 0x55a187dd8510 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.635+0000 7fd809800700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187dd8510 0x55a187dda900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.636+0000 7fd809800700 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187dd8510 0x55a187dda900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b49277c325a2b12 server_cookie=7b7067563364808 in_seq=0 out_seq=0 2024-06-13T07:03:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.636+0000 7fd7e7400700 10 client.26446 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd7e7400700 1 -- 172.21.15.7:0/163907670 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a187cad580 con 0x55a187dd8510 2024-06-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd7e7400700 10 client.26446 handle_command_reply: tid=0 2024-06-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 client.26446 shutdown 2024-06-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 msgr2=0x55a187d73610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005410 tx=0x7fd7f400bf10 comp rx=0 tx=0).stop 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 shutdown_connections 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55a187dd8510 0x55a187dda900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a187d5e300 0x55a187d57270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a187d535b0 0x55a187d73610 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 --2- 172.21.15.7:0/163907670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a187d5b4e0 0x55a187d73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 >> 172.21.15.7:0/163907670 conn(0x55a187c80170 msgr2=0x55a187d4efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd7e7400700 0 client.26446 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.638+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 shutdown_connections 2024-06-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.639+0000 7fd816b13b80 1 -- 172.21.15.7:0/163907670 wait complete. 2024-06-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.639+0000 7fd816b13b80 20 client.26446 trim_cache size 0 max 16384 2024-06-13T07:03:28.639 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.640+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 msgr2=0x7fd7ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.640+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 0x7fd7ec0a5160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc09c3a0 tx=0x7fd7fc087040 comp rx=0 tx=0).stop 2024-06-13T07:03:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.640+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 msgr2=0x7fd8041a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.640+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd8041a3ac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00cc50 tx=0x7fd7fc00cf60 comp rx=0 tx=0).stop 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 shutdown_connections 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd7ec0a2ca0 0x7fd7ec0a5160 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8041073c0 0x7fd8041a3ac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80406a6e0 0x7fd80419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 --2- 172.21.15.7:0/2213354772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd804104ed0 0x7fd80419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 >> 172.21.15.7:0/2213354772 conn(0x7fd804078340 msgr2=0x7fd804103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 shutdown_connections 2024-06-13T07:03:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:28.641+0000 7fd80c200700 1 -- 172.21.15.7:0/2213354772 wait complete. 2024-06-13T07:03:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:29 smithi189 ceph-mon[36616]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 666 op/s 2024-06-13T07:03:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:29 smithi175 ceph-mon[36886]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 666 op/s 2024-06-13T07:03:29.869 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:29 smithi007 ceph-mon[32077]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 666 op/s 2024-06-13T07:03:30.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:03:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:31.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:31 smithi007 ceph-mon[32077]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-06-13T07:03:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:31 smithi189 ceph-mon[36616]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-06-13T07:03:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:31 smithi175 ceph-mon[36886]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-06-13T07:03:32.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 450.0 s, 555.2 tps, lat 174.706 ms stddev 220.741 2024-06-13T07:03:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:33 smithi189 ceph-mon[36616]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-06-13T07:03:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:33 smithi175 ceph-mon[36886]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-06-13T07:03:33.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:33 smithi007 ceph-mon[32077]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2024-06-13T07:03:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 -- 172.21.15.7:0/3173193060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 msgr2=0x7f90b810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 --2- 172.21.15.7:0/3173193060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b810e610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009990 tx=0x7f90a4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 -- 172.21.15.7:0/3173193060 shutdown_connections 2024-06-13T07:03:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 --2- 172.21.15.7:0/3173193060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 0x7f90b8111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 --2- 172.21.15.7:0/3173193060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 --2- 172.21.15.7:0/3173193060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b810e610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 -- 172.21.15.7:0/3173193060 >> 172.21.15.7:0/3173193060 conn(0x7f90b8077590 msgr2=0x7f90b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.104+0000 7f90be600700 1 -- 172.21.15.7:0/3173193060 shutdown_connections 2024-06-13T07:03:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.105+0000 7f90be600700 1 -- 172.21.15.7:0/3173193060 wait complete. 2024-06-13T07:03:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.106+0000 7f90be600700 1 Processor -- start 2024-06-13T07:03:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 -- start start 2024-06-13T07:03:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 0x7f90b819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81149f0 con 0x7f90b8101190 2024-06-13T07:03:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8114870 con 0x7f90b8100870 2024-06-13T07:03:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90be600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8114b70 con 0x7f90b810ec80 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90b7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90b7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53240/0 (socket says 172.21.15.7:53240) 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.107+0000 7f90b7400700 1 -- 172.21.15.7:0/856112073 learned_addr learned my addr 172.21.15.7:0/856112073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7e00700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 0x7f90b819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b6a00700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 -- 172.21.15.7:0/856112073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 msgr2=0x7f90b819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 0x7f90b819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 -- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 msgr2=0x7f90b819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 -- 172.21.15.7:0/856112073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b819feb0 con 0x7f90b8100870 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.108+0000 7f90b7400700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90a800eb30 tx=0x7f90a800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.109+0000 7f90afe00700 1 -- 172.21.15.7:0/856112073 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800cc90 con 0x7f90b8100870 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.109+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81a01a0 con 0x7f90b8100870 2024-06-13T07:03:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.109+0000 7f90afe00700 1 -- 172.21.15.7:0/856112073 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a8004500 con 0x7f90b8100870 2024-06-13T07:03:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.109+0000 7f90afe00700 1 -- 172.21.15.7:0/856112073 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8010430 con 0x7f90b8100870 2024-06-13T07:03:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.109+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81a7a20 con 0x7f90b8100870 2024-06-13T07:03:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.112+0000 7f90afe00700 1 -- 172.21.15.7:0/856112073 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f90a8003f70 con 0x7f90b8100870 2024-06-13T07:03:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.113+0000 7f90afe00700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 0x7f909c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.114+0000 7f90afe00700 1 -- 172.21.15.7:0/856112073 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a80cf6b0 con 0x7f90b8100870 2024-06-13T07:03:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.114+0000 7f90b6a00700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 0x7f909c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.115+0000 7f90b6a00700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 0x7f909c0a5160 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90b8111090 tx=0x7f90a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.115+0000 7f90c9053b80 1 Processor -- start 2024-06-13T07:03:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.116+0000 7f90c9053b80 1 -- start start 2024-06-13T07:03:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595486d2e40 0x55954875e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x55954876bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559548665a10 con 0x5595486d2e40 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595486a8b00 con 0x55954875edc0 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.117+0000 7f90c9053b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595486638f0 con 0x559548769b10 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53254/0 (socket says 172.21.15.7:53254) 2024-06-13T07:03:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 -- 172.21.15.7:0/1203355476 learned_addr learned my addr 172.21.15.7:0/1203355476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 -- 172.21.15.7:0/1203355476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 msgr2=0x55954876bf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x55954876bf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 -- 172.21.15.7:0/1203355476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595486d2e40 msgr2=0x55954875e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595486d2e40 0x55954875e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.118+0000 7f90b6a00700 1 -- 172.21.15.7:0/1203355476 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559548650f90 con 0x55954875edc0 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.119+0000 7f90b6a00700 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f90a4015000 tx=0x7f90a401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55dabb7bbf2c0482 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.119+0000 7f90acc00700 1 -- 172.21.15.7:0/1203355476 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f7f0 con 0x55954875edc0 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.119+0000 7f90acc00700 1 -- 172.21.15.7:0/1203355476 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4023410 con 0x55954875edc0 2024-06-13T07:03:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.119+0000 7f90acc00700 1 -- 172.21.15.7:0/1203355476 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a402d610 con 0x55954875edc0 2024-06-13T07:03:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.120+0000 7f90c9053b80 1 -- 172.21.15.7:0/1203355476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 msgr2=0x55954875f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.120+0000 7f90c9053b80 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f90a4015000 tx=0x7f90a401e000 comp rx=0 tx=0).stop 2024-06-13T07:03:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 -- 172.21.15.7:0/1203355476 shutdown_connections 2024-06-13T07:03:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x55954876bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595486d2e40 0x55954875e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 --2- 172.21.15.7:0/1203355476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x55954875f240 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 -- 172.21.15.7:0/1203355476 >> 172.21.15.7:0/1203355476 conn(0x559548683170 msgr2=0x5595487524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.121+0000 7f90c9053b80 1 -- 172.21.15.7:0/1203355476 shutdown_connections 2024-06-13T07:03:35.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.122+0000 7f90c9053b80 1 -- 172.21.15.7:0/1203355476 wait complete. 2024-06-13T07:03:35.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.122+0000 7f90c9053b80 1 -- start start 2024-06-13T07:03:35.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.122+0000 7f90c9053b80 1 Processor -- start 2024-06-13T07:03:35.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x5595487664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x5595487669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559548644f60 con 0x559548769b10 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5595486a4ac0 con 0x55954875edc0 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55954876c770 con 0x5595486d2e40 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90c9053b80 10 client.0 _ll_register_callbacks cb 0x559548642720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90b7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x5595487664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90b7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41872/0 (socket says 172.21.15.7:41872) 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 -- 172.21.15.7:0/4013432884 learned_addr learned my addr 172.21.15.7:0/4013432884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.123+0000 7f90b7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x5595487669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 -- 172.21.15.7:0/4013432884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 msgr2=0x5595487664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x5595487664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 -- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 msgr2=0x5595487669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x5595487669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7400700 1 -- 172.21.15.7:0/4013432884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595485f78d0 con 0x5595486d2e40 2024-06-13T07:03:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b6a00700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x5595487664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.124+0000 7f90b7e00700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x5595487669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90b7400700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f90a809bb00 tx=0x7f90a809b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78122b72817dea01 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90a1600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a809bb70 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a8099e70 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a4023410 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90a809ff20 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90c9053b80 10 client.26455 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.125+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559548776e90 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4023410 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f90a80a4350 con 0x5595486d2e40 2024-06-13T07:03:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 10 client.26455 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 10 client.26455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:03:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 20 client.26455 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5595487794d0 0x55954877b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 4 client.26455 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.126+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595486b6000 con 0x5595487794d0 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.127+0000 7f90b6a00700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5595487794d0 0x55954877b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.128+0000 7f90b6a00700 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5595487794d0 0x55954877b930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a59c0aff38be61c6 server_cookie=a4cc571433a31e03 in_seq=0 out_seq=0 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.128+0000 7f90a1600700 10 client.26455 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90a1600700 1 -- 172.21.15.7:0/4013432884 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595486b6000 con 0x5595487794d0 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90a1600700 10 client.26455 handle_command_reply: tid=0 2024-06-13T07:03:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 client.26455 shutdown 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 msgr2=0x5595487691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f90a809bb00 tx=0x7f90a809b7a0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 shutdown_connections 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x5595487669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5595487794d0 0x55954877b930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595486d2e40 0x5595487691b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 --2- 172.21.15.7:0/4013432884 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55954875edc0 0x5595487664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 >> 172.21.15.7:0/4013432884 conn(0x559548683170 msgr2=0x559548750eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.129+0000 7f90a1600700 0 client.26455 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:03:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.130+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 shutdown_connections 2024-06-13T07:03:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.130+0000 7f90c9053b80 1 -- 172.21.15.7:0/4013432884 wait complete. 2024-06-13T07:03:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.130+0000 7f90c9053b80 20 client.26455 trim_cache size 0 max 16384 2024-06-13T07:03:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.150+0000 7f90c9053b80 1 Processor -- start 2024-06-13T07:03:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.150+0000 7f90c9053b80 1 -- start start 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x5595487515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 0x55954876af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x559548768cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ae8e0 con 0x559548769b10 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81ac140 con 0x5595486d2e40 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90c9053b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aa0e0 con 0x55954875edc0 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x559548768cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b7400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x5595487515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x559548768cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45844/0 (socket says 172.21.15.7:45844) 2024-06-13T07:03:35.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 0x55954876af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b7e00700 1 -- 172.21.15.7:0/2911338331 learned_addr learned my addr 172.21.15.7:0/2911338331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 -- 172.21.15.7:0/2911338331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 msgr2=0x5595487515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x5595487515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 -- 172.21.15.7:0/2911338331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 msgr2=0x559548768cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x559548768cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.151+0000 7f90b6a00700 1 -- 172.21.15.7:0/2911338331 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559548753950 con 0x55954875edc0 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.152+0000 7f90b6a00700 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 0x55954876af00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009990 tx=0x7f90a4019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682b02219e877547 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.152+0000 7f90a3e00700 1 -- 172.21.15.7:0/2911338331 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40239d0 con 0x55954875edc0 2024-06-13T07:03:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.152+0000 7f90a3e00700 1 -- 172.21.15.7:0/2911338331 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a400fcd0 con 0x55954875edc0 2024-06-13T07:03:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.152+0000 7f90a3e00700 1 -- 172.21.15.7:0/2911338331 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4037bb0 con 0x55954875edc0 2024-06-13T07:03:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 -- 172.21.15.7:0/2911338331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 msgr2=0x55954876af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 0x55954876af00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009990 tx=0x7f90a4019040 comp rx=0 tx=0).stop 2024-06-13T07:03:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 -- 172.21.15.7:0/2911338331 shutdown_connections 2024-06-13T07:03:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559548769b10 0x559548768cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x5595487515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 --2- 172.21.15.7:0/2911338331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55954875edc0 0x55954876af00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 -- 172.21.15.7:0/2911338331 >> 172.21.15.7:0/2911338331 conn(0x559548683170 msgr2=0x559548750eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 -- 172.21.15.7:0/2911338331 shutdown_connections 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.153+0000 7f90c9053b80 1 -- 172.21.15.7:0/2911338331 wait complete. 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.154+0000 7f90c9053b80 1 -- start start 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.154+0000 7f90c9053b80 1 Processor -- start 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x559548766750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x559548765040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aa660 con 0x55954875edc0 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aba70 con 0x5595486d2e40 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aacd0 con 0x559548769b10 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.155+0000 7f90c9053b80 10 client.0 _ll_register_callbacks cb 0x559548642720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45848/0 (socket says 172.21.15.7:45848) 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 -- 172.21.15.7:0/76301785 learned_addr learned my addr 172.21.15.7:0/76301785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 -- 172.21.15.7:0/76301785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 msgr2=0x559548765040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x559548765040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 -- 172.21.15.7:0/76301785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 msgr2=0x559548766750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:03:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x559548766750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.156+0000 7f90b6a00700 1 -- 172.21.15.7:0/76301785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595487656c0 con 0x55954875edc0 2024-06-13T07:03:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.157+0000 7f90b6a00700 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009ce0 tx=0x7f90a400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551350a579db5e34 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.157+0000 7f90a1600700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:03:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.157+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f770 con 0x55954875edc0 2024-06-13T07:03:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.158+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f90a400f910 con 0x55954875edc0 2024-06-13T07:03:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.158+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a400fcd0 con 0x55954875edc0 2024-06-13T07:03:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.158+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90a400fb10 con 0x55954875edc0 2024-06-13T07:03:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.163+0000 7f90c9053b80 10 client.15975 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.163+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559548771910 con 0x55954875edc0 2024-06-13T07:03:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a400fcd0 con 0x55954875edc0 2024-06-13T07:03:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f90a402d590 con 0x55954875edc0 2024-06-13T07:03:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 10 client.15975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 20 client.15975 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x559548771ba0 0x559548771f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 4 client.15975 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.164+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595486b0580 con 0x559548771ba0 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.165+0000 7f90b7400700 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x559548771ba0 0x559548771f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.166+0000 7f90a1600700 10 client.15975 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:03:35.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.168+0000 7f90b7400700 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x559548771ba0 0x559548771f80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b23bfcf8b7cd4efe server_cookie=7a67af55f0b376a9 in_seq=0 out_seq=0 2024-06-13T07:03:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.171+0000 7f90a1600700 1 -- 172.21.15.7:0/76301785 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595486b0580 con 0x559548771ba0 2024-06-13T07:03:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.171+0000 7f90a1600700 10 client.15975 handle_command_reply: tid=0 2024-06-13T07:03:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.171+0000 7f90c9053b80 1 client.15975 shutdown 2024-06-13T07:03:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.171+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 msgr2=0x559548764b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.171+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009ce0 tx=0x7f90a400ff30 comp rx=0 tx=0).stop 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 shutdown_connections 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x559548771ba0 0x559548771f80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559548769b10 0x559548765040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5595486d2e40 0x559548766750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90a1600700 0 client.15975 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 --2- 172.21.15.7:0/76301785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55954875edc0 0x559548764b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.172+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 >> 172.21.15.7:0/76301785 conn(0x559548683170 msgr2=0x5595487607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.173+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 shutdown_connections 2024-06-13T07:03:35.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.173+0000 7f90c9053b80 1 -- 172.21.15.7:0/76301785 wait complete. 2024-06-13T07:03:35.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.173+0000 7f90c9053b80 20 client.15975 trim_cache size 0 max 16384 2024-06-13T07:03:35.174 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 msgr2=0x7f909c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 0x7f909c0a5160 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90b8111090 tx=0x7f90a401a040 comp rx=0 tx=0).stop 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 msgr2=0x7f90b819aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90a800eb30 tx=0x7f90a800ee40 comp rx=0 tx=0).stop 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 shutdown_connections 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90b810ec80 0x7f90b819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f909c0a2ca0 0x7f909c0a5160 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90b8100870 0x7f90b819aea0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 --2- 172.21.15.7:0/856112073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b8101190 0x7f90b819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 >> 172.21.15.7:0/856112073 conn(0x7f90b8077590 msgr2=0x7f90b810feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 shutdown_connections 2024-06-13T07:03:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:35.177+0000 7f90be600700 1 -- 172.21.15.7:0/856112073 wait complete. 2024-06-13T07:03:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:35 smithi189 ceph-mon[36616]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-06-13T07:03:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:35 smithi175 ceph-mon[36886]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-06-13T07:03:35.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:35 smithi007 ceph-mon[32077]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 634 op/s 2024-06-13T07:03:36.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:03:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:37 smithi189 ceph-mon[36616]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 473 op/s 2024-06-13T07:03:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:37 smithi175 ceph-mon[36886]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 473 op/s 2024-06-13T07:03:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 455.0 s, 533.0 tps, lat 194.338 ms stddev 223.545 2024-06-13T07:03:37.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:37 smithi007 ceph-mon[32077]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 473 op/s 2024-06-13T07:03:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.135+0000 7fd1b9600640 1 -- 172.21.15.7:0/499709609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 msgr2=0x7fd1ac0a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.135+0000 7fd1b9600640 1 --2- 172.21.15.7:0/499709609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac0a4f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009860 tx=0x7fd1a802a340 comp rx=0 tx=0).stop 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.141+0000 7fd1b9600640 1 -- 172.21.15.7:0/499709609 shutdown_connections 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.141+0000 7fd1b9600640 1 --2- 172.21.15.7:0/499709609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0aca00 0x7fd1ac0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.141+0000 7fd1b9600640 1 --2- 172.21.15.7:0/499709609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac0ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.141+0000 7fd1b9600640 1 --2- 172.21.15.7:0/499709609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac0a4f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.141+0000 7fd1b9600640 1 -- 172.21.15.7:0/499709609 >> 172.21.15.7:0/499709609 conn(0x7fd1ac0a00d0 msgr2=0x7fd1ac0a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.143+0000 7fd1b9600640 1 -- 172.21.15.7:0/499709609 shutdown_connections 2024-06-13T07:03:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.144+0000 7fd1b9600640 1 -- 172.21.15.7:0/499709609 wait complete. 2024-06-13T07:03:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.148+0000 7fd1b9600640 1 Processor -- start 2024-06-13T07:03:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.148+0000 7fd1b9600640 1 -- start start 2024-06-13T07:03:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.149+0000 7fd1b9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.149+0000 7fd1b9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.149+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.149+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53282/0 (socket says 172.21.15.7:53282) 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b3e00640 1 -- 172.21.15.7:0/3084868703 learned_addr learned my addr 172.21.15.7:0/3084868703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0aca00 0x7fd1ac1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac0b2a70 con 0x7fd1ac0a5470 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac0b28f0 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac0b2bf0 con 0x7fd1ac0aca00 2024-06-13T07:03:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.150+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0aca00 0x7fd1ac1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.151+0000 7fd1b3e00640 1 -- 172.21.15.7:0/3084868703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0aca00 msgr2=0x7fd1ac1468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.151+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0aca00 0x7fd1ac1468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.151+0000 7fd1b3e00640 1 -- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 msgr2=0x7fd1ac13f8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.152+0000 7fd1b3400640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.152+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac13f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.152+0000 7fd1b3e00640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac146f30 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.152+0000 7fd1b3400640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac13f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.153+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8035040 tx=0x7fd1a8033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.154+0000 7fd1b0c00640 1 -- 172.21.15.7:0/3084868703 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8037040 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.154+0000 7fd1b0c00640 1 -- 172.21.15.7:0/3084868703 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8030850 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.154+0000 7fd1b0c00640 1 -- 172.21.15.7:0/3084868703 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8004920 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.158+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac1471c0 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.160+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac1476b0 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.167+0000 7fd1b0c00640 1 -- 172.21.15.7:0/3084868703 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd1a8004050 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.169+0000 7fd1b0c00640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 0x7fd19c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.170+0000 7fd1b0c00640 1 -- 172.21.15.7:0/3084868703 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd1a80eec80 con 0x7fd1ac0a4b10 2024-06-13T07:03:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.171+0000 7fd1b3400640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 0x7fd19c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.175+0000 7fd1b3400640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 0x7fd19c0a51a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac140820 tx=0x7fd1a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.326+0000 7fd1bda70740 1 Processor -- start 2024-06-13T07:03:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.326+0000 7fd1bda70740 1 -- start start 2024-06-13T07:03:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.326+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.326+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84c9eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84ca15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1bda70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc84bf4420 con 0x55dc84c9f150 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1bda70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc84b9bc50 con 0x55dc84c995f0 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1bda70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc84bf03e0 con 0x55dc84c99fe0 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53286/0 (socket says 172.21.15.7:53286) 2024-06-13T07:03:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 -- 172.21.15.7:0/1332676806 learned_addr learned my addr 172.21.15.7:0/1332676806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 -- 172.21.15.7:0/1332676806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 msgr2=0x55dc84c9eae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84c9eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.327+0000 7fd1b3e00640 1 -- 172.21.15.7:0/1332676806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 msgr2=0x55dc84ca15e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.328+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84ca15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.328+0000 7fd1b3e00640 1 -- 172.21.15.7:0/1332676806 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc84908210 con 0x55dc84c995f0 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.328+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80c38e0 tx=0x7fd1a8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564432a155a0a645 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.329+0000 7fd1a5200640 1 -- 172.21.15.7:0/1332676806 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80433a0 con 0x55dc84c995f0 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.329+0000 7fd1a5200640 1 -- 172.21.15.7:0/1332676806 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a80bb8a0 con 0x55dc84c995f0 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.329+0000 7fd1a5200640 1 -- 172.21.15.7:0/1332676806 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8004340 con 0x55dc84c995f0 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 -- 172.21.15.7:0/1332676806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 msgr2=0x55dc84c99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80c38e0 tx=0x7fd1a8005f90 comp rx=0 tx=0).stop 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 -- 172.21.15.7:0/1332676806 shutdown_connections 2024-06-13T07:03:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84ca15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84c9eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 --2- 172.21.15.7:0/1332676806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c99a10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 -- 172.21.15.7:0/1332676806 >> 172.21.15.7:0/1332676806 conn(0x55dc84c58fb0 msgr2=0x55dc84c8c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 -- 172.21.15.7:0/1332676806 shutdown_connections 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.330+0000 7fd1bda70740 1 -- 172.21.15.7:0/1332676806 wait complete. 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.331+0000 7fd1bda70740 1 -- start start 2024-06-13T07:03:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.331+0000 7fd1bda70740 1 Processor -- start 2024-06-13T07:03:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84ca35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 0x55dc84caab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dc84b9bc50 con 0x55dc84c99fe0 2024-06-13T07:03:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dc84baf0b0 con 0x55dc84c995f0 2024-06-13T07:03:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc84ca1ec0 con 0x55dc84c9f150 2024-06-13T07:03:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1bda70740 10 client.0 _ll_register_callbacks cb 0x55dc84b9af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.332+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84ca35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.333+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84ca35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53300/0 (socket says 172.21.15.7:53300) 2024-06-13T07:03:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.333+0000 7fd1b3e00640 1 -- 172.21.15.7:0/3867625107 learned_addr learned my addr 172.21.15.7:0/3867625107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.333+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 0x55dc84caab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 -- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 msgr2=0x55dc84ca35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84ca35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 -- 172.21.15.7:0/3867625107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 msgr2=0x55dc84ca3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 -- 172.21.15.7:0/3867625107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc849d4620 con 0x55dc84c9f150 2024-06-13T07:03:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 0x55dc84caab70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4051060 tx=0x7fd1b40735a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5dcd0711087c7f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd199600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.334+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b407c020 con 0x55dc84c9f150 2024-06-13T07:03:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.335+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd1b4073ba0 con 0x55dc84c9f150 2024-06-13T07:03:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.335+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a80bb8a0 con 0x55dc84c9f150 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.335+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1b4073de0 con 0x55dc84c9f150 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.335+0000 7fd1bda70740 10 client.26467 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.335+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dc84cadfb0 con 0x55dc84c9f150 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a80bb8a0 con 0x55dc84c9f150 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd1b4081600 con 0x55dc84c9f150 2024-06-13T07:03:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 10 client.26467 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 10 client.26467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 20 client.26467 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84caf760 0x55dc84cb1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 4 client.26467 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc84c01b50 con 0x55dc84caf760 2024-06-13T07:03:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.336+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84caf760 0x55dc84cb1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.338+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84caf760 0x55dc84cb1b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1f5470734666bb server_cookie=3a7032a94e5c20bd in_seq=0 out_seq=0 2024-06-13T07:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.338+0000 7fd199600640 10 client.26467 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.339+0000 7fd199600640 1 -- 172.21.15.7:0/3867625107 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc84c01b50 con 0x55dc84caf760 2024-06-13T07:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.339+0000 7fd199600640 10 client.26467 handle_command_reply: tid=0 2024-06-13T07:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.339+0000 7fd1bda70740 1 client.26467 shutdown 2024-06-13T07:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.340+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 msgr2=0x55dc84caab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.340+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 0x55dc84caab70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4051060 tx=0x7fd1b40735a0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.340+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 shutdown_connections 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.340+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84caf760 0x55dc84cb1b50 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.340+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c9f150 0x55dc84caab70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 1 --2- 172.21.15.7:0/3867625107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84ca35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 >> 172.21.15.7:0/3867625107 conn(0x55dc84c58fb0 msgr2=0x55dc84c8b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd199600640 0 client.26467 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 shutdown_connections 2024-06-13T07:03:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 1 -- 172.21.15.7:0/3867625107 wait complete. 2024-06-13T07:03:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.341+0000 7fd1bda70740 20 client.26467 trim_cache size 0 max 16384 2024-06-13T07:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.367+0000 7fd1bda70740 1 Processor -- start 2024-06-13T07:03:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 -- start start 2024-06-13T07:03:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84caab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84c8c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14ffd0 con 0x55dc84c9f150 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14d870 con 0x55dc84c995f0 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1bda70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14b700 con 0x55dc84c99fe0 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1b8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84c8c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53306/0 (socket says 172.21.15.7:53306) 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1b3e00640 1 -- 172.21.15.7:0/2315115334 learned_addr learned my addr 172.21.15.7:0/2315115334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.368+0000 7fd1b3400640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84caab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 -- 172.21.15.7:0/2315115334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 msgr2=0x55dc84caab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84caab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 -- 172.21.15.7:0/2315115334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 msgr2=0x55dc84c8c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84c8c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 -- 172.21.15.7:0/2315115334 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc84c9e5f0 con 0x55dc84c995f0 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3400640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84caab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84c8c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80eefa0 tx=0x7fd1a802a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7c1224a327a499 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd19be00640 1 -- 172.21.15.7:0/2315115334 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80b9580 con 0x55dc84c995f0 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd19be00640 1 -- 172.21.15.7:0/2315115334 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dc84c01b50 con 0x55dc84c995f0 2024-06-13T07:03:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.369+0000 7fd19be00640 1 -- 172.21.15.7:0/2315115334 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80bf170 con 0x55dc84c995f0 2024-06-13T07:03:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 -- 172.21.15.7:0/2315115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 msgr2=0x55dc84c8c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80eefa0 tx=0x7fd1a802a340 comp rx=0 tx=0).stop 2024-06-13T07:03:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 -- 172.21.15.7:0/2315115334 shutdown_connections 2024-06-13T07:03:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c9f150 0x55dc84c8c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c99fe0 0x55dc84caab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 --2- 172.21.15.7:0/2315115334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c995f0 0x55dc84c8c340 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 -- 172.21.15.7:0/2315115334 >> 172.21.15.7:0/2315115334 conn(0x55dc84c58fb0 msgr2=0x55dc84ca35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 -- 172.21.15.7:0/2315115334 shutdown_connections 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.370+0000 7fd1bda70740 1 -- 172.21.15.7:0/2315115334 wait complete. 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.371+0000 7fd1bda70740 1 -- start start 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.371+0000 7fd1bda70740 1 Processor -- start 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.371+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.372+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.372+0000 7fd1bda70740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c9f150 0x55dc84c92b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1bda70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14bd50 con 0x55dc84c99fe0 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1bda70740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14d1a0 con 0x55dc84c9f150 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1bda70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac14c400 con 0x55dc84c995f0 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1bda70740 10 client.0 _ll_register_callbacks cb 0x7fd1b4067180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1b3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41958/0 (socket says 172.21.15.7:41958) 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1b3e00640 1 -- 172.21.15.7:0/761328743 learned_addr learned my addr 172.21.15.7:0/761328743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1b3400640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.373+0000 7fd1b8c00640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c9f150 0x55dc84c92b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.374+0000 7fd1b3e00640 1 -- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c9f150 msgr2=0x55dc84c92b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.374+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c9f150 0x55dc84c92b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.374+0000 7fd1b3e00640 1 -- 172.21.15.7:0/761328743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 msgr2=0x55dc84ca5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.374+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.374+0000 7fd1b3e00640 1 -- 172.21.15.7:0/761328743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc84c93050 con 0x55dc84c995f0 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.375+0000 7fd199600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.375+0000 7fd1b3e00640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd1a802a930 tx=0x7fd1a8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b946655e943b740d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.375+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80d9040 con 0x55dc84c995f0 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.375+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc84c01b50 con 0x55dc84c995f0 2024-06-13T07:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.375+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd1a80b9240 con 0x55dc84c995f0 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1a80d9040 con 0x55dc84c995f0 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 10 client.26473 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dc84c94360 con 0x55dc84c995f0 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a809d080 con 0x55dc84c995f0 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd1a803a070 con 0x55dc84c995f0 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 10 client.26473 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 10 client.26473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 20 client.26473 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.376+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84c94820 0x55dc84c96c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.377+0000 7fd1bda70740 4 client.26473 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.377+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc84bfbea0 con 0x55dc84c94820 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.377+0000 7fd1b3400640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84c94820 0x55dc84c96c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.377+0000 7fd1b3400640 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84c94820 0x55dc84c96c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b59a60a8805824 server_cookie=fcf669b337553e00 in_seq=0 out_seq=0 2024-06-13T07:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.377+0000 7fd199600640 10 client.26473 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd199600640 1 -- 172.21.15.7:0/761328743 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc84bfbea0 con 0x55dc84c94820 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd199600640 10 client.26473 handle_command_reply: tid=0 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 client.26473 shutdown 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 msgr2=0x55dc84ca58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd1a802a930 tx=0x7fd1a8004600 comp rx=0 tx=0).stop 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 shutdown_connections 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dc84c94820 0x55dc84c96c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dc84c9f150 0x55dc84c92b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dc84c99fe0 0x55dc84ca5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 --2- 172.21.15.7:0/761328743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc84c995f0 0x55dc84ca58a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 >> 172.21.15.7:0/761328743 conn(0x55dc84c58fb0 msgr2=0x55dc84c8c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.391+0000 7fd199600640 0 client.26473 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.392+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 shutdown_connections 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.392+0000 7fd1bda70740 1 -- 172.21.15.7:0/761328743 wait complete. 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.392+0000 7fd1bda70740 20 client.26473 trim_cache size 0 max 16384 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:03:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 msgr2=0x7fd19c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 0x7fd19c0a51a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac140820 tx=0x7fd1a0009260 comp rx=0 tx=0).stop 2024-06-13T07:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 msgr2=0x7fd1ac13f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8035040 tx=0x7fd1a8033d90 comp rx=0 tx=0).stop 2024-06-13T07:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 shutdown_connections 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd19c0a2ce0 0x7fd19c0a51a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1ac0aca00 0x7fd1ac1468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1ac0a5470 0x7fd1ac13f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 --2- 172.21.15.7:0/3084868703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1ac0a4b10 0x7fd1ac13f360 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 >> 172.21.15.7:0/3084868703 conn(0x7fd1ac0a00d0 msgr2=0x7fd1ac0a24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 shutdown_connections 2024-06-13T07:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:38.393+0000 7fd1b9600640 1 -- 172.21.15.7:0/3084868703 wait complete. 2024-06-13T07:03:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:39 smithi189 ceph-mon[36616]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-06-13T07:03:39.848 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:39 smithi007 ceph-mon[32077]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-06-13T07:03:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:39 smithi175 ceph-mon[36886]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 -- 172.21.15.7:0/4193821355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc1046d0 msgr2=0x7f20cc104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 --2- 172.21.15.7:0/4193821355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc1046d0 0x7f20cc104b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00b0a0 tx=0x7f20bc00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 -- 172.21.15.7:0/4193821355 shutdown_connections 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 --2- 172.21.15.7:0/4193821355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc105090 0x7f20cc1115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 --2- 172.21.15.7:0/4193821355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 --2- 172.21.15.7:0/4193821355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc1046d0 0x7f20cc104b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.236+0000 7f20d0a00700 1 -- 172.21.15.7:0/4193821355 >> 172.21.15.7:0/4193821355 conn(0x7f20cc1001c0 msgr2=0x7f20cc1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.237+0000 7f20d0a00700 1 -- 172.21.15.7:0/4193821355 shutdown_connections 2024-06-13T07:03:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.237+0000 7f20d0a00700 1 -- 172.21.15.7:0/4193821355 wait complete. 2024-06-13T07:03:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.241+0000 7f20d0a00700 1 Processor -- start 2024-06-13T07:03:41.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.267+0000 7f20d0a00700 1 -- start start 2024-06-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.270+0000 7f20d0a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.270+0000 7f20d0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20cc105090 0x7f20cc10e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20d0a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20c9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20d0a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114b10 con 0x7f20cc105090 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20d0a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114990 con 0x7f20cc10a6d0 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20d0a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc114c90 con 0x7f20cc1046d0 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20c9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54522/0 (socket says 172.21.15.7:54522) 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20ca800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20ca800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60456/0 (socket says 172.21.15.7:60456) 2024-06-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.271+0000 7f20ca800700 1 -- 172.21.15.7:0/179316332 learned_addr learned my addr 172.21.15.7:0/179316332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.272+0000 7f20c9e00700 1 -- 172.21.15.7:0/179316332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 msgr2=0x7f20cc10e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.272+0000 7f20c9e00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.273+0000 7f20c9e00700 1 -- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20cc105090 msgr2=0x7f20cc10e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.273+0000 7f20c9e00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20cc105090 0x7f20cc10e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.273+0000 7f20c9e00700 1 -- 172.21.15.7:0/179316332 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc10f270 con 0x7f20cc1046d0 2024-06-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.273+0000 7f20ca800700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:03:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.274+0000 7f20c9e00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b450 tx=0x7f20c400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.274+0000 7f20c2a00700 1 -- 172.21.15.7:0/179316332 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4015020 con 0x7f20cc1046d0 2024-06-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.274+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc1a10f0 con 0x7f20cc1046d0 2024-06-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.275+0000 7f20c2a00700 1 -- 172.21.15.7:0/179316332 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4011d10 con 0x7f20cc1046d0 2024-06-13T07:03:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.275+0000 7f20c2a00700 1 -- 172.21.15.7:0/179316332 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4010e20 con 0x7f20cc1046d0 2024-06-13T07:03:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.277+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a1680 con 0x7f20cc1046d0 2024-06-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.283+0000 7f20c2a00700 1 -- 172.21.15.7:0/179316332 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f20c4018020 con 0x7f20cc1046d0 2024-06-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.285+0000 7f20c2a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 0x7f20b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.286+0000 7f20c2a00700 1 -- 172.21.15.7:0/179316332 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f20c40ce230 con 0x7f20cc1046d0 2024-06-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.286+0000 7f20c9400700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 0x7f20b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.287+0000 7f20c9400700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 0x7f20b00a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00b070 tx=0x7f20bc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.294+0000 7f20db3e2b80 1 Processor -- start 2024-06-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 -- start start 2024-06-13T07:03:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bacdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bace680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b9dca10 con 0x55ce0bad44b0 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ba1fb00 con 0x55ce0bace240 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.296+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b9da8f0 con 0x55ce0bad3b90 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54536/0 (socket says 172.21.15.7:54536) 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 -- 172.21.15.7:0/2532674368 learned_addr learned my addr 172.21.15.7:0/2532674368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 -- 172.21.15.7:0/2532674368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 msgr2=0x55ce0bace680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bace680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 -- 172.21.15.7:0/2532674368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 msgr2=0x55ce0bacdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bacdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.297+0000 7f20c9e00700 1 -- 172.21.15.7:0/2532674368 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0b9c7f90 con 0x55ce0bad3b90 2024-06-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.298+0000 7f20c9e00700 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f20c4096bb0 tx=0x7f20c4010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4587d7eca09f4a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.298+0000 7f20b7400700 1 -- 172.21.15.7:0/2532674368 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c409ae50 con 0x55ce0bad3b90 2024-06-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.298+0000 7f20b7400700 1 -- 172.21.15.7:0/2532674368 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c409a750 con 0x55ce0bad3b90 2024-06-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.298+0000 7f20b7400700 1 -- 172.21.15.7:0/2532674368 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40a2f30 con 0x55ce0bad3b90 2024-06-13T07:03:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.300+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2532674368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 msgr2=0x55ce0bad3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.300+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f20c4096bb0 tx=0x7f20c4010880 comp rx=0 tx=0).stop 2024-06-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2532674368 shutdown_connections 2024-06-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bace680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bad3f70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2532674368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bacdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2532674368 >> 172.21.15.7:0/2532674368 conn(0x55ce0b9fa170 msgr2=0x55ce0bac8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.302+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2532674368 shutdown_connections 2024-06-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.303+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2532674368 wait complete. 2024-06-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.303+0000 7f20db3e2b80 1 -- start start 2024-06-13T07:03:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.304+0000 7f20db3e2b80 1 Processor -- start 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bada310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad3b90 0x55ce0bad7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ce0b9bbf60 con 0x55ce0bad3b90 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ce0ba1bac0 con 0x55ce0bad44b0 2024-06-13T07:03:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ce0badae90 con 0x55ce0bace240 2024-06-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20db3e2b80 10 client.0 _ll_register_callbacks cb 0x55ce0b9b5380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.305+0000 7f20ca800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.306+0000 7f20ca800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60464/0 (socket says 172.21.15.7:60464) 2024-06-13T07:03:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.306+0000 7f20ca800700 1 -- 172.21.15.7:0/3919846718 learned_addr learned my addr 172.21.15.7:0/3919846718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.307+0000 7f20ca800700 1 -- 172.21.15.7:0/3919846718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 msgr2=0x55ce0bada310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:03:41.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.307+0000 7f20ca800700 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bada310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.307+0000 7f20ca800700 1 -- 172.21.15.7:0/3919846718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad3b90 msgr2=0x55ce0bad7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.307+0000 7f20ca800700 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad3b90 0x55ce0bad7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.307+0000 7f20ca800700 1 -- 172.21.15.7:0/3919846718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0b96e8d0 con 0x55ce0bad44b0 2024-06-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.308+0000 7f20ca800700 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e7c0 tx=0x7f20b800eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6972519ffd1dae server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.310+0000 7f20abe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:41.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.310+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8017070 con 0x55ce0bad44b0 2024-06-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.311+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f20b800f040 con 0x55ce0bad44b0 2024-06-13T07:03:41.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.311+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c409a750 con 0x55ce0bad44b0 2024-06-13T07:03:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.313+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20b8021050 con 0x55ce0bad44b0 2024-06-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.313+0000 7f20db3e2b80 10 client.26534 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.314+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ce0baf6740 con 0x55ce0bad44b0 2024-06-13T07:03:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.314+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b8013b60 con 0x55ce0bad44b0 2024-06-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.315+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f20b801cac0 con 0x55ce0bad44b0 2024-06-13T07:03:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.315+0000 7f20db3e2b80 10 client.26534 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.315+0000 7f20db3e2b80 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.315+0000 7f20db3e2b80 20 client.26534 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.316+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0baf8e20 0x55ce0bafb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.316+0000 7f20db3e2b80 4 client.26534 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.316+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f20c409a750 con 0x55ce0baf8e20 2024-06-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.316+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0baf8e20 0x55ce0bafb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.317+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0baf8e20 0x55ce0bafb210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8aac40063ac47f8 server_cookie=2e4ab6ff1d3e157e in_seq=0 out_seq=0 2024-06-13T07:03:41.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.317+0000 7f20abe00700 10 client.26534 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.319+0000 7f20abe00700 1 -- 172.21.15.7:0/3919846718 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f20c409a750 con 0x55ce0baf8e20 2024-06-13T07:03:41.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.319+0000 7f20abe00700 10 client.26534 handle_command_reply: tid=0 2024-06-13T07:03:41.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.319+0000 7f20db3e2b80 1 client.26534 shutdown 2024-06-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.320+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 msgr2=0x55ce0bad7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.320+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e7c0 tx=0x7f20b800eb80 comp rx=0 tx=0).stop 2024-06-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 shutdown_connections 2024-06-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0baf8e20 0x55ce0bafb210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad44b0 0x55ce0bad7b50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bada310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3919846718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad3b90 0x55ce0bad7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.321+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 >> 172.21.15.7:0/3919846718 conn(0x55ce0b9fa170 msgr2=0x55ce0bac2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.322+0000 7f20abe00700 0 client.26534 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.322+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 shutdown_connections 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.322+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3919846718 wait complete. 2024-06-13T07:03:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.322+0000 7f20db3e2b80 20 client.26534 trim_cache size 0 max 16384 2024-06-13T07:03:41.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 Processor -- start 2024-06-13T07:03:41.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 -- start start 2024-06-13T07:03:41.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bacdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bad6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1b7250 con 0x55ce0bad44b0 2024-06-13T07:03:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc073960 con 0x55ce0bad3b90 2024-06-13T07:03:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.357+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0717b0 con 0x55ce0bace240 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60468/0 (socket says 172.21.15.7:60468) 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 -- 172.21.15.7:0/2094770924 learned_addr learned my addr 172.21.15.7:0/2094770924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 -- 172.21.15.7:0/2094770924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 msgr2=0x55ce0bacdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bacdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 -- 172.21.15.7:0/2094770924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 msgr2=0x55ce0bad6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bad6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.358+0000 7f20c9400700 1 -- 172.21.15.7:0/2094770924 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0bad73e0 con 0x55ce0bad3b90 2024-06-13T07:03:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.359+0000 7f20c9400700 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f20cc10fca0 tx=0x7f20bc0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae08bb82c6930a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.359+0000 7f20b6a00700 1 -- 172.21.15.7:0/2094770924 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc009b80 con 0x55ce0bad3b90 2024-06-13T07:03:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.359+0000 7f20b6a00700 1 -- 172.21.15.7:0/2094770924 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20bc003d60 con 0x55ce0bad3b90 2024-06-13T07:03:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.360+0000 7f20b6a00700 1 -- 172.21.15.7:0/2094770924 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc009b80 con 0x55ce0bad3b90 2024-06-13T07:03:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2094770924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 msgr2=0x55ce0bac91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f20cc10fca0 tx=0x7f20bc0099e0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2094770924 shutdown_connections 2024-06-13T07:03:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0bad6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bace240 0x55ce0bacdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/2094770924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bad3b90 0x55ce0bac91a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.361+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2094770924 >> 172.21.15.7:0/2094770924 conn(0x55ce0b9fa170 msgr2=0x55ce0bac8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.362+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2094770924 shutdown_connections 2024-06-13T07:03:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.362+0000 7f20db3e2b80 1 -- 172.21.15.7:0/2094770924 wait complete. 2024-06-13T07:03:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.363+0000 7f20db3e2b80 1 -- start start 2024-06-13T07:03:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.363+0000 7f20db3e2b80 1 Processor -- start 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bae0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0baffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc071e20 con 0x55ce0bad44b0 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc073290 con 0x55ce0bace240 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0724f0 con 0x55ce0bad3b90 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.364+0000 7f20db3e2b80 10 client.0 _ll_register_callbacks cb 0x55ce0b9b5380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.365+0000 7f20c9400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.365+0000 7f20c9400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54550/0 (socket says 172.21.15.7:54550) 2024-06-13T07:03:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.365+0000 7f20c9400700 1 -- 172.21.15.7:0/3478338602 learned_addr learned my addr 172.21.15.7:0/3478338602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.366+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bae0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.366+0000 7f20c9400700 1 -- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 msgr2=0x55ce0bae0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.366+0000 7f20c9400700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bae0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20c9400700 1 -- 172.21.15.7:0/3478338602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 msgr2=0x55ce0baffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20c9400700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0baffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20c9400700 1 -- 172.21.15.7:0/3478338602 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0bae0ca0 con 0x55ce0bad3b90 2024-06-13T07:03:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20c9400700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007aa0 tx=0x7f20bc007f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6aef71a5708c5bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20abe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.367+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bae0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc01d720 con 0x55ce0bad3b90 2024-06-13T07:03:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc003d60 con 0x55ce0bad3b90 2024-06-13T07:03:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f20bc01d8c0 con 0x55ce0bad3b90 2024-06-13T07:03:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f20bc01dac0 con 0x55ce0bad3b90 2024-06-13T07:03:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20db3e2b80 10 client.26482 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.368+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ce0bb63020 con 0x55ce0bad3b90 2024-06-13T07:03:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.369+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20bc003d60 con 0x55ce0bad3b90 2024-06-13T07:03:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f20bc011070 con 0x55ce0bad3b90 2024-06-13T07:03:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20db3e2b80 10 client.26482 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20db3e2b80 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20db3e2b80 20 client.26482 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0bb63680 0x55ce0bb65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.370+0000 7f20db3e2b80 4 client.26482 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:03:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.371+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce0ba2d000 con 0x55ce0bb63680 2024-06-13T07:03:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.371+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0bb63680 0x55ce0bb65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.374+0000 7f20c9e00700 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0bb63680 0x55ce0bb65aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0260c5d038bd66 server_cookie=2443959391145fa2 in_seq=0 out_seq=0 2024-06-13T07:03:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.374+0000 7f20abe00700 10 client.26482 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.374+0000 7f20abe00700 1 -- 172.21.15.7:0/3478338602 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce0ba2d000 con 0x55ce0bb63680 2024-06-13T07:03:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.374+0000 7f20abe00700 10 client.26482 handle_command_reply: tid=0 2024-06-13T07:03:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.374+0000 7f20db3e2b80 1 client.26482 shutdown 2024-06-13T07:03:41.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.378+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 msgr2=0x55ce0bae0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.378+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007aa0 tx=0x7f20bc007f10 comp rx=0 tx=0).stop 2024-06-13T07:03:41.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 shutdown_connections 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ce0bb63680 0x55ce0bb65aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce0bad44b0 0x55ce0baffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ce0bace240 0x55ce0bae0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 --2- 172.21.15.7:0/3478338602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0bad3b90 0x55ce0bae0660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20abe00700 0 client.26482 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:41.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 >> 172.21.15.7:0/3478338602 conn(0x55ce0b9fa170 msgr2=0x55ce0bad4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.380+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 shutdown_connections 2024-06-13T07:03:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.381+0000 7f20db3e2b80 1 -- 172.21.15.7:0/3478338602 wait complete. 2024-06-13T07:03:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.381+0000 7f20db3e2b80 20 client.26482 trim_cache size 0 max 16384 2024-06-13T07:03:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.382+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 msgr2=0x7f20b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.382+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 0x7f20b00a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00b070 tx=0x7f20bc00b7b0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.382+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 msgr2=0x7f20cc1113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.382+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b450 tx=0x7f20c400b760 comp rx=0 tx=0).stop 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 shutdown_connections 2024-06-13T07:03:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f20b00a2c20 0x7f20b00a50e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20cc10a6d0 0x7f20cc10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20cc1046d0 0x7f20cc1113a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 --2- 172.21.15.7:0/179316332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20cc105090 0x7f20cc10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.383+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 >> 172.21.15.7:0/179316332 conn(0x7f20cc1001c0 msgr2=0x7f20cc100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.384+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 shutdown_connections 2024-06-13T07:03:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:41.384+0000 7f20d0a00700 1 -- 172.21.15.7:0/179316332 wait complete. 2024-06-13T07:03:41.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:41 smithi007 ceph-mon[32077]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-06-13T07:03:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:41 smithi189 ceph-mon[36616]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-06-13T07:03:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:41 smithi175 ceph-mon[36886]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 581 op/s 2024-06-13T07:03:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 460.0 s, 489.4 tps, lat 194.580 ms stddev 227.533 2024-06-13T07:03:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:43 smithi189 ceph-mon[36616]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-06-13T07:03:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:43 smithi175 ceph-mon[36886]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-06-13T07:03:43.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:43 smithi007 ceph-mon[32077]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-06-13T07:03:43.945 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:03:43.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:03:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:45.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:45 smithi007 ceph-mon[32077]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-06-13T07:03:45.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:45 smithi189 ceph-mon[36616]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-06-13T07:03:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:45 smithi175 ceph-mon[36886]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-06-13T07:03:47.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.1 MiB/s wr, 428 op/s 2024-06-13T07:03:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.1 MiB/s wr, 428 op/s 2024-06-13T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:47 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:03:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 465.0 s, 490.4 tps, lat 217.224 ms stddev 297.814 2024-06-13T07:03:47.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:03:47.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:47.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:03:47.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:47.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:47 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:03:48.053 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:03:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:03:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "b038ca8c-b2a4-4cf5-977e-a4c9051c3f82", 2024-06-13T07:03:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:03:48.053 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:03:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 -- 172.21.15.7:0/1118321014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec06af70 msgr2=0x7f05ec06b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 --2- 172.21.15.7:0/1118321014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec06af70 0x7f05ec06b3d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009990 tx=0x7f05d8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 -- 172.21.15.7:0/1118321014 shutdown_connections 2024-06-13T07:03:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 --2- 172.21.15.7:0/1118321014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 --2- 172.21.15.7:0/1118321014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec1073b0 0x7f05ec06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 --2- 172.21.15.7:0/1118321014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec06af70 0x7f05ec06b3d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.213+0000 7f05f3200700 1 -- 172.21.15.7:0/1118321014 >> 172.21.15.7:0/1118321014 conn(0x7f05ec078300 msgr2=0x7f05ec078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.214+0000 7f05f3200700 1 -- 172.21.15.7:0/1118321014 shutdown_connections 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.214+0000 7f05f3200700 1 -- 172.21.15.7:0/1118321014 wait complete. 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 Processor -- start 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 -- start start 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 0x7f05ec19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec1073b0 0x7f05ec1a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec118dc0 con 0x7f05ec1073b0 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec118c40 con 0x7f05ec06af70 2024-06-13T07:03:48.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05f3200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec118f40 con 0x7f05ec104ec0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.215+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 0x7f05ec19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 0x7f05ec19f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60488/0 (socket says 172.21.15.7:60488) 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05ebe00700 1 -- 172.21.15.7:0/3953696601 learned_addr learned my addr 172.21.15.7:0/3953696601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05f1200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec1073b0 0x7f05ec1a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 -- 172.21.15.7:0/3953696601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 msgr2=0x7f05ec19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 0x7f05ec19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 -- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec1073b0 msgr2=0x7f05ec1a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec1073b0 0x7f05ec1a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 -- 172.21.15.7:0/3953696601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1a4250 con 0x7f05ec104ec0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.216+0000 7f05eb400700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec19f7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000c00 tx=0x7f05d800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.217+0000 7f05e8c00700 1 -- 172.21.15.7:0/3953696601 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8026070 con 0x7f05ec104ec0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.217+0000 7f05e8c00700 1 -- 172.21.15.7:0/3953696601 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d8003d70 con 0x7f05ec104ec0 2024-06-13T07:03:48.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.217+0000 7f05e8c00700 1 -- 172.21.15.7:0/3953696601 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d800fe20 con 0x7f05ec104ec0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.217+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec1a44e0 con 0x7f05ec104ec0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.217+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1abdc0 con 0x7f05ec104ec0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.221+0000 7f05e8c00700 1 -- 172.21.15.7:0/3953696601 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f05d8029070 con 0x7f05ec104ec0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.222+0000 7f05e8c00700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 0x7f05d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.222+0000 7f05e8c00700 1 -- 172.21.15.7:0/3953696601 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05d80d0110 con 0x7f05ec104ec0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.222+0000 7f05ebe00700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 0x7f05d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.223+0000 7f05ebe00700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 0x7f05d00a50e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009ca0 tx=0x7f05dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.223+0000 7f05fdb98b80 1 Processor -- start 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 -- start start 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d20fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d21a380 0x559b5d21c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559b5d11ea90 con 0x559b5d21a380 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b5d161b80 con 0x559b5d18bec0 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.224+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b5d11c970 con 0x559b5d20f630 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05f1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d21a380 0x559b5d21c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60502/0 (socket says 172.21.15.7:60502) 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05ebe00700 1 -- 172.21.15.7:0/146216717 learned_addr learned my addr 172.21.15.7:0/146216717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d20fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 -- 172.21.15.7:0/146216717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 msgr2=0x559b5d20f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 -- 172.21.15.7:0/146216717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d21a380 msgr2=0x559b5d21c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d21a380 0x559b5d21c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 -- 172.21.15.7:0/146216717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5d10a010 con 0x559b5d20f630 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05ebe00700 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05eb400700 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d20fab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05d809c360 tx=0x7f05d8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c653b9902fe20d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05e1600700 1 -- 172.21.15.7:0/146216717 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d809bfc0 con 0x559b5d20f630 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.225+0000 7f05e1600700 1 -- 172.21.15.7:0/146216717 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d80043b0 con 0x559b5d20f630 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05e1600700 1 -- 172.21.15.7:0/146216717 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d809a960 con 0x559b5d20f630 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 -- 172.21.15.7:0/146216717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 msgr2=0x559b5d20fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d20fab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05d809c360 tx=0x7f05d8009310 comp rx=0 tx=0).stop 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 -- 172.21.15.7:0/146216717 shutdown_connections 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d21a380 0x559b5d21c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d20f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/146216717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d20fab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.226+0000 7f05fdb98b80 1 -- 172.21.15.7:0/146216717 >> 172.21.15.7:0/146216717 conn(0x559b5d13c1f0 msgr2=0x559b5d20b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.227+0000 7f05fdb98b80 1 -- 172.21.15.7:0/146216717 shutdown_connections 2024-06-13T07:03:48.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.227+0000 7f05fdb98b80 1 -- 172.21.15.7:0/146216717 wait complete. 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.227+0000 7f05fdb98b80 1 -- start start 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.227+0000 7f05fdb98b80 1 Processor -- start 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d219ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 0x559b5d216ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d2173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559b5d0fdfe0 con 0x559b5d20f630 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b5d15db40 con 0x559b5d18bec0 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b5d21cdf0 con 0x559b5d21a380 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05fdb98b80 10 client.0 _ll_register_callbacks cb 0x559b5d0fb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d219ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d219ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:60512/0 (socket says 172.21.15.7:60512) 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05ebe00700 1 -- 172.21.15.7:0/2202143678 learned_addr learned my addr 172.21.15.7:0/2202143678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05eb400700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 0x559b5d216ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.228+0000 7f05f1200700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d2173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 -- 172.21.15.7:0/2202143678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 msgr2=0x559b5d2173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d2173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 -- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 msgr2=0x559b5d219ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d219ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 -- 172.21.15.7:0/2202143678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5d0b0950 con 0x559b5d20f630 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05eb400700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 0x559b5d216ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f05d809c820 tx=0x7f05d800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de475dd245a74ad7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05d6000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d809cd60 con 0x559b5d20f630 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05d8087050 con 0x559b5d20f630 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.229+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d80043b0 con 0x559b5d20f630 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05d80a5240 con 0x559b5d20f630 2024-06-13T07:03:48.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 10 client.15993 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559b5d227540 con 0x559b5d20f630 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d80043b0 con 0x559b5d20f630 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f05d809bab0 con 0x559b5d20f630 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 20 client.15993 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d229a10 0x559b5d22be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 4 client.15993 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.230+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b5d16f080 con 0x559b5d229a10 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.231+0000 7f05ebe00700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d229a10 0x559b5d22be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.231+0000 7f05ebe00700 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d229a10 0x559b5d22be00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4668c9b1798170a1 server_cookie=1e57caa73214abb2 in_seq=0 out_seq=0 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.231+0000 7f05d6000700 10 client.15993 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05d6000700 1 -- 172.21.15.7:0/2202143678 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b5d16f080 con 0x559b5d229a10 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05d6000700 10 client.15993 handle_command_reply: tid=0 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 client.15993 shutdown 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 msgr2=0x559b5d216ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 0x559b5d216ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f05d809c820 tx=0x7f05d800f920 comp rx=0 tx=0).stop 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 shutdown_connections 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d229a10 0x559b5d22be00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d2173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d18bec0 0x559b5d219ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2202143678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d20f630 0x559b5d216ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 >> 172.21.15.7:0/2202143678 conn(0x559b5d13c1f0 msgr2=0x559b5d20a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05d6000700 0 client.15993 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.232+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 shutdown_connections 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.233+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2202143678 wait complete. 2024-06-13T07:03:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.233+0000 7f05fdb98b80 20 client.15993 trim_cache size 0 max 16384 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.276+0000 7f05fdb98b80 1 Processor -- start 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.276+0000 7f05fdb98b80 1 -- start start 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d20f630 0x559b5d21b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b6fd0 con 0x559b5d18bec0 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b4830 con 0x559b5d20f630 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.277+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b2680 con 0x559b5d21a380 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.278+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.278+0000 7f05f1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.278+0000 7f05f1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54598/0 (socket says 172.21.15.7:54598) 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.278+0000 7f05f1200700 1 -- 172.21.15.7:0/2293790722 learned_addr learned my addr 172.21.15.7:0/2293790722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:48.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.278+0000 7f05eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d20f630 0x559b5d21b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.279+0000 7f05f1200700 1 -- 172.21.15.7:0/2293790722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d20f630 msgr2=0x559b5d21b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.279+0000 7f05f1200700 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d20f630 0x559b5d21b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.279+0000 7f05f1200700 1 -- 172.21.15.7:0/2293790722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 msgr2=0x559b5d2196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.279+0000 7f05f1200700 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.279+0000 7f05f1200700 1 -- 172.21.15.7:0/2293790722 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5d202e90 con 0x559b5d21a380 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.280+0000 7f05f1200700 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f05e4007500 tx=0x7f05e400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b74bceb0c9a180 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.282+0000 7f05e0c00700 1 -- 172.21.15.7:0/2293790722 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e401b070 con 0x559b5d21a380 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.282+0000 7f05e0c00700 1 -- 172.21.15.7:0/2293790722 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e40178f0 con 0x559b5d21a380 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.282+0000 7f05e0c00700 1 -- 172.21.15.7:0/2293790722 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4016830 con 0x559b5d21a380 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.283+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2293790722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 msgr2=0x559b5d21bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.283+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f05e4007500 tx=0x7f05e400b040 comp rx=0 tx=0).stop 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.289+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2293790722 shutdown_connections 2024-06-13T07:03:48.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.289+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d21a380 0x559b5d21bcb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.289+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.289+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/2293790722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d20f630 0x559b5d21b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.289+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2293790722 >> 172.21.15.7:0/2293790722 conn(0x559b5d13c1f0 msgr2=0x559b5d20a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.290+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2293790722 shutdown_connections 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.290+0000 7f05fdb98b80 1 -- 172.21.15.7:0/2293790722 wait complete. 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.292+0000 7f05fdb98b80 1 -- start start 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.294+0000 7f05fdb98b80 1 Processor -- start 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.295+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d291f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d21a380 0x559b5d28e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b2cf0 con 0x559b5d18bec0 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b4160 con 0x559b5d21a380 2024-06-13T07:03:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b33c0 con 0x559b5d20f630 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05fdb98b80 10 client.0 _ll_register_callbacks cb 0x559b5d0fb7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.296+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d291f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50672/0 (socket says 172.21.15.7:50672) 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 -- 172.21.15.7:0/4001893173 learned_addr learned my addr 172.21.15.7:0/4001893173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05f1200700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d21a380 0x559b5d28e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 -- 172.21.15.7:0/4001893173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 msgr2=0x559b5d291f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d291f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 -- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d21a380 msgr2=0x559b5d28e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d21a380 0x559b5d28e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.298+0000 7f05ebe00700 1 -- 172.21.15.7:0/4001893173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5d28eeb0 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.300+0000 7f05ebe00700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05dc013010 tx=0x7f05dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733466861d9df326 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.300+0000 7f05d6000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.300+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc02a690 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.300+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f05dc02a830 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.301+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e40178f0 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.301+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05dc02aa30 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.302+0000 7f05fdb98b80 10 client.15999 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.302+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559b5d221cb0 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.302+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e40178f0 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f05dc027050 con 0x559b5d18bec0 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 20 client.15999 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d2223a0 0x559b5d2227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 4 client.15999 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559b5d169600 con 0x559b5d2223a0 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.303+0000 7f05eb400700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d2223a0 0x559b5d2227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.306+0000 7f05eb400700 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d2223a0 0x559b5d2227b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5b3b1e7fa17d32 server_cookie=a9990ade56cf7953 in_seq=0 out_seq=0 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.306+0000 7f05d6000700 10 client.15999 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.307+0000 7f05d6000700 1 -- 172.21.15.7:0/4001893173 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559b5d169600 con 0x559b5d2223a0 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.307+0000 7f05d6000700 10 client.15999 handle_command_reply: tid=0 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.307+0000 7f05fdb98b80 1 client.15999 shutdown 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.307+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 msgr2=0x559b5d2919d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.307+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05dc013010 tx=0x7f05dc0047c0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 shutdown_connections 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559b5d2223a0 0x559b5d2227b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b5d21a380 0x559b5d28e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559b5d18bec0 0x559b5d2919d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 --2- 172.21.15.7:0/4001893173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b5d20f630 0x559b5d291f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 >> 172.21.15.7:0/4001893173 conn(0x559b5d13c1f0 msgr2=0x559b5d211040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05d6000700 0 client.15999 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 shutdown_connections 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.308+0000 7f05fdb98b80 1 -- 172.21.15.7:0/4001893173 wait complete. 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.309+0000 7f05fdb98b80 20 client.15999 trim_cache size 0 max 16384 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 msgr2=0x7f05d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 0x7f05d00a50e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009ca0 tx=0x7f05dc009260 comp rx=0 tx=0).stop 2024-06-13T07:03:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 msgr2=0x7f05ec19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec19f7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f05d8000c00 tx=0x7f05d800bab0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 shutdown_connections 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec1073b0 0x7f05ec1a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f05d00a2c20 0x7f05d00a50e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05ec06af70 0x7f05ec19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 --2- 172.21.15.7:0/3953696601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05ec104ec0 0x7f05ec19f7d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 >> 172.21.15.7:0/3953696601 conn(0x7f05ec078300 msgr2=0x7f05ec103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 shutdown_connections 2024-06-13T07:03:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:47.310+0000 7f05f3200700 1 -- 172.21.15.7:0/3953696601 wait complete. 2024-06-13T07:03:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.1 MiB/s wr, 428 op/s 2024-06-13T07:03:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:03:48.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:03:48.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:47 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:03:48.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:48 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:03:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:03:48.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:03:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:48.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:48 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:49.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:03:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:03:49.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:03:49.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:48 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:03:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:49 smithi007 ceph-mon[32077]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2024-06-13T07:03:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:49 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:49.943 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:03:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:49 smithi189 ceph-mon[36616]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2024-06-13T07:03:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:49 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:49 smithi175 ceph-mon[36886]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2024-06-13T07:03:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:49 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:50.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:50.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:50.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:50 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:51.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:51.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:51.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:50 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:03:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:03:51.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:50 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:03:51.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:51 smithi007 ceph-mon[32077]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-06-13T07:03:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 -- 172.21.15.7:0/629694916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 msgr2=0x7f8a740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 --2- 172.21.15.7:0/629694916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a740a61d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009900 tx=0x7f8a6802a420 comp rx=0 tx=0).stop 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 -- 172.21.15.7:0/629694916 shutdown_connections 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 --2- 172.21.15.7:0/629694916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a6710 0x7f8a740aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 --2- 172.21.15.7:0/629694916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a740a61d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 --2- 172.21.15.7:0/629694916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a740a4c40 0x7f8a740a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 -- 172.21.15.7:0/629694916 >> 172.21.15.7:0/629694916 conn(0x7f8a740a00d0 msgr2=0x7f8a740a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 -- 172.21.15.7:0/629694916 shutdown_connections 2024-06-13T07:03:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.077+0000 7f8a81a00640 1 -- 172.21.15.7:0/629694916 wait complete. 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.078+0000 7f8a81a00640 1 Processor -- start 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 -- start start 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 0x7f8a74143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a74143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a740a6710 0x7f8a7414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740b9080 con 0x7f8a740a6710 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740b8f00 con 0x7f8a740a5d70 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.079+0000 7f8a81a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a740b9200 con 0x7f8a740a4c40 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a74143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 0x7f8a74143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 0x7f8a74143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34398/0 (socket says 172.21.15.7:34398) 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7be00640 1 -- 172.21.15.7:0/1239303887 learned_addr learned my addr 172.21.15.7:0/1239303887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 -- 172.21.15.7:0/1239303887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 msgr2=0x7f8a74143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 0x7f8a74143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 -- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a740a6710 msgr2=0x7f8a7414abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a740a6710 0x7f8a7414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.080+0000 7f8a7b400640 1 -- 172.21.15.7:0/1239303887 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7414b260 con 0x7f8a740a5d70 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.081+0000 7f8a7b400640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a74143b80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009230 tx=0x7f8a68005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.081+0000 7f8a78c00640 1 -- 172.21.15.7:0/1239303887 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68040030 con 0x7f8a740a5d70 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.081+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7414b4f0 con 0x7f8a740a5d70 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.081+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7414ba00 con 0x7f8a740a5d70 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.082+0000 7f8a78c00640 1 -- 172.21.15.7:0/1239303887 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a68005630 con 0x7f8a740a5d70 2024-06-13T07:03:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.082+0000 7f8a78c00640 1 -- 172.21.15.7:0/1239303887 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68031440 con 0x7f8a740a5d70 2024-06-13T07:03:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:51 smithi189 ceph-mon[36616]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-06-13T07:03:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.090+0000 7f8a85fce740 1 Processor -- start 2024-06-13T07:03:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 -- start start 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccbde0 0x559214ccc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559214c2f420 con 0x559214ccbde0 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559214bd6c50 con 0x559214cd1940 2024-06-13T07:03:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.091+0000 7f8a85fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559214c2b3e0 con 0x559214ccc7d0 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34410/0 (socket says 172.21.15.7:34410) 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 -- 172.21.15.7:0/1497794568 learned_addr learned my addr 172.21.15.7:0/1497794568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 -- 172.21.15.7:0/1497794568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 msgr2=0x559214cd3dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd3dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.092+0000 7f8a7be00640 1 -- 172.21.15.7:0/1497794568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccbde0 msgr2=0x559214ccc200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.093+0000 7f8a78c00640 1 -- 172.21.15.7:0/1239303887 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8a68043030 con 0x7f8a740a5d70 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.093+0000 7f8a7be00640 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccbde0 0x559214ccc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.093+0000 7f8a7be00640 1 -- 172.21.15.7:0/1497794568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559214943210 con 0x559214ccc7d0 2024-06-13T07:03:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.095+0000 7f8a7be00640 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009d70 tx=0x7f8a700117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc6aacd3ad756bb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.095+0000 7f8a6d600640 1 -- 172.21.15.7:0/1497794568 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70009e60 con 0x559214ccc7d0 2024-06-13T07:03:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.095+0000 7f8a6d600640 1 -- 172.21.15.7:0/1497794568 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a70023b40 con 0x559214ccc7d0 2024-06-13T07:03:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.095+0000 7f8a6d600640 1 -- 172.21.15.7:0/1497794568 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70009e60 con 0x559214ccc7d0 2024-06-13T07:03:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.096+0000 7f8a78c00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 0x7f8a640a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.097+0000 7f8a81000640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 0x7f8a640a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.097+0000 7f8a78c00640 1 -- 172.21.15.7:0/1239303887 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8a680efa20 con 0x7f8a740a5d70 2024-06-13T07:03:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.098+0000 7f8a85fce740 1 -- 172.21.15.7:0/1497794568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 msgr2=0x559214cd12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.098+0000 7f8a85fce740 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009d70 tx=0x7f8a700117b0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.099+0000 7f8a85fce740 1 -- 172.21.15.7:0/1497794568 shutdown_connections 2024-06-13T07:03:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.099+0000 7f8a85fce740 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.099+0000 7f8a85fce740 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccc7d0 0x559214cd12d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.099+0000 7f8a85fce740 1 --2- 172.21.15.7:0/1497794568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccbde0 0x559214ccc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.099+0000 7f8a85fce740 1 -- 172.21.15.7:0/1497794568 >> 172.21.15.7:0/1497794568 conn(0x559214c93fb0 msgr2=0x559214cc7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.100+0000 7f8a85fce740 1 -- 172.21.15.7:0/1497794568 shutdown_connections 2024-06-13T07:03:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.100+0000 7f8a85fce740 1 -- 172.21.15.7:0/1497794568 wait complete. 2024-06-13T07:03:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.100+0000 7f8a85fce740 1 -- start start 2024-06-13T07:03:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.101+0000 7f8a85fce740 1 Processor -- start 2024-06-13T07:03:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.102+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214cd61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.102+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cd66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.102+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a81000640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 0x7f8a640a5120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c067280 tx=0x7f8a7c0506f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a81000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a81000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40462/0 (socket says 172.21.15.7:40462) 2024-06-13T07:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a85fce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559214cd48f0 con 0x559214ccc7d0 2024-06-13T07:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a85fce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559214bea0b0 con 0x559214cd1940 2024-06-13T07:03:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.105+0000 7f8a85fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559214cd4a70 con 0x559214ccbde0 2024-06-13T07:03:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.106+0000 7f8a85fce740 10 client.0 _ll_register_callbacks cb 0x559214bd5f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.106+0000 7f8a81000640 1 -- 172.21.15.7:0/3343924304 learned_addr learned my addr 172.21.15.7:0/3343924304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.107+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214cd61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a7b400640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cd66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 -- 172.21.15.7:0/3343924304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 msgr2=0x559214cd61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214cd61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 -- 172.21.15.7:0/3343924304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 msgr2=0x559214cd66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cd66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 -- 172.21.15.7:0/3343924304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559214a0f620 con 0x559214cd1940 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.108+0000 7f8a81000640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c04fea0 tx=0x7f8a7c04fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d673acbcaed81219 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.109+0000 7f8a62000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.109+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c08a060 con 0x559214cd1940 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.109+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8a7c0787d0 con 0x559214cd1940 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.109+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a70023b40 con 0x559214cd1940 2024-06-13T07:03:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.110+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8a7c07ae50 con 0x559214cd1940 2024-06-13T07:03:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.110+0000 7f8a85fce740 10 client.26564 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.110+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559214ce08d0 con 0x559214cd1940 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.111+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7c07ae50 con 0x559214cd1940 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f8a7c085c10 con 0x559214cd1940 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 10 client.26564 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 20 client.26564 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214ce4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 4 client.26564 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559214c3cb50 con 0x559214ce1ff0 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.112+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214ce4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.113+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214ce4440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9daade4f83295ec0 server_cookie=9dbe378a6f6a5ae2 in_seq=0 out_seq=0 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.113+0000 7f8a62000640 10 client.26564 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.115+0000 7f8a62000640 1 -- 172.21.15.7:0/3343924304 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559214c3cb50 con 0x559214ce1ff0 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.115+0000 7f8a62000640 10 client.26564 handle_command_reply: tid=0 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.115+0000 7f8a85fce740 1 client.26564 shutdown 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.115+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 msgr2=0x559214cdd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.115+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c04fea0 tx=0x7f8a7c04fd50 comp rx=0 tx=0).stop 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 shutdown_connections 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214ce4440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cdd720 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a62000640 0 client.26564 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cd66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3343924304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214cd61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 >> 172.21.15.7:0/3343924304 conn(0x559214c93fb0 msgr2=0x559214cc69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.116+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 shutdown_connections 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.117+0000 7f8a85fce740 1 -- 172.21.15.7:0/3343924304 wait complete. 2024-06-13T07:03:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.117+0000 7f8a85fce740 20 client.26564 trim_cache size 0 max 16384 2024-06-13T07:03:52.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:51 smithi175 ceph-mon[36886]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 566 op/s 2024-06-13T07:03:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.155+0000 7f8a85fce740 1 Processor -- start 2024-06-13T07:03:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.157+0000 7f8a85fce740 1 -- start start 2024-06-13T07:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214ccaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cdd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c0123d0 con 0x559214ccc7d0 2024-06-13T07:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00fc70 con 0x559214cd1940 2024-06-13T07:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.158+0000 7f8a85fce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00da80 con 0x559214ccbde0 2024-06-13T07:03:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cdd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214ccaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cdd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41598/0 (socket says 172.21.15.7:41598) 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7b400640 1 -- 172.21.15.7:0/3930936570 learned_addr learned my addr 172.21.15.7:0/3930936570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 -- 172.21.15.7:0/3930936570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 msgr2=0x559214cd4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a81000640 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 -- 172.21.15.7:0/3930936570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 msgr2=0x559214cdd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cdd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a7be00640 1 -- 172.21.15.7:0/3930936570 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559214ccd5e0 con 0x559214ccbde0 2024-06-13T07:03:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.159+0000 7f8a81000640 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.160+0000 7f8a7be00640 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214ccaf40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x559214cc8710 tx=0x7f8a700058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4830f553ee324a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.160+0000 7f8a6cc00640 1 -- 172.21.15.7:0/3930936570 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000f630 con 0x559214ccbde0 2024-06-13T07:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.160+0000 7f8a6cc00640 1 -- 172.21.15.7:0/3930936570 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7003a070 con 0x559214ccbde0 2024-06-13T07:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.160+0000 7f8a6cc00640 1 -- 172.21.15.7:0/3930936570 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000f850 con 0x559214ccbde0 2024-06-13T07:03:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.161+0000 7f8a85fce740 1 -- 172.21.15.7:0/3930936570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 msgr2=0x559214ccaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214ccaf40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x559214cc8710 tx=0x7f8a700058f0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 -- 172.21.15.7:0/3930936570 shutdown_connections 2024-06-13T07:03:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214cd1940 0x559214cd4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cdd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 --2- 172.21.15.7:0/3930936570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214ccbde0 0x559214ccaf40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.162+0000 7f8a85fce740 1 -- 172.21.15.7:0/3930936570 >> 172.21.15.7:0/3930936570 conn(0x559214c93fb0 msgr2=0x559214cca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.163+0000 7f8a85fce740 1 -- 172.21.15.7:0/3930936570 shutdown_connections 2024-06-13T07:03:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.163+0000 7f8a85fce740 1 -- 172.21.15.7:0/3930936570 wait complete. 2024-06-13T07:03:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.164+0000 7f8a85fce740 1 -- start start 2024-06-13T07:03:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.164+0000 7f8a85fce740 1 Processor -- start 2024-06-13T07:03:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.165+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.165+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cda7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.165+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a85fce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214cd1940 0x559214d4ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40480/0 (socket says 172.21.15.7:40480) 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a85fce740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00e130 con 0x559214ccc7d0 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 -- 172.21.15.7:0/459242673 learned_addr learned my addr 172.21.15.7:0/459242673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00f5a0 con 0x559214ccbde0 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c00e800 con 0x559214cd1940 2024-06-13T07:03:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a85fce740 10 client.0 _ll_register_callbacks cb 0x7f8a7c06a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 -- 172.21.15.7:0/459242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214cd1940 msgr2=0x559214d4ad10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:03:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214cd1940 0x559214d4ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 -- 172.21.15.7:0/459242673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 msgr2=0x559214cda7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cda7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.166+0000 7f8a7be00640 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559214cdd8c0 con 0x559214ccbde0 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.167+0000 7f8a7be00640 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002d040 tx=0x7f8a700050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0680a0cbb80d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.167+0000 7f8a62000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.167+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7004d060 con 0x559214ccbde0 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.167+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8a70023460 con 0x559214ccbde0 2024-06-13T07:03:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.167+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a7003a070 con 0x559214ccbde0 2024-06-13T07:03:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.168+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8a70023660 con 0x559214ccbde0 2024-06-13T07:03:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.168+0000 7f8a85fce740 10 client.26567 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.168+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559214d4c170 con 0x559214ccbde0 2024-06-13T07:03:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.168+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7003a070 con 0x559214ccbde0 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f8a70003850 con 0x559214ccbde0 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 10 client.26567 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 20 client.26567 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214d4e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 4 client.26567 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.169+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559214c36ea0 con 0x559214ce1ff0 2024-06-13T07:03:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.170+0000 7f8a7b400640 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214d4e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.173+0000 7f8a62000640 10 client.26567 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.173+0000 7f8a7b400640 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214d4e7e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a0b84e7cc932f2 server_cookie=2870a97557498e4 in_seq=0 out_seq=0 2024-06-13T07:03:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.214+0000 7f8a62000640 1 -- 172.21.15.7:0/459242673 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559214c36ea0 con 0x559214ce1ff0 2024-06-13T07:03:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.214+0000 7f8a62000640 10 client.26567 handle_command_reply: tid=0 2024-06-13T07:03:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.214+0000 7f8a85fce740 1 client.26567 shutdown 2024-06-13T07:03:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.215+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 msgr2=0x559214cda2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.215+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002d040 tx=0x7f8a700050c0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 shutdown_connections 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559214ce1ff0 0x559214d4e7e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559214cd1940 0x559214d4ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559214ccc7d0 0x559214cda7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 --2- 172.21.15.7:0/459242673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559214ccbde0 0x559214cda2b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a62000640 0 client.26567 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 >> 172.21.15.7:0/459242673 conn(0x559214c93fb0 msgr2=0x559214cd4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 shutdown_connections 2024-06-13T07:03:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.216+0000 7f8a85fce740 1 -- 172.21.15.7:0/459242673 wait complete. 2024-06-13T07:03:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.217+0000 7f8a85fce740 20 client.26567 trim_cache size 0 max 16384 2024-06-13T07:03:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:03:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:03:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:03:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 msgr2=0x7f8a640a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 0x7f8a640a5120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c067280 tx=0x7f8a7c0506f0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 msgr2=0x7f8a74143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a74143b80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009230 tx=0x7f8a68005230 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 shutdown_connections 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a640a2c60 0x7f8a640a5120 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a740a6710 0x7f8a7414abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740a5d70 0x7f8a74143b80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 --2- 172.21.15.7:0/1239303887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a740a4c40 0x7f8a74143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.218+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 >> 172.21.15.7:0/1239303887 conn(0x7f8a740a00d0 msgr2=0x7f8a740a6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.219+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 shutdown_connections 2024-06-13T07:03:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:03:52.219+0000 7f8a81a00640 1 -- 172.21.15.7:0/1239303887 wait complete. 2024-06-13T07:03:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 470.0 s, 368.1 tps, lat 265.286 ms stddev 290.563 2024-06-13T07:03:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.457+0000 7f76d0200700 1 -- 172.21.15.7:0/649556622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c810a6d0 msgr2=0x7f76c810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.457+0000 7f76d0200700 1 --2- 172.21.15.7:0/649556622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c810a6d0 0x7f76c810aab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098f0 tx=0x7f76bc009c00 comp rx=0 tx=0).stop 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 -- 172.21.15.7:0/649556622 shutdown_connections 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 --2- 172.21.15.7:0/649556622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c8105090 0x7f76c81115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 --2- 172.21.15.7:0/649556622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c810a6d0 0x7f76c810aab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 --2- 172.21.15.7:0/649556622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c81046d0 0x7f76c8104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 -- 172.21.15.7:0/649556622 >> 172.21.15.7:0/649556622 conn(0x7f76c81001c0 msgr2=0x7f76c81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 -- 172.21.15.7:0/649556622 shutdown_connections 2024-06-13T07:03:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.458+0000 7f76d0200700 1 -- 172.21.15.7:0/649556622 wait complete. 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.460+0000 7f76d0200700 1 Processor -- start 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.460+0000 7f76d0200700 1 -- start start 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81046d0 0x7f76c80769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 0x7f76c8077420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8114a20 con 0x7f76c81046d0 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81148a0 con 0x7f76c810a6d0 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76d0200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8114ba0 con 0x7f76c8105090 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76cce00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.462+0000 7f76cce00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34450/0 (socket says 172.21.15.7:34450) 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.462+0000 7f76cce00700 1 -- 172.21.15.7:0/250386881 learned_addr learned my addr 172.21.15.7:0/250386881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.461+0000 7f76ce200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 0x7f76c8077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.462+0000 7f76cd800700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81046d0 0x7f76c80769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 -- 172.21.15.7:0/250386881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 msgr2=0x7f76c8077420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 0x7f76c8077420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 -- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81046d0 msgr2=0x7f76c80769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81046d0 0x7f76c80769a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 -- 172.21.15.7:0/250386881 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c81ab540 con 0x7f76c8105090 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76ce200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 0x7f76c8077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.463+0000 7f76cce00700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ec40 tx=0x7f76b800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.464+0000 7f76c6000700 1 -- 172.21.15.7:0/250386881 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800ce10 con 0x7f76c8105090 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.464+0000 7f76c6000700 1 -- 172.21.15.7:0/250386881 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b8004500 con 0x7f76c8105090 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.464+0000 7f76c6000700 1 -- 172.21.15.7:0/250386881 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8010690 con 0x7f76c8105090 2024-06-13T07:03:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.464+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c81ab830 con 0x7f76c8105090 2024-06-13T07:03:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.465+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81abd20 con 0x7f76c8105090 2024-06-13T07:03:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.473+0000 7f76dab4db80 1 Processor -- start 2024-06-13T07:03:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.473+0000 7f76c6000700 1 -- 172.21.15.7:0/250386881 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f76b8003880 con 0x7f76c8105090 2024-06-13T07:03:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.474+0000 7f76dab4db80 1 -- start start 2024-06-13T07:03:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f385f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f380680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55917f28ea10 con 0x55917f380240 2024-06-13T07:03:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55917f2d1b00 con 0x55917f3864b0 2024-06-13T07:03:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76dab4db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55917f28c8f0 con 0x55917f385b90 2024-06-13T07:03:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76cd800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.475+0000 7f76cd800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40498/0 (socket says 172.21.15.7:40498) 2024-06-13T07:03:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.476+0000 7f76cd800700 1 -- 172.21.15.7:0/1735302897 learned_addr learned my addr 172.21.15.7:0/1735302897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.476+0000 7f76ce200700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f385f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.477+0000 7f76cce00700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f380680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.477+0000 7f76cd800700 1 -- 172.21.15.7:0/1735302897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 msgr2=0x55917f385f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.477+0000 7f76cd800700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f385f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.477+0000 7f76cd800700 1 -- 172.21.15.7:0/1735302897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 msgr2=0x55917f380680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.478+0000 7f76cd800700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f380680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.478+0000 7f76c6000700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 0x7f76b00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.478+0000 7f76cd800700 1 -- 172.21.15.7:0/1735302897 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917f279f90 con 0x55917f3864b0 2024-06-13T07:03:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.479+0000 7f76cce00700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f380680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.479+0000 7f76c6000700 1 -- 172.21.15.7:0/250386881 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f76b80cf6c0 con 0x7f76c8105090 2024-06-13T07:03:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.479+0000 7f76ce200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 0x7f76b00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.480+0000 7f76cd800700 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098f0 tx=0x7f76bc009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38aaf9f35f549850 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.480+0000 7f76b6a00700 1 -- 172.21.15.7:0/1735302897 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc027070 con 0x55917f3864b0 2024-06-13T07:03:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.481+0000 7f76ce200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 0x7f76b00a5210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009fd0 tx=0x7f76c0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.481+0000 7f76b6a00700 1 -- 172.21.15.7:0/1735302897 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc005520 con 0x55917f3864b0 2024-06-13T07:03:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.481+0000 7f76b6a00700 1 -- 172.21.15.7:0/1735302897 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004d60 con 0x55917f3864b0 2024-06-13T07:03:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 -- 172.21.15.7:0/1735302897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 msgr2=0x55917f37fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098f0 tx=0x7f76bc009610 comp rx=0 tx=0).stop 2024-06-13T07:03:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 -- 172.21.15.7:0/1735302897 shutdown_connections 2024-06-13T07:03:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f380680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f385f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1735302897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f37fd00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.482+0000 7f76dab4db80 1 -- 172.21.15.7:0/1735302897 >> 172.21.15.7:0/1735302897 conn(0x55917f2ac170 msgr2=0x55917f37aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.483+0000 7f76dab4db80 1 -- 172.21.15.7:0/1735302897 shutdown_connections 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.483+0000 7f76dab4db80 1 -- 172.21.15.7:0/1735302897 wait complete. 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.484+0000 7f76dab4db80 1 -- start start 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.484+0000 7f76dab4db80 1 Processor -- start 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.485+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f38c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.485+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f3894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.485+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f389a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76dab4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55917f26df60 con 0x55917f380240 2024-06-13T07:03:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76dab4db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55917f2cdac0 con 0x55917f3864b0 2024-06-13T07:03:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76dab4db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55917f38ce30 con 0x55917f385b90 2024-06-13T07:03:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76dab4db80 10 client.0 _ll_register_callbacks cb 0x55917f267380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76cce00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f3894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76cce00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f3894c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34476/0 (socket says 172.21.15.7:34476) 2024-06-13T07:03:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.486+0000 7f76cce00700 1 -- 172.21.15.7:0/2704926866 learned_addr learned my addr 172.21.15.7:0/2704926866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.487+0000 7f76cd800700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f38c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.487+0000 7f76ce200700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f389a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.488+0000 7f76ce200700 1 -- 172.21.15.7:0/2704926866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 msgr2=0x55917f3894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.488+0000 7f76ce200700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f3894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.488+0000 7f76ce200700 1 -- 172.21.15.7:0/2704926866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 msgr2=0x55917f38c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.488+0000 7f76ce200700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f38c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.488+0000 7f76ce200700 1 -- 172.21.15.7:0/2704926866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917f2208d0 con 0x55917f3864b0 2024-06-13T07:03:53.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.491+0000 7f76ce200700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f389a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007a50 tx=0x7f76c000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbca43dec958ecd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.491+0000 7f76ab400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:03:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.495+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0026030 con 0x55917f3864b0 2024-06-13T07:03:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.496+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76bc005520 con 0x55917f3864b0 2024-06-13T07:03:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.496+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f76c0015a30 con 0x55917f3864b0 2024-06-13T07:03:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.497+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76c0030050 con 0x55917f3864b0 2024-06-13T07:03:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.497+0000 7f76dab4db80 10 client.26579 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.497+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55917f397580 con 0x55917f3864b0 2024-06-13T07:03:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.497+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc005520 con 0x55917f3864b0 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.497+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f76c0004400 con 0x55917f3864b0 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 10 client.26579 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 20 client.26579 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f399b10 0x55917f39bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 4 client.26579 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.498+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917f2df000 con 0x55917f399b10 2024-06-13T07:03:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.499+0000 7f76cd800700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f399b10 0x55917f39bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.504+0000 7f76cd800700 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f399b10 0x55917f39bf70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717732abcb64665a server_cookie=3cb2cd790ad8256 in_seq=0 out_seq=0 2024-06-13T07:03:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.505+0000 7f76ab400700 10 client.26579 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:53.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.506+0000 7f76ab400700 1 -- 172.21.15.7:0/2704926866 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55917f2df000 con 0x55917f399b10 2024-06-13T07:03:53.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.506+0000 7f76ab400700 10 client.26579 handle_command_reply: tid=0 2024-06-13T07:03:53.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.506+0000 7f76dab4db80 1 client.26579 shutdown 2024-06-13T07:03:53.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 msgr2=0x55917f389a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f389a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f76c0007a50 tx=0x7f76c000b040 comp rx=0 tx=0).stop 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 shutdown_connections 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f399b10 0x55917f39bf70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f3864b0 0x55917f389a00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f38c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 --2- 172.21.15.7:0/2704926866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f385b90 0x55917f3894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76ab400700 0 client.26579 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.507+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 >> 172.21.15.7:0/2704926866 conn(0x55917f2ac170 msgr2=0x55917f374790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.508+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 shutdown_connections 2024-06-13T07:03:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.508+0000 7f76dab4db80 1 -- 172.21.15.7:0/2704926866 wait complete. 2024-06-13T07:03:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.508+0000 7f76dab4db80 20 client.26579 trim_cache size 0 max 16384 2024-06-13T07:03:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.557+0000 7f76dab4db80 1 Processor -- start 2024-06-13T07:03:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.557+0000 7f76dab4db80 1 -- start start 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 0x55917f37ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f387120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b70c0 con 0x55917f385b90 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b4920 con 0x55917f380240 2024-06-13T07:03:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76dab4db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b2770 con 0x55917f3864b0 2024-06-13T07:03:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76cce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 0x55917f37ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76cd800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76cd800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40528/0 (socket says 172.21.15.7:40528) 2024-06-13T07:03:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.558+0000 7f76cd800700 1 -- 172.21.15.7:0/162156880 learned_addr learned my addr 172.21.15.7:0/162156880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.559+0000 7f76cce00700 1 -- 172.21.15.7:0/162156880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 msgr2=0x55917f387120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76ce200700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f387120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76cce00700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f387120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76cce00700 1 -- 172.21.15.7:0/162156880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 msgr2=0x55917f37fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76cce00700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76cce00700 1 -- 172.21.15.7:0/162156880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917f38a390 con 0x55917f385b90 2024-06-13T07:03:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76cd800700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:03:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.560+0000 7f76ce200700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f387120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.563+0000 7f76cce00700 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 0x55917f37ae00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f76b80a42c0 tx=0x7f76b80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d11beac3bb4254e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.563+0000 7f76b6000700 1 -- 172.21.15.7:0/162156880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b809b4e0 con 0x55917f385b90 2024-06-13T07:03:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.563+0000 7f76b6000700 1 -- 172.21.15.7:0/162156880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b8003680 con 0x55917f385b90 2024-06-13T07:03:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.563+0000 7f76b6000700 1 -- 172.21.15.7:0/162156880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b80cf3e0 con 0x55917f385b90 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 -- 172.21.15.7:0/162156880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 msgr2=0x55917f37ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 0x55917f37ae00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f76b80a42c0 tx=0x7f76b80096c0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 -- 172.21.15.7:0/162156880 shutdown_connections 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f387120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f380240 0x55917f37fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 --2- 172.21.15.7:0/162156880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f385b90 0x55917f37ae00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.568+0000 7f76dab4db80 1 -- 172.21.15.7:0/162156880 >> 172.21.15.7:0/162156880 conn(0x55917f2ac170 msgr2=0x55917f38acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.569+0000 7f76dab4db80 1 -- 172.21.15.7:0/162156880 shutdown_connections 2024-06-13T07:03:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.569+0000 7f76dab4db80 1 -- 172.21.15.7:0/162156880 wait complete. 2024-06-13T07:03:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.569+0000 7f76dab4db80 1 -- start start 2024-06-13T07:03:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.570+0000 7f76dab4db80 1 Processor -- start 2024-06-13T07:03:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f392870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f385b90 0x55917f3a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b2de0 con 0x55917f380240 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b4250 con 0x55917f385b90 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76dab4db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81b34b0 con 0x55917f3864b0 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76ce200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76ce200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34504/0 (socket says 172.21.15.7:34504) 2024-06-13T07:03:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.571+0000 7f76ce200700 1 -- 172.21.15.7:0/1676142622 learned_addr learned my addr 172.21.15.7:0/1676142622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76dab4db80 10 client.0 _ll_register_callbacks cb 0x55917f3840a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76cce00700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f385b90 0x55917f3a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76ce200700 1 -- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f385b90 msgr2=0x55917f3a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76ce200700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f385b90 0x55917f3a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76ce200700 1 -- 172.21.15.7:0/1676142622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 msgr2=0x55917f392870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:03:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.572+0000 7f76cd800700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f392870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.573+0000 7f76ce200700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f392870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.573+0000 7f76ce200700 1 -- 172.21.15.7:0/1676142622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917f3a1f10 con 0x55917f3864b0 2024-06-13T07:03:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.573+0000 7f76cd800700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f392870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:03:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.574+0000 7f76ce200700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f76c00048a0 tx=0x7f76c0015720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f161d9c258bb4375 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:03:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.574+0000 7f76ab400700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:03:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.575+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0026040 con 0x55917f3864b0 2024-06-13T07:03:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.576+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f76c0004d40 con 0x55917f3864b0 2024-06-13T07:03:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.576+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b8003680 con 0x55917f3864b0 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.576+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f76c0045050 con 0x55917f3864b0 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.576+0000 7f76dab4db80 10 client.26530 fetch_fsmap learned FSMap version 37 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.576+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55917f404830 con 0x55917f3864b0 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b8003680 con 0x55917f3864b0 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f76c003a020 con 0x55917f3864b0 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 10 client.26530 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:03:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:03:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 20 client.26530 populate_metadata read hostname 'smithi007' 2024-06-13T07:03:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f404f70 0x55917f407390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:03:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 4 client.26530 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:03:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.577+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55917f2d9580 con 0x55917f404f70 2024-06-13T07:03:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.578+0000 7f76cd800700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f404f70 0x55917f407390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:03:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.579+0000 7f76ab400700 10 client.26530 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.579+0000 7f76cd800700 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f404f70 0x55917f407390 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c490eb8a7a6f4fa7 server_cookie=4a9ff650ed49f98f in_seq=0 out_seq=0 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.580+0000 7f76ab400700 1 -- 172.21.15.7:0/1676142622 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55917f2d9580 con 0x55917f404f70 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.580+0000 7f76ab400700 10 client.26530 handle_command_reply: tid=0 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 client.26530 shutdown 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 msgr2=0x55917f3a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f76c00048a0 tx=0x7f76c0015720 comp rx=0 tx=0).stop 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 shutdown_connections 2024-06-13T07:03:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55917f404f70 0x55917f407390 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55917f3864b0 0x55917f3a16d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55917f380240 0x55917f392870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 --2- 172.21.15.7:0/1676142622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55917f385b90 0x55917f3a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 >> 172.21.15.7:0/1676142622 conn(0x55917f2ac170 msgr2=0x55917f384e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.581+0000 7f76ab400700 0 client.26530 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.582+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 shutdown_connections 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.582+0000 7f76dab4db80 1 -- 172.21.15.7:0/1676142622 wait complete. 2024-06-13T07:03:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.582+0000 7f76dab4db80 20 client.26530 trim_cache size 0 max 16384 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "b038ca8c-b2a4-4cf5-977e-a4c9051c3f82": { 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "b038ca8c-b2a4-4cf5-977e-a4c9051c3f82", 2024-06-13T07:03:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:03:53.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:03:53.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:03:53.585 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:03:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.584+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 msgr2=0x7f76b00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.584+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 0x7f76b00a5210 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009fd0 tx=0x7f76c0009040 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 msgr2=0x7f76c8076ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ec40 tx=0x7f76b800c2d0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 shutdown_connections 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f76b00a2d50 0x7f76b00a5210 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c810a6d0 0x7f76c8077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c81046d0 0x7f76c80769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 --2- 172.21.15.7:0/250386881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76c8105090 0x7f76c8076ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 >> 172.21.15.7:0/250386881 conn(0x7f76c81001c0 msgr2=0x7f76c81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 shutdown_connections 2024-06-13T07:03:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:03:53.585+0000 7f76d0200700 1 -- 172.21.15.7:0/250386881 wait complete. 2024-06-13T07:03:53.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:53 smithi007 ceph-mon[32077]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.0 MiB/s wr, 513 op/s 2024-06-13T07:03:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:53 smithi189 ceph-mon[36616]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.0 MiB/s wr, 513 op/s 2024-06-13T07:03:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:53 smithi175 ceph-mon[36886]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.0 MiB/s wr, 513 op/s 2024-06-13T07:03:54.443 INFO:tasks.cephfs.filesystem:scrub status for tag:b038ca8c-b2a4-4cf5-977e-a4c9051c3f82 - {'path': '/', 'tag': 'b038ca8c-b2a4-4cf5-977e-a4c9051c3f82', 'options': 'recursive,force'} 2024-06-13T07:03:55.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:55 smithi007 ceph-mon[32077]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.0 MiB/s wr, 612 op/s 2024-06-13T07:03:56.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:55 smithi189 ceph-mon[36616]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.0 MiB/s wr, 612 op/s 2024-06-13T07:03:56.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:55 smithi175 ceph-mon[36886]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.0 MiB/s wr, 612 op/s 2024-06-13T07:03:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 475.0 s, 548.6 tps, lat 182.446 ms stddev 207.249 2024-06-13T07:03:58.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:57 smithi189 ceph-mon[36616]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2024-06-13T07:03:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:57 smithi175 ceph-mon[36886]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2024-06-13T07:03:58.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:57 smithi007 ceph-mon[32077]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 3.5 MiB/s wr, 458 op/s 2024-06-13T07:03:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:58 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:03:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:58 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:03:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:58 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:03:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:58 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:03:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:58 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:03:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:58 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:03:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:58 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:03:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:58 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:03:59.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:58 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:04:00.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:03:59 smithi189 ceph-mon[36616]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-06-13T07:04:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:03:59 smithi175 ceph-mon[36886]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-06-13T07:04:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:03:59 smithi007 ceph-mon[32077]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.1 MiB/s wr, 528 op/s 2024-06-13T07:04:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:01 smithi189 ceph-mon[36616]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-06-13T07:04:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:01 smithi175 ceph-mon[36886]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-06-13T07:04:02.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:01 smithi007 ceph-mon[32077]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.1 MiB/s wr, 546 op/s 2024-06-13T07:04:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 480.0 s, 655.4 tps, lat 151.835 ms stddev 172.807 2024-06-13T07:04:04.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:03 smithi189 ceph-mon[36616]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:04.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:03 smithi175 ceph-mon[36886]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:04.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:03 smithi007 ceph-mon[32077]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 -- 172.21.15.7:0/921170313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5381035b0 msgr2=0x7fa538109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 --2- 172.21.15.7:0/921170313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5381035b0 0x7fa538109e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa530009ca0 tx=0x7fa53002a8c0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 -- 172.21.15.7:0/921170313 shutdown_connections 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 --2- 172.21.15.7:0/921170313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5381035b0 0x7fa538109e40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 --2- 172.21.15.7:0/921170313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa538102bf0 0x7fa538103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 --2- 172.21.15.7:0/921170313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538101a00 0x7fa538101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.438+0000 7fa53f200640 1 -- 172.21.15.7:0/921170313 >> 172.21.15.7:0/921170313 conn(0x7fa5380fcf90 msgr2=0x7fa5380ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.439+0000 7fa53f200640 1 -- 172.21.15.7:0/921170313 shutdown_connections 2024-06-13T07:04:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.439+0000 7fa53f200640 1 -- 172.21.15.7:0/921170313 wait complete. 2024-06-13T07:04:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.440+0000 7fa53f200640 1 Processor -- start 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 -- start start 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 0x7fa538197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 0x7fa538198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 0x7fa53819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa53810d6b0 con 0x7fa5381035b0 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa53810d530 con 0x7fa538102bf0 2024-06-13T07:04:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa53f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa53810d830 con 0x7fa538101a00 2024-06-13T07:04:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 0x7fa538197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 0x7fa538197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42388/0 (socket says 172.21.15.7:42388) 2024-06-13T07:04:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa537e00640 1 -- 172.21.15.7:0/1277347725 learned_addr learned my addr 172.21.15.7:0/1277347725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.441+0000 7fa537400640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 0x7fa538198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa53d200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 0x7fa53819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa537400640 1 -- 172.21.15.7:0/1277347725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 msgr2=0x7fa538197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa537400640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 0x7fa538197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa537400640 1 -- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 msgr2=0x7fa53819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa537400640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 0x7fa53819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.442+0000 7fa537400640 1 -- 172.21.15.7:0/1277347725 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53819f7f0 con 0x7fa538102bf0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa53d200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 0x7fa53819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa537400640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 0x7fa538198110 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9f0 tx=0x7fa52400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa534c00640 1 -- 172.21.15.7:0/1277347725 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400ccc0 con 0x7fa538102bf0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa534c00640 1 -- 172.21.15.7:0/1277347725 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa524010910 con 0x7fa538102bf0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53819fae0 con 0x7fa538102bf0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.443+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a0030 con 0x7fa538102bf0 2024-06-13T07:04:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.444+0000 7fa534c00640 1 -- 172.21.15.7:0/1277347725 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524018960 con 0x7fa538102bf0 2024-06-13T07:04:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.450+0000 7fa534c00640 1 -- 172.21.15.7:0/1277347725 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa524018b00 con 0x7fa538102bf0 2024-06-13T07:04:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.452+0000 7fa534c00640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 0x7fa51c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.453+0000 7fa534c00640 1 -- 172.21.15.7:0/1277347725 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa5240d0610 con 0x7fa538102bf0 2024-06-13T07:04:05.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.454+0000 7fa537e00640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 0x7fa51c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.461+0000 7fa537e00640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 0x7fa51c0a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5280095f0 tx=0x7fa528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.463+0000 7fa541f8e740 1 Processor -- start 2024-06-13T07:04:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.463+0000 7fa541f8e740 1 -- start start 2024-06-13T07:04:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a44e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a4530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a455bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c02a3b1420 con 0x55c02a44e5f0 2024-06-13T07:04:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c02a358c50 con 0x55c02a44dc00 2024-06-13T07:04:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.464+0000 7fa541f8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c02a3ad3e0 con 0x55c02a453760 2024-06-13T07:04:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a44e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a44e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37744/0 (socket says 172.21.15.7:37744) 2024-06-13T07:04:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537e00640 1 -- 172.21.15.7:0/3518148205 learned_addr learned my addr 172.21.15.7:0/3518148205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a4530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa53d200640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a455bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 -- 172.21.15.7:0/3518148205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 msgr2=0x55c02a455bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a455bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 -- 172.21.15.7:0/3518148205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 msgr2=0x55c02a44e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a44e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa537400640 1 -- 172.21.15.7:0/3518148205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02a0c5210 con 0x55c02a44e5f0 2024-06-13T07:04:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.465+0000 7fa53d200640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a455bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.466+0000 7fa537400640 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a4530f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ce140 tx=0x7fa52400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d0d55ec1906330 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.467+0000 7fa52d600640 1 -- 172.21.15.7:0/3518148205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52409de60 con 0x55c02a44e5f0 2024-06-13T07:04:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.467+0000 7fa52d600640 1 -- 172.21.15.7:0/3518148205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5240a57a0 con 0x55c02a44e5f0 2024-06-13T07:04:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.467+0000 7fa52d600640 1 -- 172.21.15.7:0/3518148205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524010ac0 con 0x55c02a44e5f0 2024-06-13T07:04:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.468+0000 7fa541f8e740 1 -- 172.21.15.7:0/3518148205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 msgr2=0x55c02a4530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.468+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a4530f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ce140 tx=0x7fa52400cf60 comp rx=0 tx=0).stop 2024-06-13T07:04:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 -- 172.21.15.7:0/3518148205 shutdown_connections 2024-06-13T07:04:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a455bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a4530f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3518148205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a44e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 -- 172.21.15.7:0/3518148205 >> 172.21.15.7:0/3518148205 conn(0x55c02a415fb0 msgr2=0x55c02a449680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 -- 172.21.15.7:0/3518148205 shutdown_connections 2024-06-13T07:04:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.469+0000 7fa541f8e740 1 -- 172.21.15.7:0/3518148205 wait complete. 2024-06-13T07:04:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.470+0000 7fa541f8e740 1 -- start start 2024-06-13T07:04:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.470+0000 7fa541f8e740 1 Processor -- start 2024-06-13T07:04:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.471+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.471+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 0x55c02a45c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.471+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a4583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.471+0000 7fa537e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.471+0000 7fa537e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56448/0 (socket says 172.21.15.7:56448) 2024-06-13T07:04:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa537400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 0x55c02a45c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa537e00640 1 -- 172.21.15.7:0/547740042 learned_addr learned my addr 172.21.15.7:0/547740042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa53d200640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a4583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c02a45ab20 con 0x55c02a44dc00 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c02a36c0b0 con 0x55c02a44e5f0 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c02a45aca0 con 0x55c02a453760 2024-06-13T07:04:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.472+0000 7fa541f8e740 10 client.0 _ll_register_callbacks cb 0x55c02a357f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.473+0000 7fa537e00640 1 -- 172.21.15.7:0/547740042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 msgr2=0x55c02a4583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.473+0000 7fa537e00640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a4583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.473+0000 7fa537e00640 1 -- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 msgr2=0x55c02a45c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.473+0000 7fa537e00640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 0x55c02a45c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.473+0000 7fa537e00640 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02a191620 con 0x55c02a44dc00 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.474+0000 7fa537400640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 0x55c02a45c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.474+0000 7fa537e00640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa5280058c0 tx=0x7fa528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c094e2c95b19bf21 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.474+0000 7fa522000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:04:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.474+0000 7fa53d200640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a4583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.475+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52802b060 con 0x55c02a44dc00 2024-06-13T07:04:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.475+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa52801ca60 con 0x55c02a44dc00 2024-06-13T07:04:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.475+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5240a57a0 con 0x55c02a44dc00 2024-06-13T07:04:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.478+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa52801cc60 con 0x55c02a44dc00 2024-06-13T07:04:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.478+0000 7fa541f8e740 10 client.16032 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.478+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c02a45a570 con 0x55c02a44dc00 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.478+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5240a57a0 con 0x55c02a44dc00 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.478+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa52800e4e0 con 0x55c02a44dc00 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 10 client.16032 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 20 client.16032 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a46c6c0 0x55c02a46eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 4 client.16032 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c02a3beb50 con 0x55c02a46c6c0 2024-06-13T07:04:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.479+0000 7fa537400640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a46c6c0 0x55c02a46eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.480+0000 7fa537400640 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a46c6c0 0x55c02a46eb20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64aa994465f7f23 server_cookie=a8cf9e8c0a2832d1 in_seq=0 out_seq=0 2024-06-13T07:04:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.480+0000 7fa522000640 10 client.16032 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.481+0000 7fa522000640 1 -- 172.21.15.7:0/547740042 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c02a3beb50 con 0x55c02a46c6c0 2024-06-13T07:04:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.481+0000 7fa522000640 10 client.16032 handle_command_reply: tid=0 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.481+0000 7fa541f8e740 1 client.16032 shutdown 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.481+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 msgr2=0x55c02a45c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.481+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa5280058c0 tx=0x7fa528004730 comp rx=0 tx=0).stop 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 shutdown_connections 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a46c6c0 0x55c02a46eb20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a4583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44e5f0 0x55c02a45c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 --2- 172.21.15.7:0/547740042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44dc00 0x55c02a45c350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 >> 172.21.15.7:0/547740042 conn(0x55c02a415fb0 msgr2=0x55c02a448840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.482+0000 7fa522000640 0 client.16032 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.484+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 shutdown_connections 2024-06-13T07:04:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.484+0000 7fa541f8e740 1 -- 172.21.15.7:0/547740042 wait complete. 2024-06-13T07:04:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.484+0000 7fa541f8e740 20 client.16032 trim_cache size 0 max 16384 2024-06-13T07:04:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.516+0000 7fa541f8e740 1 Processor -- start 2024-06-13T07:04:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.516+0000 7fa541f8e740 1 -- start start 2024-06-13T07:04:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a449100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a45bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa518012400 con 0x55c02a44e5f0 2024-06-13T07:04:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa51800fca0 con 0x55c02a44dc00 2024-06-13T07:04:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa541f8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa51800daf0 con 0x55c02a453760 2024-06-13T07:04:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.517+0000 7fa537400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56454/0 (socket says 172.21.15.7:56454) 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 -- 172.21.15.7:0/1422393935 learned_addr learned my addr 172.21.15.7:0/1422393935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa53d200640 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a45bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 -- 172.21.15.7:0/1422393935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 msgr2=0x55c02a45bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a45bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 -- 172.21.15.7:0/1422393935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 msgr2=0x55c02a449100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a449100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa537400640 1 -- 172.21.15.7:0/1422393935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c02a45d110 con 0x55c02a44e5f0 2024-06-13T07:04:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.518+0000 7fa53d200640 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a45bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.519+0000 7fa537400640 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa52409c970 tx=0x7fa5240a1060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b06948c2e1df9f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.522+0000 7fa52cc00640 1 -- 172.21.15.7:0/1422393935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240bb040 con 0x55c02a44e5f0 2024-06-13T07:04:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.523+0000 7fa52cc00640 1 -- 172.21.15.7:0/1422393935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5240d03a0 con 0x55c02a44e5f0 2024-06-13T07:04:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.523+0000 7fa52cc00640 1 -- 172.21.15.7:0/1422393935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240bb040 con 0x55c02a44e5f0 2024-06-13T07:04:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.524+0000 7fa541f8e740 1 -- 172.21.15.7:0/1422393935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 msgr2=0x55c02a449640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.524+0000 7fa541f8e740 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa52409c970 tx=0x7fa5240a1060 comp rx=0 tx=0).stop 2024-06-13T07:04:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.525+0000 7fa541f8e740 1 -- 172.21.15.7:0/1422393935 shutdown_connections 2024-06-13T07:04:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.525+0000 7fa541f8e740 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a453760 0x55c02a45bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.525+0000 7fa541f8e740 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a44e5f0 0x55c02a449640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.525+0000 7fa541f8e740 1 --2- 172.21.15.7:0/1422393935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a449100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.525+0000 7fa541f8e740 1 -- 172.21.15.7:0/1422393935 >> 172.21.15.7:0/1422393935 conn(0x55c02a415fb0 msgr2=0x55c02a46e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.527+0000 7fa541f8e740 1 -- 172.21.15.7:0/1422393935 shutdown_connections 2024-06-13T07:04:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.527+0000 7fa541f8e740 1 -- 172.21.15.7:0/1422393935 wait complete. 2024-06-13T07:04:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.528+0000 7fa541f8e740 1 -- start start 2024-06-13T07:04:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.529+0000 7fa541f8e740 1 Processor -- start 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 0x55c02a47db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a453760 0x55c02a4d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa51800e160 con 0x55c02a453760 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa51800f5d0 con 0x55c02a44dc00 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa51800e830 con 0x55c02a44e5f0 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa541f8e740 10 client.0 _ll_register_callbacks cb 0x7fa5280097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa537400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 0x55c02a47db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa537e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa537e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37766/0 (socket says 172.21.15.7:37766) 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa537e00640 1 -- 172.21.15.7:0/3713015031 learned_addr learned my addr 172.21.15.7:0/3713015031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.530+0000 7fa53d200640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a453760 0x55c02a4d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 -- 172.21.15.7:0/3713015031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 msgr2=0x55c02a47db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 0x55c02a47db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 -- 172.21.15.7:0/3713015031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a453760 msgr2=0x55c02a4d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a453760 0x55c02a4d5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 -- 172.21.15.7:0/3713015031 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c02a480bc0 con 0x55c02a44dc00 2024-06-13T07:04:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537400640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 0x55c02a47db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa537e00640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa528011d10 tx=0x7fa528011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=737eb23b37f791 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.531+0000 7fa522000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.532+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52802b040 con 0x55c02a44dc00 2024-06-13T07:04:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.532+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5240d03a0 con 0x55c02a44dc00 2024-06-13T07:04:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.533+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa528023040 con 0x55c02a44dc00 2024-06-13T07:04:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.534+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa52803f050 con 0x55c02a44dc00 2024-06-13T07:04:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.534+0000 7fa541f8e740 10 client.26597 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.534+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c02a461cc0 con 0x55c02a44dc00 2024-06-13T07:04:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.535+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5240d03a0 con 0x55c02a44dc00 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fa528004680 con 0x55c02a44dc00 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 10 client.26597 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 10 client.26597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 20 client.26597 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a462690 0x55c02a464ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 4 client.26597 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c02a3b8ea0 con 0x55c02a462690 2024-06-13T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.536+0000 7fa537400640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a462690 0x55c02a464ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.538+0000 7fa537400640 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a462690 0x55c02a464ab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b839e440e32122 server_cookie=6f041bc3b166cb21 in_seq=0 out_seq=0 2024-06-13T07:04:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.538+0000 7fa522000640 10 client.26597 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.599+0000 7fa522000640 1 -- 172.21.15.7:0/3713015031 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c02a3b8ea0 con 0x55c02a462690 2024-06-13T07:04:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.599+0000 7fa522000640 10 client.26597 handle_command_reply: tid=0 2024-06-13T07:04:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.599+0000 7fa541f8e740 1 client.26597 shutdown 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.599+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 msgr2=0x55c02a47d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.599+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa528011d10 tx=0x7fa528011e30 comp rx=0 tx=0).stop 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 shutdown_connections 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c02a462690 0x55c02a464ab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c02a453760 0x55c02a4d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c02a44e5f0 0x55c02a47db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 --2- 172.21.15.7:0/3713015031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c02a44dc00 0x55c02a47d650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa522000640 0 client.26597 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.600+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 >> 172.21.15.7:0/3713015031 conn(0x55c02a415fb0 msgr2=0x55c02a46e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.601+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 shutdown_connections 2024-06-13T07:04:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.601+0000 7fa541f8e740 1 -- 172.21.15.7:0/3713015031 wait complete. 2024-06-13T07:04:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.601+0000 7fa541f8e740 20 client.26597 trim_cache size 0 max 16384 2024-06-13T07:04:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:04:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:04:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:04:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.602+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 msgr2=0x7fa51c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.602+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 0x7fa51c0a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5280095f0 tx=0x7fa528005f90 comp rx=0 tx=0).stop 2024-06-13T07:04:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.602+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 msgr2=0x7fa538198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.602+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 0x7fa538198110 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9f0 tx=0x7fa52400eeb0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 shutdown_connections 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa51c0a2da0 0x7fa51c0a5260 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5381035b0 0x7fa53819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa538102bf0 0x7fa538198110 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 --2- 172.21.15.7:0/1277347725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa538101a00 0x7fa538197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 >> 172.21.15.7:0/1277347725 conn(0x7fa5380fcf90 msgr2=0x7fa538105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.603+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 shutdown_connections 2024-06-13T07:04:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:05.604+0000 7fa53f200640 1 -- 172.21.15.7:0/1277347725 wait complete. 2024-06-13T07:04:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:05 smithi189 ceph-mon[36616]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 563 op/s 2024-06-13T07:04:06.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:05 smithi175 ceph-mon[36886]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 563 op/s 2024-06-13T07:04:06.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:05 smithi007 ceph-mon[32077]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 563 op/s 2024-06-13T07:04:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 485.0 s, 469.8 tps, lat 209.473 ms stddev 312.023 2024-06-13T07:04:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:07 smithi189 ceph-mon[36616]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-06-13T07:04:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:07 smithi175 ceph-mon[36886]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-06-13T07:04:08.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:07 smithi007 ceph-mon[32077]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-06-13T07:04:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:09 smithi189 ceph-mon[36616]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:04:10.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:09 smithi175 ceph-mon[36886]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:04:10.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:09 smithi007 ceph-mon[32077]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:04:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:11 smithi189 ceph-mon[36616]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-06-13T07:04:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:11 smithi175 ceph-mon[36886]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-06-13T07:04:12.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:11 smithi007 ceph-mon[32077]: pgmap v753: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-06-13T07:04:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 490.0 s, 610.2 tps, lat 166.792 ms stddev 190.544 2024-06-13T07:04:13.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:12 smithi189 ceph-mon[36616]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-13T07:04:13.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:12 smithi175 ceph-mon[36886]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-13T07:04:13.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:12 smithi007 ceph-mon[32077]: pgmap v754: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 491 op/s 2024-06-13T07:04:15.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:15 smithi189 ceph-mon[36616]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-06-13T07:04:15.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:15 smithi175 ceph-mon[36886]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-06-13T07:04:15.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:15 smithi007 ceph-mon[32077]: pgmap v755: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-06-13T07:04:17.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:17 smithi189 ceph-mon[36616]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 MiB/s wr, 434 op/s 2024-06-13T07:04:17.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:17 smithi175 ceph-mon[36886]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 MiB/s wr, 434 op/s 2024-06-13T07:04:17.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:17 smithi007 ceph-mon[32077]: pgmap v756: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 MiB/s wr, 434 op/s 2024-06-13T07:04:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 495.0 s, 517.9 tps, lat 184.805 ms stddev 244.376 2024-06-13T07:04:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:19 smithi175 ceph-mon[36886]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-06-13T07:04:19.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:19 smithi007 ceph-mon[32077]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-06-13T07:04:19.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:19 smithi189 ceph-mon[36616]: pgmap v757: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-06-13T07:04:21.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:21 smithi175 ceph-mon[36886]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-06-13T07:04:21.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:21 smithi007 ceph-mon[32077]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-06-13T07:04:21.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:21 smithi189 ceph-mon[36616]: pgmap v758: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 -- 172.21.15.7:0/2226292565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 msgr2=0x7ff63810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 --2- 172.21.15.7:0/2226292565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff63810de70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff62002a3f0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 -- 172.21.15.7:0/2226292565 shutdown_connections 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 --2- 172.21.15.7:0/2226292565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63810e500 0x7ff6381109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 --2- 172.21.15.7:0/2226292565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff63810de70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 --2- 172.21.15.7:0/2226292565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63806a7d0 0x7ff63810b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.678+0000 7ff63d200640 1 -- 172.21.15.7:0/2226292565 >> 172.21.15.7:0/2226292565 conn(0x7ff63806ebd0 msgr2=0x7ff638071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.679+0000 7ff63d200640 1 -- 172.21.15.7:0/2226292565 shutdown_connections 2024-06-13T07:04:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.679+0000 7ff63d200640 1 -- 172.21.15.7:0/2226292565 wait complete. 2024-06-13T07:04:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.680+0000 7ff63d200640 1 Processor -- start 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.680+0000 7ff63d200640 1 -- start start 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806a7d0 0x7ff6381a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff6381a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 0x7ff6381a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff638115f10 con 0x7ff63806a7d0 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff638115d90 con 0x7ff63810e500 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff63d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff638116090 con 0x7ff63810ba80 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff636800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 0x7ff6381a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff6381a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff636800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 0x7ff6381a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57318/0 (socket says 172.21.15.7:57318) 2024-06-13T07:04:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.681+0000 7ff636800640 1 -- 172.21.15.7:0/3410192653 learned_addr learned my addr 172.21.15.7:0/3410192653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 -- 172.21.15.7:0/3410192653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 msgr2=0x7ff6381a7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 0x7ff6381a7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 -- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806a7d0 msgr2=0x7ff6381a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806a7d0 0x7ff6381a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 -- 172.21.15.7:0/3410192653 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6381a80a0 con 0x7ff63810ba80 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.682+0000 7ff62fe00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff6381a09c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6200314e0 tx=0x7ff620032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.683+0000 7ff62ea00640 1 -- 172.21.15.7:0/3410192653 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62003f030 con 0x7ff63810ba80 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.683+0000 7ff62ea00640 1 -- 172.21.15.7:0/3410192653 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff620003ce0 con 0x7ff63810ba80 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.683+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6381a8330 con 0x7ff63810ba80 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.683+0000 7ff62ea00640 1 -- 172.21.15.7:0/3410192653 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620035070 con 0x7ff63810ba80 2024-06-13T07:04:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.683+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381a87f0 con 0x7ff63810ba80 2024-06-13T07:04:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.692+0000 7ff63ffff740 1 Processor -- start 2024-06-13T07:04:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.693+0000 7ff63ffff740 1 -- start start 2024-06-13T07:04:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.693+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.693+0000 7ff62ea00640 1 -- 172.21.15.7:0/3410192653 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff620042030 con 0x7ff63810ba80 2024-06-13T07:04:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.695+0000 7ff636800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.695+0000 7ff636800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37938/0 (socket says 172.21.15.7:37938) 2024-06-13T07:04:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.695+0000 7ff636800640 1 -- 172.21.15.7:0/773296478 learned_addr learned my addr 172.21.15.7:0/773296478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.695+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675aed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff62ea00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 0x7ff61c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675af0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564675a47420 con 0x564675ae8010 2024-06-13T07:04:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5646759eec50 con 0x564675aedb70 2024-06-13T07:04:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff635e00640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675aed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.698+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564675a433e0 con 0x564675ae8a00 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff62fe00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 0x7ff61c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.696+0000 7ff62ea00640 1 -- 172.21.15.7:0/3410192653 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff6200ef4d0 con 0x7ff63810ba80 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.698+0000 7ff636800640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675af0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 -- 172.21.15.7:0/773296478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 msgr2=0x564675af0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675af0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 -- 172.21.15.7:0/773296478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 msgr2=0x564675ae8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 -- 172.21.15.7:0/773296478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467575b210 con 0x564675ae8a00 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff636800640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff635e00640 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675aed500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b30 tx=0x7ff62800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0641c3617627b52 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.699+0000 7ff627400640 1 -- 172.21.15.7:0/773296478 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62800ed80 con 0x564675ae8a00 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.700+0000 7ff627400640 1 -- 172.21.15.7:0/773296478 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628005500 con 0x564675ae8a00 2024-06-13T07:04:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.700+0000 7ff627400640 1 -- 172.21.15.7:0/773296478 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628010690 con 0x564675ae8a00 2024-06-13T07:04:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 msgr2=0x564675aed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675aed500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b30 tx=0x7ff62800e350 comp rx=0 tx=0).stop 2024-06-13T07:04:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 shutdown_connections 2024-06-13T07:04:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675af0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675aed500 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 --2- 172.21.15.7:0/773296478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.701+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 >> 172.21.15.7:0/773296478 conn(0x564675aabfb0 msgr2=0x564675adf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.702+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 shutdown_connections 2024-06-13T07:04:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.702+0000 7ff62fe00640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 0x7ff61c0a5260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff6200bb920 tx=0x7ff620043000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.702+0000 7ff63ffff740 1 -- 172.21.15.7:0/773296478 wait complete. 2024-06-13T07:04:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.703+0000 7ff63ffff740 1 -- start start 2024-06-13T07:04:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.703+0000 7ff63ffff740 1 Processor -- start 2024-06-13T07:04:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564675af0b20 con 0x564675ae8a00 2024-06-13T07:04:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564675a020b0 con 0x564675ae8010 2024-06-13T07:04:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564675af0ca0 con 0x564675aedb70 2024-06-13T07:04:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.706+0000 7ff63ffff740 10 client.0 _ll_register_callbacks cb 0x5646759edf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:22.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.707+0000 7ff635e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.707+0000 7ff636800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.707+0000 7ff636800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57344/0 (socket says 172.21.15.7:57344) 2024-06-13T07:04:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.707+0000 7ff636800640 1 -- 172.21.15.7:0/770186873 learned_addr learned my addr 172.21.15.7:0/770186873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.708+0000 7ff636800640 1 -- 172.21.15.7:0/770186873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 msgr2=0x564675af98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.708+0000 7ff636800640 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.708+0000 7ff636800640 1 -- 172.21.15.7:0/770186873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 msgr2=0x564675af2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.708+0000 7ff636800640 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.708+0000 7ff636800640 1 -- 172.21.15.7:0/770186873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564675827620 con 0x564675ae8010 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.709+0000 7ff636800640 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff630013900 tx=0x7ff630013dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4699d33a1f8f0bc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.709+0000 7ff61be00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.709+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63001a070 con 0x564675ae8010 2024-06-13T07:04:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.709+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff630003970 con 0x564675ae8010 2024-06-13T07:04:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.710+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628005500 con 0x564675ae8010 2024-06-13T07:04:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.710+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff630003b70 con 0x564675ae8010 2024-06-13T07:04:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.711+0000 7ff63ffff740 10 client.26609 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.711+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564675afcd00 con 0x564675ae8010 2024-06-13T07:04:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff628005500 con 0x564675ae8010 2024-06-13T07:04:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7ff6300057c0 con 0x564675ae8010 2024-06-13T07:04:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 10 client.26609 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 10 client.26609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 20 client.26609 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675afe3f0 0x564675b00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 4 client.26609 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.713+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564675a54b50 con 0x564675afe3f0 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.714+0000 7ff635e00640 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675afe3f0 0x564675b00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.714+0000 7ff61be00640 10 client.26609 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.715+0000 7ff635e00640 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675afe3f0 0x564675b00840 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79146b3472ec1d0 server_cookie=4b90f57205819d10 in_seq=0 out_seq=0 2024-06-13T07:04:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.717+0000 7ff61be00640 1 -- 172.21.15.7:0/770186873 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564675a54b50 con 0x564675afe3f0 2024-06-13T07:04:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.717+0000 7ff61be00640 10 client.26609 handle_command_reply: tid=0 2024-06-13T07:04:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.717+0000 7ff63ffff740 1 client.26609 shutdown 2024-06-13T07:04:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.717+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 msgr2=0x564675af2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.717+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff630013900 tx=0x7ff630013dc0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 shutdown_connections 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675afe3f0 0x564675b00840 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 --2- 172.21.15.7:0/770186873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675af2350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 >> 172.21.15.7:0/770186873 conn(0x564675aabfb0 msgr2=0x564675ade840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff61be00640 0 client.26609 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 shutdown_connections 2024-06-13T07:04:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 1 -- 172.21.15.7:0/770186873 wait complete. 2024-06-13T07:04:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.718+0000 7ff63ffff740 20 client.26609 trim_cache size 0 max 16384 2024-06-13T07:04:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.765+0000 7ff63ffff740 1 Processor -- start 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.765+0000 7ff63ffff740 1 -- start start 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675adf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b3300 con 0x564675ae8a00 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b0ba0 con 0x564675ae8010 2024-06-13T07:04:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.766+0000 7ff63ffff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381ae9f0 con 0x564675aedb70 2024-06-13T07:04:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff635e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51810/0 (socket says 172.21.15.7:51810) 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 -- 172.21.15.7:0/1648187085 learned_addr learned my addr 172.21.15.7:0/1648187085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 -- 172.21.15.7:0/1648187085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 msgr2=0x564675adf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675adf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 -- 172.21.15.7:0/1648187085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 msgr2=0x564675af9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.767+0000 7ff636800640 1 -- 172.21.15.7:0/1648187085 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564675aed000 con 0x564675aedb70 2024-06-13T07:04:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.771+0000 7ff636800640 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff630009930 tx=0x7ff630013f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78eb8cf3fd395ecf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.771+0000 7ff626a00640 1 -- 172.21.15.7:0/1648187085 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63001a070 con 0x564675aedb70 2024-06-13T07:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.771+0000 7ff626a00640 1 -- 172.21.15.7:0/1648187085 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff630018070 con 0x564675aedb70 2024-06-13T07:04:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.771+0000 7ff626a00640 1 -- 172.21.15.7:0/1648187085 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630005910 con 0x564675aedb70 2024-06-13T07:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 -- 172.21.15.7:0/1648187085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 msgr2=0x564675af1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff630009930 tx=0x7ff630013f10 comp rx=0 tx=0).stop 2024-06-13T07:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 -- 172.21.15.7:0/1648187085 shutdown_connections 2024-06-13T07:04:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675aedb70 0x564675af1f60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8a00 0x564675af9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 --2- 172.21.15.7:0/1648187085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675ae8010 0x564675adf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.773+0000 7ff63ffff740 1 -- 172.21.15.7:0/1648187085 >> 172.21.15.7:0/1648187085 conn(0x564675aabfb0 msgr2=0x564675adf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.775+0000 7ff63ffff740 1 -- 172.21.15.7:0/1648187085 shutdown_connections 2024-06-13T07:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.775+0000 7ff63ffff740 1 -- 172.21.15.7:0/1648187085 wait complete. 2024-06-13T07:04:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.775+0000 7ff63ffff740 1 -- start start 2024-06-13T07:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.776+0000 7ff63ffff740 1 Processor -- start 2024-06-13T07:04:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.778+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff63ffff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675ae6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff636800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff636800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37958/0 (socket says 172.21.15.7:37958) 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff636800640 1 -- 172.21.15.7:0/3957950653 learned_addr learned my addr 172.21.15.7:0/3957950653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675b630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.779+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381af060 con 0x564675ae8010 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.780+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381b04d0 con 0x564675aedb70 2024-06-13T07:04:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.780+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6381af730 con 0x564675ae8a00 2024-06-13T07:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.780+0000 7ff63ffff740 10 client.0 _ll_register_callbacks cb 0x7ff630013540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.781+0000 7ff636800640 1 -- 172.21.15.7:0/3957950653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 msgr2=0x564675ae6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.781+0000 7ff636800640 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675ae6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.781+0000 7ff636800640 1 -- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 msgr2=0x564675b630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.781+0000 7ff636800640 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675b630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.781+0000 7ff636800640 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564675af9a40 con 0x564675ae8010 2024-06-13T07:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.782+0000 7ff636800640 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff630013650 tx=0x7ff63000fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617d2f0c3d4469b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.782+0000 7ff61be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.786+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63001a070 con 0x564675ae8010 2024-06-13T07:04:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.786+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff630018070 con 0x564675ae8010 2024-06-13T07:04:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.786+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff6300058a0 con 0x564675ae8010 2024-06-13T07:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.788+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff630044050 con 0x564675ae8010 2024-06-13T07:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.788+0000 7ff63ffff740 10 client.16050 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.788+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564675b64590 con 0x564675ae8010 2024-06-13T07:04:22.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.788+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff630018070 con 0x564675ae8010 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.790+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7ff63001ead0 con 0x564675ae8010 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.790+0000 7ff63ffff740 10 client.16050 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.790+0000 7ff63ffff740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.790+0000 7ff63ffff740 20 client.16050 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.790+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675b64e40 0x564675b67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.791+0000 7ff63ffff740 4 client.16050 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.791+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564675a4eea0 con 0x564675b64e40 2024-06-13T07:04:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.791+0000 7ff635e00640 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675b64e40 0x564675b67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.792+0000 7ff635e00640 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675b64e40 0x564675b67260 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b1d5a193bd2e36 server_cookie=f288ed1aacece413 in_seq=0 out_seq=0 2024-06-13T07:04:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.792+0000 7ff61be00640 10 client.16050 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 500.0 s, 473.9 tps, lat 214.748 ms stddev 320.579 2024-06-13T07:04:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.898+0000 7ff61be00640 1 -- 172.21.15.7:0/3957950653 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564675a4eea0 con 0x564675b64e40 2024-06-13T07:04:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.898+0000 7ff61be00640 10 client.16050 handle_command_reply: tid=0 2024-06-13T07:04:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.898+0000 7ff63ffff740 1 client.16050 shutdown 2024-06-13T07:04:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 msgr2=0x564675ae69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff630013650 tx=0x7ff63000fea0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 shutdown_connections 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564675b64e40 0x564675b67260 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564675aedb70 0x564675b630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564675ae8a00 0x564675ae6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 --2- 172.21.15.7:0/3957950653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564675ae8010 0x564675ae69d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 >> 172.21.15.7:0/3957950653 conn(0x564675aabfb0 msgr2=0x564675af1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff61be00640 0 client.16050 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 shutdown_connections 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.899+0000 7ff63ffff740 1 -- 172.21.15.7:0/3957950653 wait complete. 2024-06-13T07:04:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.900+0000 7ff63ffff740 20 client.16050 trim_cache size 0 max 16384 2024-06-13T07:04:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:04:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:04:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:04:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 msgr2=0x7ff61c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 0x7ff61c0a5260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff6200bb920 tx=0x7ff620043000 comp rx=0 tx=0).stop 2024-06-13T07:04:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 msgr2=0x7ff6381a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff6381a09c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6200314e0 tx=0x7ff620032ce0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 shutdown_connections 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff61c0a2da0 0x7ff61c0a5260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff63810e500 0x7ff6381a7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63810ba80 0x7ff6381a09c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 --2- 172.21.15.7:0/3410192653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806a7d0 0x7ff6381a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 >> 172.21.15.7:0/3410192653 conn(0x7ff63806ebd0 msgr2=0x7ff6381099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.906+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 shutdown_connections 2024-06-13T07:04:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:22.907+0000 7ff63d200640 1 -- 172.21.15.7:0/3410192653 wait complete. 2024-06-13T07:04:23.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:23 smithi175 ceph-mon[36886]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:23.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:23 smithi007 ceph-mon[32077]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:23 smithi189 ceph-mon[36616]: pgmap v759: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-06-13T07:04:24.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:04:25.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:25 smithi175 ceph-mon[36886]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-06-13T07:04:25.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:25 smithi007 ceph-mon[32077]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-06-13T07:04:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:25 smithi189 ceph-mon[36616]: pgmap v760: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-06-13T07:04:25.644 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:27.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:27 smithi007 ceph-mon[32077]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-13T07:04:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:27 smithi189 ceph-mon[36616]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-13T07:04:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:27 smithi175 ceph-mon[36886]: pgmap v761: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-06-13T07:04:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 505.0 s, 429.2 tps, lat 239.317 ms stddev 279.307 2024-06-13T07:04:29.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:29 smithi007 ceph-mon[32077]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-06-13T07:04:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:04:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:04:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:04:29.580 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:04:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.061+0000 7fd043e00700 1 -- 172.21.15.7:0/759545923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04410ed90 msgr2=0x7fd044111180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.061+0000 7fd043e00700 1 --2- 172.21.15.7:0/759545923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04410ed90 0x7fd044111180 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b0a0 tx=0x7fd03c00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 -- 172.21.15.7:0/759545923 shutdown_connections 2024-06-13T07:04:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 --2- 172.21.15.7:0/759545923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd04410ed90 0x7fd044111180 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 --2- 172.21.15.7:0/759545923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0441011e0 0x7fd0441015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 --2- 172.21.15.7:0/759545923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 0x7fd04410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 -- 172.21.15.7:0/759545923 >> 172.21.15.7:0/759545923 conn(0x7fd0440fce40 msgr2=0x7fd0440ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 -- 172.21.15.7:0/759545923 shutdown_connections 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.062+0000 7fd043e00700 1 -- 172.21.15.7:0/759545923 wait complete. 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 Processor -- start 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 -- start start 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 0x7fd04419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04410ed90 0x7fd04419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441149c0 con 0x7fd0441011e0 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd044114840 con 0x7fd04410ed90 2024-06-13T07:04:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd043e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd044114b40 con 0x7fd044101b90 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.063+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37974/0 (socket says 172.21.15.7:37974) 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 -- 172.21.15.7:0/674923366 learned_addr learned my addr 172.21.15.7:0/674923366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042000700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 0x7fd04419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd043400700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04410ed90 0x7fd04419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 -- 172.21.15.7:0/674923366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 msgr2=0x7fd04419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 0x7fd04419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 -- 172.21.15.7:0/674923366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04410ed90 msgr2=0x7fd04419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04410ed90 0x7fd04419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 -- 172.21.15.7:0/674923366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04419fe80 con 0x7fd0441011e0 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd042a00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd03400eb30 tx=0x7fd03400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd03b400700 1 -- 172.21.15.7:0/674923366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03400cbd0 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd03b400700 1 -- 172.21.15.7:0/674923366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd034004500 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd03b400700 1 -- 172.21.15.7:0/674923366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034010430 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.064+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0441a0170 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.065+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441a79f0 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.069+0000 7fd03b400700 1 -- 172.21.15.7:0/674923366 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd034003f00 con 0x7fd0441011e0 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.070+0000 7fd05447ab80 1 Processor -- start 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.070+0000 7fd05447ab80 1 -- start start 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd03b400700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 0x7fd0280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfec4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfecf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5619dfdd1a10 con 0x5619dfecd570 2024-06-13T07:04:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619dfe14b00 con 0x5619dfec4960 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd05447ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5619dfdcf8f0 con 0x5619dfe3ee40 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd03b400700 1 -- 172.21.15.7:0/674923366 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd0340cf740 con 0x7fd0441011e0 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd043400700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 0x7fd0280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51840/0 (socket says 172.21.15.7:51840) 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 -- 172.21.15.7:0/3566028051 learned_addr learned my addr 172.21.15.7:0/3566028051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd043400700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfecf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042000700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfec4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 -- 172.21.15.7:0/3566028051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 msgr2=0x5619dfec4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfec4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 -- 172.21.15.7:0/3566028051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 msgr2=0x5619dfecf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfecf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042a00700 1 -- 172.21.15.7:0/3566028051 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dfdbcf90 con 0x5619dfe3ee40 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.071+0000 7fd042000700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfec4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.072+0000 7fd042a00700 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0340985d0 tx=0x7fd034097f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d4ee8cd3a28275 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.072+0000 7fd02fe00700 1 -- 172.21.15.7:0/3566028051 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03409b560 con 0x5619dfe3ee40 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.072+0000 7fd02fe00700 1 -- 172.21.15.7:0/3566028051 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03409c300 con 0x5619dfe3ee40 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.072+0000 7fd02fe00700 1 -- 172.21.15.7:0/3566028051 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034086910 con 0x5619dfe3ee40 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- 172.21.15.7:0/3566028051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 msgr2=0x5619dfec4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd0340985d0 tx=0x7fd034097f20 comp rx=0 tx=0).stop 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd043400700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 0x7fd0280a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b780 tx=0x7fd03c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- 172.21.15.7:0/3566028051 shutdown_connections 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfecf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfe3ee40 0x5619dfec4420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 --2- 172.21.15.7:0/3566028051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfec4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- 172.21.15.7:0/3566028051 >> 172.21.15.7:0/3566028051 conn(0x5619dfdef170 msgr2=0x5619dfebe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- 172.21.15.7:0/3566028051 shutdown_connections 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- 172.21.15.7:0/3566028051 wait complete. 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.073+0000 7fd05447ab80 1 -- start start 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 Processor -- start 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfeccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfeca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5619dfdb0f60 con 0x5619dfe3ee40 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5619dfe10ac0 con 0x5619dfec4960 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5619dfed0120 con 0x5619dfecd570 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd05447ab80 10 client.0 _ll_register_callbacks cb 0x5619dfdae720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfeccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd042000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd042000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57398/0 (socket says 172.21.15.7:57398) 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd042000700 1 -- 172.21.15.7:0/2454883319 learned_addr learned my addr 172.21.15.7:0/2454883319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfeccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37988/0 (socket says 172.21.15.7:37988) 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.074+0000 7fd043400700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfeca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 -- 172.21.15.7:0/2454883319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 msgr2=0x5619dfeca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfeca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 -- 172.21.15.7:0/2454883319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 msgr2=0x5619dfeccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfeccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 -- 172.21.15.7:0/2454883319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dfd638d0 con 0x5619dfec4960 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd042000700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0300116f0 tx=0x7fd030011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9745903352e87019 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd02cc00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03001f060 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03409c300 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.075+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd030004260 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd030004460 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 10 client.26621 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5619dfeda9a0 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03409c300 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd03001a8d0 con 0x5619dfec4960 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 10 client.26621 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 20 client.26621 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dfedcf30 0x5619dfedf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 4 client.26621 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619dfe22000 con 0x5619dfedcf30 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.076+0000 7fd042a00700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dfedcf30 0x5619dfedf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.077+0000 7fd042a00700 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dfedcf30 0x5619dfedf320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a8792702b4c2d3 server_cookie=ce853938dece19ce in_seq=0 out_seq=0 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.077+0000 7fd02cc00700 10 client.26621 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.077+0000 7fd02cc00700 1 -- 172.21.15.7:0/2454883319 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619dfe22000 con 0x5619dfedcf30 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.077+0000 7fd02cc00700 10 client.26621 handle_command_reply: tid=0 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 client.26621 shutdown 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 msgr2=0x5619dfeca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0300116f0 tx=0x7fd030011a00 comp rx=0 tx=0).stop 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 shutdown_connections 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dfedcf30 0x5619dfedf320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfeca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfeccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd02cc00700 0 client.26621 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2454883319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfeca090 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 >> 172.21.15.7:0/2454883319 conn(0x5619dfdef170 msgr2=0x5619dfebe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 shutdown_connections 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 1 -- 172.21.15.7:0/2454883319 wait complete. 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.078+0000 7fd05447ab80 20 client.26621 trim_cache size 0 max 16384 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.100+0000 7fd05447ab80 1 Processor -- start 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.100+0000 7fd05447ab80 1 -- start start 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.100+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfec4960 0x5619dfeca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfee4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd05447ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441b2c10 con 0x5619dfecd570 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd05447ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441b0470 con 0x5619dfe3ee40 2024-06-13T07:04:29.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd05447ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441ae2c0 con 0x5619dfec4960 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd043400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfee4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfec4960 0x5619dfeca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57400/0 (socket says 172.21.15.7:57400) 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 -- 172.21.15.7:0/2013033068 learned_addr learned my addr 172.21.15.7:0/2013033068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 -- 172.21.15.7:0/2013033068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfec4960 msgr2=0x5619dfeca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfec4960 0x5619dfeca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 -- 172.21.15.7:0/2013033068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 msgr2=0x5619dfee4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfee4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.101+0000 7fd042a00700 1 -- 172.21.15.7:0/2013033068 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619dfebfa20 con 0x5619dfe3ee40 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd042a00700 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0340044d0 tx=0x7fd034005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddcdea2cfc5e2c03 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd02f400700 1 -- 172.21.15.7:0/2013033068 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340b8030 con 0x5619dfe3ee40 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd02f400700 1 -- 172.21.15.7:0/2013033068 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03409bc00 con 0x5619dfe3ee40 2024-06-13T07:04:29.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd02f400700 1 -- 172.21.15.7:0/2013033068 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034099ca0 con 0x5619dfe3ee40 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 -- 172.21.15.7:0/2013033068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 msgr2=0x5619dfeca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0340044d0 tx=0x7fd034005bb0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 -- 172.21.15.7:0/2013033068 shutdown_connections 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfecd570 0x5619dfee4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfe3ee40 0x5619dfeca050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 --2- 172.21.15.7:0/2013033068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfec4960 0x5619dfeca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 -- 172.21.15.7:0/2013033068 >> 172.21.15.7:0/2013033068 conn(0x5619dfdef170 msgr2=0x5619dfec8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.102+0000 7fd05447ab80 1 -- 172.21.15.7:0/2013033068 shutdown_connections 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 -- 172.21.15.7:0/2013033068 wait complete. 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 -- start start 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 Processor -- start 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfed10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfed1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfed1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441ae930 con 0x5619dfe3ee40 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441afda0 con 0x5619dfec4960 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.103+0000 7fd05447ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441af000 con 0x5619dfecd570 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd05447ab80 10 client.0 _ll_register_callbacks cb 0x5619dfdae720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfed10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfed1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfed10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38010/0 (socket says 172.21.15.7:38010) 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042a00700 1 -- 172.21.15.7:0/862782266 learned_addr learned my addr 172.21.15.7:0/862782266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd043400700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfed1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 -- 172.21.15.7:0/862782266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 msgr2=0x5619dfed1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfed1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 -- 172.21.15.7:0/862782266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 msgr2=0x5619dfed10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfed10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 -- 172.21.15.7:0/862782266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619dff46c00 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd042000700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfed1600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd03001aee0 tx=0x7fd0300057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ab9e262b8a7605 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.104+0000 7fd02cc00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030030020 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03409bc00 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd030004260 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd03003b050 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 10 client.26627 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5619dff47e70 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03409bc00 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7fd030003b70 con 0x5619dfec4960 2024-06-13T07:04:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 10 client.26627 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 10 client.26627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 20 client.26627 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dff48570 0x5619dff4a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 4 client.26627 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.105+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619dfe1c580 con 0x5619dff48570 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd042a00700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dff48570 0x5619dff4a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd042a00700 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dff48570 0x5619dff4a990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d75ca4d2acffe0 server_cookie=d8c171cfa52bc4f4 in_seq=0 out_seq=0 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd02cc00700 10 client.26627 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd02cc00700 1 -- 172.21.15.7:0/862782266 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619dfe1c580 con 0x5619dff48570 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd02cc00700 10 client.26627 handle_command_reply: tid=0 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.106+0000 7fd05447ab80 1 client.26627 shutdown 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 msgr2=0x5619dfed1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfed1600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd03001aee0 tx=0x7fd0300057f0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 shutdown_connections 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5619dff48570 0x5619dff4a990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619dfecd570 0x5619dfed1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619dfe3ee40 0x5619dfed10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd02cc00700 0 client.26627 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 --2- 172.21.15.7:0/862782266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5619dfec4960 0x5619dfed1600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 >> 172.21.15.7:0/862782266 conn(0x5619dfdef170 msgr2=0x5619dfec8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 shutdown_connections 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 1 -- 172.21.15.7:0/862782266 wait complete. 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.107+0000 7fd05447ab80 20 client.26627 trim_cache size 0 max 16384 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 msgr2=0x7fd0280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 0x7fd0280a50e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b780 tx=0x7fd03c009360 comp rx=0 tx=0).stop 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 msgr2=0x7fd04419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd03400eb30 tx=0x7fd03400ee40 comp rx=0 tx=0).stop 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 shutdown_connections 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd0280a2c20 0x7fd0280a50e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04410ed90 0x7fd04419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441011e0 0x7fd04419ae70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 --2- 172.21.15.7:0/674923366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd044101b90 0x7fd04419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 >> 172.21.15.7:0/674923366 conn(0x7fd0440fce40 msgr2=0x7fd04410ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 shutdown_connections 2024-06-13T07:04:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:29.108+0000 7fd043e00700 1 -- 172.21.15.7:0/674923366 wait complete. 2024-06-13T07:04:29.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:29 smithi189 ceph-mon[36616]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-06-13T07:04:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:29 smithi175 ceph-mon[36886]: pgmap v762: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-06-13T07:04:31.364 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:04:31.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:04:31.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:31 smithi007 ceph-mon[32077]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T07:04:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:31 smithi189 ceph-mon[36616]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T07:04:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:31 smithi175 ceph-mon[36886]: pgmap v763: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 611 op/s 2024-06-13T07:04:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 510.0 s, 428.8 tps, lat 224.475 ms stddev 281.835 2024-06-13T07:04:33.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:33 smithi007 ceph-mon[32077]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:04:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:33 smithi189 ceph-mon[36616]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:04:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:33 smithi175 ceph-mon[36886]: pgmap v764: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:04:35.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:35 smithi007 ceph-mon[32077]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-06-13T07:04:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:35 smithi189 ceph-mon[36616]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-06-13T07:04:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:35 smithi175 ceph-mon[36886]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.904+0000 7f7f44a00700 1 -- 172.21.15.7:0/1475026350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 msgr2=0x7f7f40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.904+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1475026350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f40074aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009990 tx=0x7f7f30009ca0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 -- 172.21.15.7:0/1475026350 shutdown_connections 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1475026350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f4010ee90 0x7f7f40111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1475026350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f40074aa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1475026350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 -- 172.21.15.7:0/1475026350 >> 172.21.15.7:0/1475026350 conn(0x7f7f400795f0 msgr2=0x7f7f40079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 -- 172.21.15.7:0/1475026350 shutdown_connections 2024-06-13T07:04:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.905+0000 7f7f44a00700 1 -- 172.21.15.7:0/1475026350 wait complete. 2024-06-13T07:04:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.906+0000 7f7f44a00700 1 Processor -- start 2024-06-13T07:04:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 -- start start 2024-06-13T07:04:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f4010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 0x7f7f40105550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40104130 con 0x7f7f4010ee90 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40103fb0 con 0x7f7f40074fe0 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.907+0000 7f7f44a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f401042b0 con 0x7f7f400746c0 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.908+0000 7f7f3de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f4010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.908+0000 7f7f3d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.908+0000 7f7f3d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47462/0 (socket says 172.21.15.7:47462) 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.908+0000 7f7f3d400700 1 -- 172.21.15.7:0/1990079988 learned_addr learned my addr 172.21.15.7:0/1990079988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.908+0000 7f7f3e800700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 0x7f7f40105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:35.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 -- 172.21.15.7:0/1990079988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 msgr2=0x7f7f4010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:35.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f4010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 -- 172.21.15.7:0/1990079988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 msgr2=0x7f7f4010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:35.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 -- 172.21.15.7:0/1990079988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f40105e50 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f3e800700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 0x7f7f40105550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f38007ee0 tx=0x7f7f3800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004db0 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f38009e20 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38015400 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.909+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f40106140 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.910+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4010a000 con 0x7f7f4010ee90 2024-06-13T07:04:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.911+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f40075db0 con 0x7f7f4010ee90 2024-06-13T07:04:35.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.913+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7f3800db20 con 0x7f7f4010ee90 2024-06-13T07:04:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.914+0000 7f7f36a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 0x7f7f240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.915+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7f380cec60 con 0x7f7f4010ee90 2024-06-13T07:04:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.915+0000 7f7f3de00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 0x7f7f240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.918+0000 7f7f3de00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 0x7f7f240a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b5c0 tx=0x7f7f30005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:35.922+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f38010070 con 0x7f7f4010ee90 2024-06-13T07:04:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.139+0000 7f980c200640 1 -- 172.21.15.7:0/4274325994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 msgr2=0x7f9804074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.139+0000 7f980c200640 1 --2- 172.21.15.7:0/4274325994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f9804074880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009930 tx=0x7f97f002a440 comp rx=0 tx=0).stop 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.139+0000 7f980c200640 1 -- 172.21.15.7:0/4274325994 shutdown_connections 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.139+0000 7f980c200640 1 --2- 172.21.15.7:0/4274325994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 0x7f98041103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.140+0000 7f980c200640 1 --2- 172.21.15.7:0/4274325994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f9804074880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.140+0000 7f980c200640 1 --2- 172.21.15.7:0/4274325994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.140+0000 7f980c200640 1 -- 172.21.15.7:0/4274325994 >> 172.21.15.7:0/4274325994 conn(0x7f98040fcf70 msgr2=0x7f98040ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.140+0000 7f980c200640 1 -- 172.21.15.7:0/4274325994 shutdown_connections 2024-06-13T07:04:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.140+0000 7f980c200640 1 -- 172.21.15.7:0/4274325994 wait complete. 2024-06-13T07:04:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.141+0000 7f980c200640 1 Processor -- start 2024-06-13T07:04:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.142+0000 7f980c200640 1 -- start start 2024-06-13T07:04:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f98040702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 0x7f98041ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113d70 con 0x7f980410dfd0 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113bf0 con 0x7f9804074400 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f980c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9804113ef0 con 0x7f98040759e0 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f9809800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f98040702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f9809800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f98040702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47474/0 (socket says 172.21.15.7:47474) 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.143+0000 7f9809800640 1 -- 172.21.15.7:0/3283278790 learned_addr learned my addr 172.21.15.7:0/3283278790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.144+0000 7f980a200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 0x7f98041ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.144+0000 7f9808e00640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 -- 172.21.15.7:0/3283278790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 msgr2=0x7f98040702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f98040702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 -- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 msgr2=0x7f98041ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 0x7f98041ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 -- 172.21.15.7:0/3283278790 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9804073950 con 0x7f98040759e0 2024-06-13T07:04:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.145+0000 7f9808e00640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804070820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97f0031400 tx=0x7f97f0004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.146+0000 7f9802000640 1 -- 172.21.15.7:0/3283278790 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0040070 con 0x7f98040759e0 2024-06-13T07:04:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.146+0000 7f9802000640 1 -- 172.21.15.7:0/3283278790 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f0031c80 con 0x7f98040759e0 2024-06-13T07:04:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.146+0000 7f9802000640 1 -- 172.21.15.7:0/3283278790 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0035020 con 0x7f98040759e0 2024-06-13T07:04:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.147+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98041aea20 con 0x7f98040759e0 2024-06-13T07:04:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.147+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98041aef10 con 0x7f98040759e0 2024-06-13T07:04:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.158+0000 7f980f04e740 1 Processor -- start 2024-06-13T07:04:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.158+0000 7f9802000640 1 -- 172.21.15.7:0/3283278790 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f97f0043070 con 0x7f98040759e0 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.158+0000 7f980f04e740 1 -- start start 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a34020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 0x561da5a3bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561da5997420 con 0x561da5a39760 2024-06-13T07:04:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561da593ec50 con 0x561da5a345f0 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980f04e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561da59933e0 con 0x561da5a33c00 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f9809800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f9809800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47486/0 (socket says 172.21.15.7:47486) 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f9809800640 1 -- 172.21.15.7:0/441015991 learned_addr learned my addr 172.21.15.7:0/441015991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.159+0000 7f980a200640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a34020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.160+0000 7f9809800640 1 -- 172.21.15.7:0/441015991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 msgr2=0x561da5a34020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.160+0000 7f9809800640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a34020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.160+0000 7f9809800640 1 -- 172.21.15.7:0/441015991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 msgr2=0x561da5a3bbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:04:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.160+0000 7f9808e00640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 0x561da5a3bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.161+0000 7f9802000640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 0x7f97ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.162+0000 7f9809800640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 0x561da5a3bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.162+0000 7f9809800640 1 -- 172.21.15.7:0/441015991 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da56ab210 con 0x561da5a345f0 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.162+0000 7f9802000640 1 -- 172.21.15.7:0/3283278790 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f97f00f0330 con 0x7f98040759e0 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.162+0000 7f9809800640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97f8007840 tx=0x7f97f8007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1223f3d1a45311b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.162+0000 7f9808e00640 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 0x561da5a3bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.163+0000 7f97f6a00640 1 -- 172.21.15.7:0/441015991 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f800d950 con 0x561da5a345f0 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.163+0000 7f97f6a00640 1 -- 172.21.15.7:0/441015991 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8018070 con 0x561da5a345f0 2024-06-13T07:04:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.163+0000 7f97f6a00640 1 -- 172.21.15.7:0/441015991 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f80123e0 con 0x561da5a345f0 2024-06-13T07:04:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.164+0000 7f980a200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 0x7f97ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.164+0000 7f980f04e740 1 -- 172.21.15.7:0/441015991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 msgr2=0x561da5a390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.166+0000 7f980a200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 0x7f97ec0a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x561da5a34e50 tx=0x7f97fc00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.166+0000 7f980f04e740 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97f8007840 tx=0x7f97f8007d00 comp rx=0 tx=0).stop 2024-06-13T07:04:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.167+0000 7f980f04e740 1 -- 172.21.15.7:0/441015991 shutdown_connections 2024-06-13T07:04:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.167+0000 7f980f04e740 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a39760 0x561da5a3bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.167+0000 7f980f04e740 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a345f0 0x561da5a390f0 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97f8007840 tx=0x7f97f8007d00 comp rx=0 tx=0).stop 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.167+0000 7f980f04e740 1 --2- 172.21.15.7:0/441015991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a34020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.167+0000 7f980f04e740 1 -- 172.21.15.7:0/441015991 >> 172.21.15.7:0/441015991 conn(0x561da59fbfb0 msgr2=0x561da5a2f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.168+0000 7f980f04e740 1 -- 172.21.15.7:0/441015991 shutdown_connections 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.168+0000 7f980f04e740 1 -- 172.21.15.7:0/441015991 wait complete. 2024-06-13T07:04:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.169+0000 7f980f04e740 1 -- start start 2024-06-13T07:04:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.170+0000 7f980f04e740 1 Processor -- start 2024-06-13T07:04:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a3e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 0x561da5a3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 0x561da5a45b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561da5a3c710 con 0x561da5a33c00 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561da59520b0 con 0x561da5a43710 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561da5a3c890 con 0x561da5a39760 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.171+0000 7f980f04e740 10 client.0 _ll_register_callbacks cb 0x561da593df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.172+0000 7f980a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 0x561da5a45b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.172+0000 7f980a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 0x561da5a45b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47490/0 (socket says 172.21.15.7:47490) 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.172+0000 7f980a200640 1 -- 172.21.15.7:0/2565577806 learned_addr learned my addr 172.21.15.7:0/2565577806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.172+0000 7f9808e00640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 0x561da5a3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 -- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 msgr2=0x561da5a45b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 0x561da5a45b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 -- 172.21.15.7:0/2565577806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 msgr2=0x561da5a3e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a3e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 -- 172.21.15.7:0/2565577806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da5777620 con 0x561da5a39760 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f9808e00640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 0x561da5a3e750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f97f00edbe0 tx=0x7f97f00038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37bf66eb0f27c53 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.173+0000 7f97eb400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.174+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00bc250 con 0x561da5a39760 2024-06-13T07:04:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.174+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f97f00035f0 con 0x561da5a39760 2024-06-13T07:04:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.174+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f8018070 con 0x561da5a39760 2024-06-13T07:04:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.178+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97f00baa30 con 0x561da5a39760 2024-06-13T07:04:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.181+0000 7f980f04e740 10 client.26587 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.181+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561da5a3c710 con 0x561da5a39760 2024-06-13T07:04:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.181+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8018070 con 0x561da5a39760 2024-06-13T07:04:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.182+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f97f004d060 con 0x561da5a39760 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.182+0000 7f980f04e740 10 client.26587 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.182+0000 7f980f04e740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.183+0000 7f980f04e740 20 client.26587 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.183+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5a5b3f0 0x561da5a5d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.183+0000 7f980f04e740 4 client.26587 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.183+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561da59a4b50 con 0x561da5a5b3f0 2024-06-13T07:04:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.183+0000 7f9809800640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5a5b3f0 0x561da5a5d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.184+0000 7f9809800640 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5a5b3f0 0x561da5a5d820 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e55dbb7f445059 server_cookie=2bdb0671419c5ff2 in_seq=0 out_seq=0 2024-06-13T07:04:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.184+0000 7f97eb400640 10 client.26587 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.185+0000 7f97eb400640 1 -- 172.21.15.7:0/2565577806 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561da59a4b50 con 0x561da5a5b3f0 2024-06-13T07:04:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.185+0000 7f97eb400640 10 client.26587 handle_command_reply: tid=0 2024-06-13T07:04:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.185+0000 7f980f04e740 1 client.26587 shutdown 2024-06-13T07:04:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 msgr2=0x561da5a3e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 0x561da5a3e750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f97f00edbe0 tx=0x7f97f00038b0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 shutdown_connections 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5a5b3f0 0x561da5a5d820 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a43710 0x561da5a45b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a39760 0x561da5a3e750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 --2- 172.21.15.7:0/2565577806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a3e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 >> 172.21.15.7:0/2565577806 conn(0x561da59fbfb0 msgr2=0x561da5a2e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f97eb400640 0 client.26587 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 shutdown_connections 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.186+0000 7f980f04e740 1 -- 172.21.15.7:0/2565577806 wait complete. 2024-06-13T07:04:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.187+0000 7f980f04e740 20 client.26587 trim_cache size 0 max 16384 2024-06-13T07:04:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.225+0000 7f980f04e740 1 Processor -- start 2024-06-13T07:04:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.227+0000 7f980f04e740 1 -- start start 2024-06-13T07:04:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.227+0000 7f980f04e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a35b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.227+0000 7f980f04e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 0x561da5a36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.227+0000 7f980f04e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980f04e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98041b3350 con 0x561da5a33c00 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980f04e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98041b0bf0 con 0x561da5a2f160 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980f04e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f980410d860 con 0x561da5a43710 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47496/0 (socket says 172.21.15.7:47496) 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f980a200640 1 -- 172.21.15.7:0/971179047 learned_addr learned my addr 172.21.15.7:0/971179047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.228+0000 7f9808e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 0x561da5a36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.229+0000 7f9809800640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a35b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.229+0000 7f980a200640 1 -- 172.21.15.7:0/971179047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 msgr2=0x561da5a36070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.229+0000 7f980a200640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 0x561da5a36070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.229+0000 7f980a200640 1 -- 172.21.15.7:0/971179047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 msgr2=0x561da5a35b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.229+0000 7f980a200640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a35b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.230+0000 7f980a200640 1 -- 172.21.15.7:0/971179047 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da5a2b020 con 0x561da5a2f160 2024-06-13T07:04:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.231+0000 7f9809800640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a35b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.231+0000 7f9808e00640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 0x561da5a36070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.231+0000 7f980a200640 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97fc018520 tx=0x7f97fc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b52ccc644ba2927 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.233+0000 7f97f6000640 1 -- 172.21.15.7:0/971179047 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0166a0 con 0x561da5a2f160 2024-06-13T07:04:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.234+0000 7f97f6000640 1 -- 172.21.15.7:0/971179047 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97fc029070 con 0x561da5a2f160 2024-06-13T07:04:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.234+0000 7f97f6000640 1 -- 172.21.15.7:0/971179047 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc024d20 con 0x561da5a2f160 2024-06-13T07:04:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 -- 172.21.15.7:0/971179047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 msgr2=0x561da5a2f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97fc018520 tx=0x7f97fc017000 comp rx=0 tx=0).stop 2024-06-13T07:04:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 -- 172.21.15.7:0/971179047 shutdown_connections 2024-06-13T07:04:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a2f5c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a43710 0x561da5a36070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 --2- 172.21.15.7:0/971179047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a33c00 0x561da5a35b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.235+0000 7f980f04e740 1 -- 172.21.15.7:0/971179047 >> 172.21.15.7:0/971179047 conn(0x561da59fbfb0 msgr2=0x561da5a3cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.237+0000 7f980f04e740 1 -- 172.21.15.7:0/971179047 shutdown_connections 2024-06-13T07:04:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.237+0000 7f980f04e740 1 -- 172.21.15.7:0/971179047 wait complete. 2024-06-13T07:04:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.238+0000 7f980f04e740 1 -- start start 2024-06-13T07:04:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.238+0000 7f980f04e740 1 Processor -- start 2024-06-13T07:04:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.239+0000 7f980f04e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a6cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.239+0000 7f980f04e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.239+0000 7f980f04e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 0x561da5a6e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f9809800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a6cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f980f04e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98041af1a0 con 0x561da5a43710 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f980f04e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98041b0520 con 0x561da5a2f160 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f9808e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f9808e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:58690/0 (socket says 172.21.15.7:58690) 2024-06-13T07:04:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f9808e00640 1 -- 172.21.15.7:0/443003705 learned_addr learned my addr 172.21.15.7:0/443003705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98041af780 con 0x561da5a33c00 2024-06-13T07:04:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f980f04e740 10 client.0 _ll_register_callbacks cb 0x7f97f0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.240+0000 7f980a200640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 0x561da5a6e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.241+0000 7f9809800640 1 -- 172.21.15.7:0/443003705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 msgr2=0x561da5a6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f9809800640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f9809800640 1 -- 172.21.15.7:0/443003705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 msgr2=0x561da5a6e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f9809800640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 0x561da5a6e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f9809800640 1 -- 172.21.15.7:0/443003705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da5a6e6d0 con 0x561da5a2f160 2024-06-13T07:04:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f9808e00640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.242+0000 7f980a200640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 0x561da5a6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.243+0000 7f9809800640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a6cb90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x561da5a34ab0 tx=0x7f97f8006f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31afb02843ee5b79 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.243+0000 7f97eb400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.245+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f800dd60 con 0x561da5a2f160 2024-06-13T07:04:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.245+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f97f80135a0 con 0x561da5a2f160 2024-06-13T07:04:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.245+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97fc029070 con 0x561da5a2f160 2024-06-13T07:04:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.245+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97f802e050 con 0x561da5a2f160 2024-06-13T07:04:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.246+0000 7f980f04e740 10 client.26651 fetch_fsmap learned FSMap version 37 2024-06-13T07:04:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.246+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561da5a6f9c0 con 0x561da5a2f160 2024-06-13T07:04:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.246+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97fc029070 con 0x561da5a2f160 2024-06-13T07:04:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.246+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f97f8003690 con 0x561da5a2f160 2024-06-13T07:04:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 10 client.26651 fetch_fsmap finished waiting for FSMap version 37 2024-06-13T07:04:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 20 client.26651 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5ac62c0 0x561da5ac86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 4 client.26651 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561da599eea0 con 0x561da5ac62c0 2024-06-13T07:04:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.247+0000 7f9808e00640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5ac62c0 0x561da5ac86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.248+0000 7f9808e00640 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5ac62c0 0x561da5ac86b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcba5dd46c0a992 server_cookie=d19e41cd136e53c3 in_seq=0 out_seq=0 2024-06-13T07:04:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.248+0000 7f97eb400640 10 client.26651 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.309+0000 7f97eb400640 1 -- 172.21.15.7:0/443003705 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561da599eea0 con 0x561da5ac62c0 2024-06-13T07:04:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f97eb400640 10 client.26651 handle_command_reply: tid=0 2024-06-13T07:04:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 client.26651 shutdown 2024-06-13T07:04:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 msgr2=0x561da5a6cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a6cb90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x561da5a34ab0 tx=0x7f97f8006f30 comp rx=0 tx=0).stop 2024-06-13T07:04:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 shutdown_connections 2024-06-13T07:04:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561da5ac62c0 0x561da5ac86b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561da5a43710 0x561da5a6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561da5a33c00 0x561da5a6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 --2- 172.21.15.7:0/443003705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561da5a2f160 0x561da5a6cb90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 >> 172.21.15.7:0/443003705 conn(0x561da59fbfb0 msgr2=0x561da5a35b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.310+0000 7f97eb400640 0 client.26651 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.311+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 shutdown_connections 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.311+0000 7f980f04e740 1 -- 172.21.15.7:0/443003705 wait complete. 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.311+0000 7f980f04e740 20 client.26651 trim_cache size 0 max 16384 2024-06-13T07:04:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:04:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:04:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:04:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 msgr2=0x7f97ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 0x7f97ec0a5170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x561da5a34e50 tx=0x7f97fc00d040 comp rx=0 tx=0).stop 2024-06-13T07:04:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 msgr2=0x7f9804070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804070820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97f0031400 tx=0x7f97f0004b80 comp rx=0 tx=0).stop 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 shutdown_connections 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f97ec0a2cb0 0x7f97ec0a5170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f980410dfd0 0x7f98041ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98040759e0 0x7f9804070820 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 --2- 172.21.15.7:0/3283278790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9804074400 0x7f98040702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.312+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 >> 172.21.15.7:0/3283278790 conn(0x7f98040fcf70 msgr2=0x7f9804101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.313+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 shutdown_connections 2024-06-13T07:04:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:36.313+0000 7f980c200640 1 -- 172.21.15.7:0/3283278790 wait complete. 2024-06-13T07:04:36.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.377+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f4010a3c0 con 0x7f7f4010ee90 2024-06-13T07:04:36.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.378+0000 7f7f36a00700 1 -- 172.21.15.7:0/1990079988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11869 (secure 0 0 0) 0x7f7f3809b820 con 0x7f7f4010ee90 2024-06-13T07:04:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:04:36.378 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:02:25.094486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T07:04:36.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 msgr2=0x7f7f240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 0x7f7f240a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b5c0 tx=0x7f7f30005fd0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 msgr2=0x7f7f40105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 0x7f7f40105550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7f38007ee0 tx=0x7f7f3800d3b0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 shutdown_connections 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4010ee90 0x7f7f40105550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f400746c0 0x7f7f4010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7f240a2ca0 0x7f7f240a5160 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 --2- 172.21.15.7:0/1990079988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f40074fe0 0x7f7f4010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.381+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 >> 172.21.15.7:0/1990079988 conn(0x7f7f400795f0 msgr2=0x7f7f400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:36.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.382+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 shutdown_connections 2024-06-13T07:04:36.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:36.382+0000 7f7f44a00700 1 -- 172.21.15.7:0/1990079988 wait complete. 2024-06-13T07:04:36.382 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-06-13T07:04:37.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:04:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:37 smithi189 ceph-mon[36616]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-06-13T07:04:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:37 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1990079988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:04:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:37 smithi175 ceph-mon[36886]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-06-13T07:04:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:37 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1990079988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:04:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:37 smithi007 ceph-mon[32077]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 428 op/s 2024-06-13T07:04:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:37 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1990079988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:04:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 515.0 s, 480.8 tps, lat 212.748 ms stddev 229.059 2024-06-13T07:04:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:39 smithi189 ceph-mon[36616]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-06-13T07:04:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:39 smithi175 ceph-mon[36886]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-06-13T07:04:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:39 smithi007 ceph-mon[32077]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-06-13T07:04:41.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:41 smithi007 ceph-mon[32077]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-06-13T07:04:41.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:41 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:41 smithi189 ceph-mon[36616]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-06-13T07:04:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:41 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:41 smithi175 ceph-mon[36886]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 568 op/s 2024-06-13T07:04:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:41 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:04:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 -- 172.21.15.7:0/3199413758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec108500 msgr2=0x7fefec1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 --2- 172.21.15.7:0/3199413758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec108500 0x7fefec1088e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009f90 tx=0x7fefdc00b330 comp rx=0 tx=0).stop 2024-06-13T07:04:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 -- 172.21.15.7:0/3199413758 shutdown_connections 2024-06-13T07:04:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 --2- 172.21.15.7:0/3199413758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06ab30 0x7fefec10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 --2- 172.21.15.7:0/3199413758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec108500 0x7fefec1088e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 --2- 172.21.15.7:0/3199413758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec06a170 0x7fefec06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.388+0000 7feff0c00700 1 -- 172.21.15.7:0/3199413758 >> 172.21.15.7:0/3199413758 conn(0x7fefec078340 msgr2=0x7fefec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.389+0000 7feff0c00700 1 -- 172.21.15.7:0/3199413758 shutdown_connections 2024-06-13T07:04:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.389+0000 7feff0c00700 1 -- 172.21.15.7:0/3199413758 wait complete. 2024-06-13T07:04:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.391+0000 7feff0c00700 1 Processor -- start 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.391+0000 7feff0c00700 1 -- start start 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec06ab30 0x7fefec19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec108500 0x7fefec19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112800 con 0x7fefec108500 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112680 con 0x7fefec06ab30 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7feff0c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112980 con 0x7fefec06a170 2024-06-13T07:04:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7fefe9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7fefe9e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54050/0 (socket says 172.21.15.7:54050) 2024-06-13T07:04:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7fefe9e00700 1 -- 172.21.15.7:0/294534951 learned_addr learned my addr 172.21.15.7:0/294534951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.392+0000 7fefea800700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec108500 0x7fefec19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe3e00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec06ab30 0x7fefec19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe9e00700 1 -- 172.21.15.7:0/294534951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec06ab30 msgr2=0x7fefec19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe9e00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec06ab30 0x7fefec19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe9e00700 1 -- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec108500 msgr2=0x7fefec19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe9e00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec108500 0x7fefec19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.393+0000 7fefe9e00700 1 -- 172.21.15.7:0/294534951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec19fe00 con 0x7fefec06a170 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.394+0000 7fefe9e00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fefdc004fc0 tx=0x7fefdc00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.394+0000 7fefe2a00700 1 -- 172.21.15.7:0/294534951 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc020070 con 0x7fefec06a170 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.394+0000 7fefe2a00700 1 -- 172.21.15.7:0/294534951 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc007dd0 con 0x7fefec06a170 2024-06-13T07:04:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.394+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec1a0090 con 0x7fefec06a170 2024-06-13T07:04:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.395+0000 7fefe2a00700 1 -- 172.21.15.7:0/294534951 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc01c6a0 con 0x7fefec06a170 2024-06-13T07:04:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.395+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec109260 con 0x7fefec06a170 2024-06-13T07:04:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.402+0000 7fefe2a00700 1 -- 172.21.15.7:0/294534951 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fefdc01a040 con 0x7fefec06a170 2024-06-13T07:04:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.404+0000 7feffb646b80 1 Processor -- start 2024-06-13T07:04:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.404+0000 7feffb646b80 1 -- start start 2024-06-13T07:04:42.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.405+0000 7fefe2a00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 0x7fefd00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7fefe2a00700 1 -- 172.21.15.7:0/294534951 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fefdc00f070 con 0x7fefec06a170 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7fefea800700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 0x7fefd00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 0x55ccacf79f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccace82a10 con 0x55ccacf74240 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccacec5b00 con 0x55ccacf79b90 2024-06-13T07:04:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.406+0000 7feffb646b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ccace808f0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 0x55ccacf79f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe9e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 0x55ccacf79f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58392/0 (socket says 172.21.15.7:58392) 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe9e00700 1 -- 172.21.15.7:0/970842395 learned_addr learned my addr 172.21.15.7:0/970842395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 -- 172.21.15.7:0/970842395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 msgr2=0x55ccacf79f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 0x55ccacf79f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 -- 172.21.15.7:0/970842395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 msgr2=0x55ccacf74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.407+0000 7fefe3e00700 1 -- 172.21.15.7:0/970842395 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccace6df90 con 0x55ccacf7a4b0 2024-06-13T07:04:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.408+0000 7fefe3e00700 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf73d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fefd4005fd0 tx=0x7fefd4011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8200b9ff7365919a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.408+0000 7fefdb400700 1 -- 172.21.15.7:0/970842395 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011d70 con 0x55ccacf7a4b0 2024-06-13T07:04:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.409+0000 7fefdb400700 1 -- 172.21.15.7:0/970842395 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4004510 con 0x55ccacf7a4b0 2024-06-13T07:04:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.409+0000 7fefdb400700 1 -- 172.21.15.7:0/970842395 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40053a0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 -- 172.21.15.7:0/970842395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 msgr2=0x55ccacf73d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf73d00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fefd4005fd0 tx=0x7fefd4011440 comp rx=0 tx=0).stop 2024-06-13T07:04:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7fefea800700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 0x7fefd00a5220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009c40 tx=0x7fefe40103d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 -- 172.21.15.7:0/970842395 shutdown_connections 2024-06-13T07:04:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf79b90 0x55ccacf79f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 --2- 172.21.15.7:0/970842395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf73d00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 -- 172.21.15.7:0/970842395 >> 172.21.15.7:0/970842395 conn(0x55ccacea0170 msgr2=0x55ccacf6eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 -- 172.21.15.7:0/970842395 shutdown_connections 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.410+0000 7feffb646b80 1 -- 172.21.15.7:0/970842395 wait complete. 2024-06-13T07:04:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.411+0000 7feffb646b80 1 -- start start 2024-06-13T07:04:42.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.411+0000 7feffb646b80 1 Processor -- start 2024-06-13T07:04:42.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf74240 0x55ccacf80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccace61f60 con 0x55ccacf79b90 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccacec1ac0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ccacf80e90 con 0x55ccacf74240 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.412+0000 7feffb646b80 10 client.0 _ll_register_callbacks cb 0x55ccace5b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.413+0000 7fefea800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.413+0000 7fefea800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58408/0 (socket says 172.21.15.7:58408) 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.413+0000 7fefea800700 1 -- 172.21.15.7:0/2450151675 learned_addr learned my addr 172.21.15.7:0/2450151675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.413+0000 7fefe9e00700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf74240 0x55ccacf80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.413+0000 7fefe3e00700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.414+0000 7fefea800700 1 -- 172.21.15.7:0/2450151675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf74240 msgr2=0x55ccacf80200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.414+0000 7fefea800700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf74240 0x55ccacf80200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.414+0000 7fefea800700 1 -- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 msgr2=0x55ccacf7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.414+0000 7fefea800700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.414+0000 7fefea800700 1 -- 172.21.15.7:0/2450151675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccace148d0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.415+0000 7fefea800700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009650 tx=0x7fefe40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d742eeff79bc7d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.415+0000 7fefcfe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.416+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4004460 con 0x55ccacf7a4b0 2024-06-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.416+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fefe4003680 con 0x55ccacf7a4b0 2024-06-13T07:04:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.416+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd4004510 con 0x55ccacf7a4b0 2024-06-13T07:04:42.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.417+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefe402e050 con 0x55ccacf7a4b0 2024-06-13T07:04:42.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.417+0000 7feffb646b80 10 client.26663 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:42.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.417+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ccacf8b680 con 0x55ccacf7a4b0 2024-06-13T07:04:42.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4004510 con 0x55ccacf7a4b0 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fefe4020a20 con 0x55ccacf7a4b0 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 10 client.26663 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 10 client.26663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 20 client.26663 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacf8dd10 0x55ccacf90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 4 client.26663 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.418+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccaced3000 con 0x55ccacf8dd10 2024-06-13T07:04:42.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.419+0000 7fefe9e00700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacf8dd10 0x55ccacf90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.419+0000 7fefe9e00700 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacf8dd10 0x55ccacf90140 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b7e1198d574bbdb server_cookie=26aa18f3b7f07bd8 in_seq=0 out_seq=0 2024-06-13T07:04:42.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.419+0000 7fefcfe00700 10 client.26663 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:04:42.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.420+0000 7fefcfe00700 1 -- 172.21.15.7:0/2450151675 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccaced3000 con 0x55ccacf8dd10 2024-06-13T07:04:42.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.420+0000 7fefcfe00700 10 client.26663 handle_command_reply: tid=0 2024-06-13T07:04:42.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.420+0000 7feffb646b80 1 client.26663 shutdown 2024-06-13T07:04:42.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 msgr2=0x55ccacf7da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009650 tx=0x7fefe40047c0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 shutdown_connections 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacf8dd10 0x55ccacf90140 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf7da40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf74240 0x55ccacf80200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 --2- 172.21.15.7:0/2450151675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 >> 172.21.15.7:0/2450151675 conn(0x55ccacea0170 msgr2=0x55ccacf68790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7fefcfe00700 0 client.26663 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 shutdown_connections 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 1 -- 172.21.15.7:0/2450151675 wait complete. 2024-06-13T07:04:42.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.421+0000 7feffb646b80 20 client.26663 trim_cache size 0 max 16384 2024-06-13T07:04:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.444+0000 7feffb646b80 1 Processor -- start 2024-06-13T07:04:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.444+0000 7feffb646b80 1 -- start start 2024-06-13T07:04:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.444+0000 7feffb646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 0x55ccacf73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.444+0000 7feffb646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf6f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.444+0000 7feffb646b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7feffb646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fefc80112e0 con 0x55ccacf79b90 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7feffb646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefc800eb70 con 0x55ccacf74240 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7feffb646b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fefc800c9c0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf6f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf6f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59352/0 (socket says 172.21.15.7:59352) 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe3e00700 1 -- 172.21.15.7:0/1177378045 learned_addr learned my addr 172.21.15.7:0/1177378045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 0x55ccacf73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefea800700 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe9e00700 1 -- 172.21.15.7:0/1177378045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 msgr2=0x55ccacf69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe9e00700 1 -- 172.21.15.7:0/1177378045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 msgr2=0x55ccacf6f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.445+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf6f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.446+0000 7fefe9e00700 1 -- 172.21.15.7:0/1177378045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccacf7e8a0 con 0x55ccacf74240 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.446+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 0x55ccacf73d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0987f0 tx=0x7fefdc0ccf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863eebe800ea3e4f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.446+0000 7fefdaa00700 1 -- 172.21.15.7:0/1177378045 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0b9070 con 0x55ccacf74240 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.446+0000 7fefdaa00700 1 -- 172.21.15.7:0/1177378045 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc09be20 con 0x55ccacf74240 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.446+0000 7fefdaa00700 1 -- 172.21.15.7:0/1177378045 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc029040 con 0x55ccacf74240 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 -- 172.21.15.7:0/1177378045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 msgr2=0x55ccacf73d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 0x55ccacf73d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0987f0 tx=0x7fefdc0ccf00 comp rx=0 tx=0).stop 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 -- 172.21.15.7:0/1177378045 shutdown_connections 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf7a4b0 0x55ccacf69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf74240 0x55ccacf73d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 --2- 172.21.15.7:0/1177378045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf79b90 0x55ccacf6f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.447+0000 7feffb646b80 1 -- 172.21.15.7:0/1177378045 >> 172.21.15.7:0/1177378045 conn(0x55ccacea0170 msgr2=0x55ccacf6ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.449+0000 7feffb646b80 1 -- 172.21.15.7:0/1177378045 shutdown_connections 2024-06-13T07:04:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.449+0000 7feffb646b80 1 -- 172.21.15.7:0/1177378045 wait complete. 2024-06-13T07:04:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.449+0000 7feffb646b80 1 -- start start 2024-06-13T07:04:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.450+0000 7feffb646b80 1 Processor -- start 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf79b90 0x55ccacf866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fefc800d030 con 0x55ccacf74240 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fefc800e4a0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fefc800d700 con 0x55ccacf79b90 2024-06-13T07:04:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7feffb646b80 10 client.0 _ll_register_callbacks cb 0x55ccace5b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7fefea800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7fefea800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58428/0 (socket says 172.21.15.7:58428) 2024-06-13T07:04:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.451+0000 7fefea800700 1 -- 172.21.15.7:0/1034078035 learned_addr learned my addr 172.21.15.7:0/1034078035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 -- 172.21.15.7:0/1034078035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf79b90 msgr2=0x55ccacf866f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf79b90 0x55ccacf866f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 -- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 msgr2=0x55ccacf861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 -- 172.21.15.7:0/1034078035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccacf86de0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefea800700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009620 tx=0x7fefe40045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950faa6b1abe685f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.452+0000 7fefcfe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4020860 con 0x55ccacf7a4b0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefdc09be20 con 0x55ccacf7a4b0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fefe4020a00 con 0x55ccacf7a4b0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefe4046050 con 0x55ccacf7a4b0 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7feffb646b80 10 client.26669 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.453+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ccacff7ff0 con 0x55ccacf7a4b0 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc09be20 con 0x55ccacf7a4b0 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fefe403d480 con 0x55ccacf7a4b0 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 10 client.26669 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 10 client.26669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 20 client.26669 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacff86a0 0x55ccacffaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 4 client.26669 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:04:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.454+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccacecd580 con 0x55ccacff86a0 2024-06-13T07:04:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.455+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacff86a0 0x55ccacffaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.458+0000 7fefe9e00700 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacff86a0 0x55ccacffaac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aca088c90c69235d server_cookie=72f1fb5a8d93b5cf in_seq=0 out_seq=0 2024-06-13T07:04:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.458+0000 7fefcfe00700 10 client.26669 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:04:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7fefcfe00700 1 -- 172.21.15.7:0/1034078035 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccacecd580 con 0x55ccacff86a0 2024-06-13T07:04:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7fefcfe00700 10 client.26669 handle_command_reply: tid=0 2024-06-13T07:04:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 client.26669 shutdown 2024-06-13T07:04:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 msgr2=0x55ccacf94a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009620 tx=0x7fefe40045c0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 shutdown_connections 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55ccacff86a0 0x55ccacffaac0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccacf7a4b0 0x55ccacf94a40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccacf74240 0x55ccacf861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 --2- 172.21.15.7:0/1034078035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccacf79b90 0x55ccacf866f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 >> 172.21.15.7:0/1034078035 conn(0x55ccacea0170 msgr2=0x55ccacf7a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7fefcfe00700 0 client.26669 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:04:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.460+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 shutdown_connections 2024-06-13T07:04:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.461+0000 7feffb646b80 1 -- 172.21.15.7:0/1034078035 wait complete. 2024-06-13T07:04:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.461+0000 7feffb646b80 20 client.26669 trim_cache size 0 max 16384 2024-06-13T07:04:42.461 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:04:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.462+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 msgr2=0x7fefd00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.462+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 0x7fefd00a5220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009c40 tx=0x7fefe40103d0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.462+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 msgr2=0x7fefec19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.462+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fefdc004fc0 tx=0x7fefdc00bf90 comp rx=0 tx=0).stop 2024-06-13T07:04:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 shutdown_connections 2024-06-13T07:04:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefec108500 0x7fefec19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefec06a170 0x7fefec19adf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fefd00a2d60 0x7fefd00a5220 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 --2- 172.21.15.7:0/294534951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fefec06ab30 0x7fefec19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 >> 172.21.15.7:0/294534951 conn(0x7fefec078340 msgr2=0x7fefec10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.464+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 shutdown_connections 2024-06-13T07:04:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:42.465+0000 7feff0c00700 1 -- 172.21.15.7:0/294534951 wait complete. 2024-06-13T07:04:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 520.0 s, 445.6 tps, lat 229.260 ms stddev 309.620 2024-06-13T07:04:43.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:43 smithi007 ceph-mon[32077]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-06-13T07:04:43.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:04:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:43 smithi189 ceph-mon[36616]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-06-13T07:04:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:43 smithi175 ceph-mon[36886]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.6 MiB/s wr, 402 op/s 2024-06-13T07:04:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:45.206 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:45 smithi007 ceph-mon[32077]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:04:45.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:45 smithi189 ceph-mon[36616]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:04:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:45 smithi175 ceph-mon[36886]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-06-13T07:04:47.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:47 smithi189 ceph-mon[36616]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-06-13T07:04:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:47 smithi175 ceph-mon[36886]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-06-13T07:04:47.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:47 smithi007 ceph-mon[32077]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-06-13T07:04:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 525.0 s, 469.8 tps, lat 200.096 ms stddev 281.583 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 -- 172.21.15.7:0/4049610058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797811bc70 msgr2=0x7f797811e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 --2- 172.21.15.7:0/4049610058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797811bc70 0x7f797811e060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f797000b0a0 tx=0x7f797000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 -- 172.21.15.7:0/4049610058 shutdown_connections 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 --2- 172.21.15.7:0/4049610058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797811bc70 0x7f797811e060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 --2- 172.21.15.7:0/4049610058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7978110c70 0x7f7978109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 --2- 172.21.15.7:0/4049610058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 0x7f7978109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.465+0000 7f797d400700 1 -- 172.21.15.7:0/4049610058 >> 172.21.15.7:0/4049610058 conn(0x7f797806d600 msgr2=0x7f797806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.466+0000 7f797d400700 1 -- 172.21.15.7:0/4049610058 shutdown_connections 2024-06-13T07:04:48.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.466+0000 7f797d400700 1 -- 172.21.15.7:0/4049610058 wait complete. 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 Processor -- start 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 -- start start 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 0x7f797811a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 0x7f797811a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7978121650 con 0x7f7978110c70 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79781214d0 con 0x7f797811bc70 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.467+0000 7f797d400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79781217d0 con 0x7f79781099d0 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7976800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 0x7f797811a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58440/0 (socket says 172.21.15.7:58440) 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 -- 172.21.15.7:0/1180368779 learned_addr learned my addr 172.21.15.7:0/1180368779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:48.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7975e00700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 0x7f797811a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 -- 172.21.15.7:0/1180368779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 msgr2=0x7f797811a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 0x7f797811a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 -- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 msgr2=0x7f797811a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 0x7f797811a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 -- 172.21.15.7:0/1180368779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7978113c70 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7975e00700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 0x7f797811a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.468+0000 7f7977200700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7970006070 tx=0x7f7970004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.469+0000 7f796f400700 1 -- 172.21.15.7:0/1180368779 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970021070 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.469+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7978113f00 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.469+0000 7f796f400700 1 -- 172.21.15.7:0/1180368779 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7970003c00 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.469+0000 7f796f400700 1 -- 172.21.15.7:0/1180368779 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970004830 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.469+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7978117e60 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.476+0000 7f796f400700 1 -- 172.21.15.7:0/1180368779 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7970007980 con 0x7f797811bc70 2024-06-13T07:04:48.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.479+0000 7f796f400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 0x7f795c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.479+0000 7f796f400700 1 -- 172.21.15.7:0/1180368779 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f797000f020 con 0x7f797811bc70 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.479+0000 7f7976800700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 0x7f795c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.481+0000 7f7976800700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 0x7f795c0a5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7968009be0 tx=0x7f7968009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.491+0000 7f7987d42b80 1 Processor -- start 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.491+0000 7f7987d42b80 1 -- start start 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x5602926e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 0x5602926f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7976800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x5602926e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7976800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x5602926e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58444/0 (socket says 172.21.15.7:58444) 2024-06-13T07:04:48.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54120/0 (socket says 172.21.15.7:54120) 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7975e00700 1 -- 172.21.15.7:0/3003270600 learned_addr learned my addr 172.21.15.7:0/3003270600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.492+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602925f7a10 con 0x5602926f3300 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.493+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56029263ab00 con 0x560292664e40 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.493+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602925f58f0 con 0x5602926e85b0 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.493+0000 7f7977200700 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 0x5602926f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.494+0000 7f7975e00700 1 -- 172.21.15.7:0/3003270600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 msgr2=0x5602926e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.494+0000 7f7975e00700 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x5602926e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.494+0000 7f7975e00700 1 -- 172.21.15.7:0/3003270600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 msgr2=0x5602926f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.494+0000 7f7975e00700 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 0x5602926f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.494+0000 7f7975e00700 1 -- 172.21.15.7:0/3003270600 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602925e2f90 con 0x5602926e85b0 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.495+0000 7f7977200700 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 0x5602926f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.495+0000 7f7975e00700 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7964009d00 tx=0x7f79640116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdefb20a0f68ad26 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.496+0000 7f7963e00700 1 -- 172.21.15.7:0/3003270600 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964011c20 con 0x5602926e85b0 2024-06-13T07:04:48.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.497+0000 7f7963e00700 1 -- 172.21.15.7:0/3003270600 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7964011dc0 con 0x5602926e85b0 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.497+0000 7f7963e00700 1 -- 172.21.15.7:0/3003270600 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640126b0 con 0x5602926e85b0 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 msgr2=0x5602926e8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7964009d00 tx=0x7f79640116c0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 shutdown_connections 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926f3300 0x5602926f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x5602926e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3003270600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926e8a30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.498+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 >> 172.21.15.7:0/3003270600 conn(0x560292615170 msgr2=0x5602926e44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.499+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 shutdown_connections 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.499+0000 7f7987d42b80 1 -- 172.21.15.7:0/3003270600 wait complete. 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.499+0000 7f7987d42b80 1 -- start start 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.500+0000 7f7987d42b80 1 Processor -- start 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.503+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.503+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.503+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54122/0 (socket says 172.21.15.7:54122) 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7975e00700 1 -- 172.21.15.7:0/354090410 learned_addr learned my addr 172.21.15.7:0/354090410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602925d6f60 con 0x560292664e40 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560292636ac0 con 0x5602926f3300 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602926f5fc0 con 0x5602926e85b0 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.504+0000 7f7987d42b80 10 client.0 _ll_register_callbacks cb 0x5602925d4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.505+0000 7f7975e00700 1 -- 172.21.15.7:0/354090410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 msgr2=0x5602926f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.505+0000 7f7975e00700 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.505+0000 7f7975e00700 1 -- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 msgr2=0x5602926f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.505+0000 7f7975e00700 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.505+0000 7f7975e00700 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602925898d0 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.506+0000 7f7975e00700 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79640059d0 tx=0x7f79640052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4dec4b2270fae5a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.508+0000 7f7960c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.508+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964027020 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.508+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7964004960 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.508+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7964011dc0 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.510+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7964004b60 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.510+0000 7f7987d42b80 10 client.26617 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.510+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5602927006b0 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.510+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7964011dc0 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f7964012ce0 con 0x5602926e85b0 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 10 client.26617 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:48.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 10 client.26617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 20 client.26617 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x560292704f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 4 client.26617 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.511+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560292648000 con 0x560292702b80 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.512+0000 7f7976800700 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x560292704f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.516+0000 7f7976800700 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x560292704f70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4441c45562a30a1 server_cookie=9074010a9e130c40 in_seq=0 out_seq=0 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.516+0000 7f7960c00700 10 client.26617 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.519+0000 7f7960c00700 1 -- 172.21.15.7:0/354090410 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560292648000 con 0x560292702b80 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.519+0000 7f7960c00700 10 client.26617 handle_command_reply: tid=0 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.519+0000 7f7987d42b80 1 client.26617 shutdown 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.521+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 msgr2=0x5602926efd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.521+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79640059d0 tx=0x7f79640052c0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.521+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 shutdown_connections 2024-06-13T07:04:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x560292704f70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 --2- 172.21.15.7:0/354090410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926efd30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7960c00700 0 client.26617 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 >> 172.21.15.7:0/354090410 conn(0x560292615170 msgr2=0x5602926e3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 shutdown_connections 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.522+0000 7f7987d42b80 1 -- 172.21.15.7:0/354090410 wait complete. 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.523+0000 7f7987d42b80 20 client.26617 trim_cache size 0 max 16384 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.562+0000 7f7987d42b80 1 Processor -- start 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.564+0000 7f7987d42b80 1 -- start start 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.564+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.564+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.564+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7987d42b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c8080 con 0x560292664e40 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7987d42b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c5910 con 0x5602926f3300 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7987d42b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c3760 con 0x5602926e85b0 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7975e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54124/0 (socket says 172.21.15.7:54124) 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7975e00700 1 -- 172.21.15.7:0/3245604206 learned_addr learned my addr 172.21.15.7:0/3245604206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7976800700 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 -- 172.21.15.7:0/3245604206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 msgr2=0x5602926f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 -- 172.21.15.7:0/3245604206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 msgr2=0x5602926f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.565+0000 7f7977200700 1 -- 172.21.15.7:0/3245604206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602926ee2a0 con 0x5602926f3300 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.566+0000 7f7977200700 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926eb070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7970009fd0 tx=0x7f797009cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ac73270cc8003f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.566+0000 7f7963400700 1 -- 172.21.15.7:0/3245604206 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700a3040 con 0x5602926f3300 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.566+0000 7f7963400700 1 -- 172.21.15.7:0/3245604206 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700a5810 con 0x5602926f3300 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.566+0000 7f7963400700 1 -- 172.21.15.7:0/3245604206 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797009ce90 con 0x5602926f3300 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 -- 172.21.15.7:0/3245604206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 msgr2=0x5602926eb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926eb070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7970009fd0 tx=0x7f797009cc80 comp rx=0 tx=0).stop 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 -- 172.21.15.7:0/3245604206 shutdown_connections 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5602926f3300 0x5602926eb070 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560292664e40 0x5602926f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 --2- 172.21.15.7:0/3245604206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926e85b0 0x5602926f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.567+0000 7f7987d42b80 1 -- 172.21.15.7:0/3245604206 >> 172.21.15.7:0/3245604206 conn(0x560292615170 msgr2=0x5602926ee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.569+0000 7f7987d42b80 1 -- 172.21.15.7:0/3245604206 shutdown_connections 2024-06-13T07:04:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.570+0000 7f7987d42b80 1 -- 172.21.15.7:0/3245604206 wait complete. 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.570+0000 7f7987d42b80 1 -- start start 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.570+0000 7f7987d42b80 1 Processor -- start 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.571+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926e85b0 0x56029270a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926f3300 0x5602926fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7976800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7976800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58478/0 (socket says 172.21.15.7:58478) 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7976800700 1 -- 172.21.15.7:0/2606423430 learned_addr learned my addr 172.21.15.7:0/2606423430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c3dd0 con 0x5602926e85b0 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c5240 con 0x560292664e40 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79781c44a0 con 0x5602926f3300 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7987d42b80 10 client.0 _ll_register_callbacks cb 0x5602925d4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7977200700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926f3300 0x5602926fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.574+0000 7f7975e00700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926e85b0 0x56029270a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.575+0000 7f7976800700 1 -- 172.21.15.7:0/2606423430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926f3300 msgr2=0x5602926fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.575+0000 7f7976800700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926f3300 0x5602926fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.575+0000 7f7976800700 1 -- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926e85b0 msgr2=0x56029270a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.575+0000 7f7976800700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926e85b0 0x56029270a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.575+0000 7f7976800700 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602926fa840 con 0x560292664e40 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.576+0000 7f7976800700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f79680117d0 tx=0x7f79680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef9f52d5cb78914 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.576+0000 7f7960c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.576+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968020900 con 0x560292664e40 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.576+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7968020aa0 con 0x560292664e40 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.576+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79700a5810 con 0x560292664e40 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.578+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7968037050 con 0x560292664e40 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.578+0000 7f7987d42b80 10 client.26684 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.578+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5602926fbb80 con 0x560292664e40 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.578+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79700a5810 con 0x560292664e40 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f796801e420 con 0x560292664e40 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 10 client.26684 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 10 client.26684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 20 client.26684 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x56029276f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 4 client.26684 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560292642580 con 0x560292702b80 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.579+0000 7f7975e00700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x56029276f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.582+0000 7f7960c00700 10 client.26684 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.582+0000 7f7975e00700 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x56029276f6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2765622044affaa server_cookie=9132968edbad12d1 in_seq=0 out_seq=0 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7960c00700 1 -- 172.21.15.7:0/2606423430 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560292642580 con 0x560292702b80 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7960c00700 10 client.26684 handle_command_reply: tid=0 2024-06-13T07:04:48.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 client.26684 shutdown 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 msgr2=0x560292709ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f79680117d0 tx=0x7f79680047c0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 shutdown_connections 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x560292702b80 0x56029276f6c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602926f3300 0x5602926fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560292664e40 0x560292709ae0 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f79680117d0 tx=0x7f79680047c0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 --2- 172.21.15.7:0/2606423430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602926e85b0 0x56029270a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 >> 172.21.15.7:0/2606423430 conn(0x560292615170 msgr2=0x5602926e2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7960c00700 0 client.26684 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 shutdown_connections 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.584+0000 7f7987d42b80 1 -- 172.21.15.7:0/2606423430 wait complete. 2024-06-13T07:04:48.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.585+0000 7f7987d42b80 20 client.26684 trim_cache size 0 max 16384 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 msgr2=0x7f795c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 0x7f795c0a5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7968009be0 tx=0x7f7968009260 comp rx=0 tx=0).stop 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 msgr2=0x7f79781133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7970006070 tx=0x7f7970004c60 comp rx=0 tx=0).stop 2024-06-13T07:04:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 shutdown_connections 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f795c0a2ca0 0x7f795c0a5160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797811bc70 0x7f79781133b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79781099d0 0x7f797811a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 --2- 172.21.15.7:0/1180368779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7978110c70 0x7f797811a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 >> 172.21.15.7:0/1180368779 conn(0x7f797806d600 msgr2=0x7f7978071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.589+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 shutdown_connections 2024-06-13T07:04:48.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:48.591+0000 7f797d400700 1 -- 172.21.15.7:0/1180368779 wait complete. 2024-06-13T07:04:49.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:49 smithi189 ceph-mon[36616]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-06-13T07:04:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:49 smithi175 ceph-mon[36886]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-06-13T07:04:49.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:49 smithi007 ceph-mon[32077]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-06-13T07:04:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.058+0000 7f3ceb000640 1 -- 172.21.15.7:0/1152849960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a5d70 msgr2=0x7f3cdc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.058+0000 7f3ceb000640 1 --2- 172.21.15.7:0/1152849960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc0a61d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009900 tx=0x7f3cd002a420 comp rx=0 tx=0).stop 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.059+0000 7f3ceb000640 1 -- 172.21.15.7:0/1152849960 shutdown_connections 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.059+0000 7f3ceb000640 1 --2- 172.21.15.7:0/1152849960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a6710 0x7f3cdc0aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.059+0000 7f3ceb000640 1 --2- 172.21.15.7:0/1152849960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc0a61d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.059+0000 7f3ceb000640 1 --2- 172.21.15.7:0/1152849960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.059+0000 7f3ceb000640 1 -- 172.21.15.7:0/1152849960 >> 172.21.15.7:0/1152849960 conn(0x7f3cdc0a00d0 msgr2=0x7f3cdc0a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.062+0000 7f3ceb000640 1 -- 172.21.15.7:0/1152849960 shutdown_connections 2024-06-13T07:04:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.062+0000 7f3ceb000640 1 -- 172.21.15.7:0/1152849960 wait complete. 2024-06-13T07:04:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.063+0000 7f3ceb000640 1 Processor -- start 2024-06-13T07:04:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.064+0000 7f3ceb000640 1 -- start start 2024-06-13T07:04:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.065+0000 7f3ceb000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.065+0000 7f3ceb000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.065+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.065+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54142/0 (socket says 172.21.15.7:54142) 2024-06-13T07:04:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.065+0000 7f3ce9c00640 1 -- 172.21.15.7:0/432750605 learned_addr learned my addr 172.21.15.7:0/432750605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a6710 0x7f3cdc14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0b9120 con 0x7f3cdc0a5d70 2024-06-13T07:04:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0b8fa0 con 0x7f3cdc0a6710 2024-06-13T07:04:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0b92a0 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9200640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3cea600640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a6710 0x7f3cdc14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9c00640 1 -- 172.21.15.7:0/432750605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a6710 msgr2=0x7f3cdc14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a6710 0x7f3cdc14acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9c00640 1 -- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a5d70 msgr2=0x7f3cdc143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.066+0000 7f3ce9c00640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc14b320 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.067+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b450 tx=0x7f3cd800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.069+0000 7f3ce2000640 1 -- 172.21.15.7:0/432750605 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8015020 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.069+0000 7f3ce2000640 1 -- 172.21.15.7:0/432750605 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd8011d70 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.069+0000 7f3ce2000640 1 -- 172.21.15.7:0/432750605 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800f340 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.069+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc14b610 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.069+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc14bb60 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.077+0000 7f3cef466740 1 Processor -- start 2024-06-13T07:04:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.077+0000 7f3cef466740 1 -- start start 2024-06-13T07:04:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3ce2000640 1 -- 172.21.15.7:0/432750605 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f3cd8018020 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cea600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cea600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54156/0 (socket says 172.21.15.7:54156) 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cea600640 1 -- 172.21.15.7:0/362488190 learned_addr learned my addr 172.21.15.7:0/362488190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 0x55a91db89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a91dae5420 con 0x55a91db87940 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a91da8cc50 con 0x55a91db827d0 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.078+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a91dae13e0 con 0x55a91db81de0 2024-06-13T07:04:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.079+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.079+0000 7f3ce9200640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 0x55a91db89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.080+0000 7f3ce9200640 1 -- 172.21.15.7:0/362488190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 msgr2=0x55a91db82200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.080+0000 7f3ce9200640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.080+0000 7f3ce9200640 1 -- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 msgr2=0x55a91db872d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.080+0000 7f3ce9200640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db872d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.080+0000 7f3ce9200640 1 -- 172.21.15.7:0/362488190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91d7f9210 con 0x55a91db87940 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.081+0000 7f3ce2000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.081+0000 7f3ce2000640 1 -- 172.21.15.7:0/432750605 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3cd80cee10 con 0x7f3cdc0a4c40 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.081+0000 7f3cea600640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.081+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db872d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.082+0000 7f3ce9200640 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 0x55a91db89dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002a940 tx=0x7f3cd0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43de48d5dd879a6d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.082+0000 7f3cea600640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.082+0000 7f3cd6a00640 1 -- 172.21.15.7:0/362488190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd003f070 con 0x55a91db87940 2024-06-13T07:04:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.082+0000 7f3cd6a00640 1 -- 172.21.15.7:0/362488190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0031b90 con 0x55a91db87940 2024-06-13T07:04:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.082+0000 7f3cd6a00640 1 -- 172.21.15.7:0/362488190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0035070 con 0x55a91db87940 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.083+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 msgr2=0x55a91db89dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.083+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 0x55a91db89dd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002a940 tx=0x7f3cd0004170 comp rx=0 tx=0).stop 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.083+0000 7f3cea600640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067650 tx=0x7f3ce406e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 shutdown_connections 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db87940 0x55a91db89dd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db872d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 --2- 172.21.15.7:0/362488190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db81de0 0x55a91db82200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 >> 172.21.15.7:0/362488190 conn(0x55a91db49fb0 msgr2=0x55a91db7d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 shutdown_connections 2024-06-13T07:04:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.084+0000 7f3cef466740 1 -- 172.21.15.7:0/362488190 wait complete. 2024-06-13T07:04:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.085+0000 7f3cef466740 1 -- start start 2024-06-13T07:04:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.085+0000 7f3cef466740 1 Processor -- start 2024-06-13T07:04:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.086+0000 7f3cef466740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.086+0000 7f3cef466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.086+0000 7f3cef466740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.086+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.086+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59492/0 (socket says 172.21.15.7:59492) 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.087+0000 7f3ce9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.087+0000 7f3ce9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58518/0 (socket says 172.21.15.7:58518) 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.087+0000 7f3cea600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.087+0000 7f3cea600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54158/0 (socket says 172.21.15.7:54158) 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.087+0000 7f3ce9c00640 1 -- 172.21.15.7:0/3809582815 learned_addr learned my addr 172.21.15.7:0/3809582815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.088+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a91db8a890 con 0x55a91db81de0 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.088+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a91daa00b0 con 0x55a91db827d0 2024-06-13T07:04:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.088+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a91db8aa10 con 0x55a91db918e0 2024-06-13T07:04:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.088+0000 7f3cef466740 10 client.0 _ll_register_callbacks cb 0x55a91da8bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3cea600640 1 -- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 msgr2=0x55a91db8c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3cea600640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3cea600640 1 -- 172.21.15.7:0/3809582815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 msgr2=0x55a91db8c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3cea600640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3cea600640 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91d8c5620 con 0x55a91db918e0 2024-06-13T07:04:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.089+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:04:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.091+0000 7f3ce9200640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.091+0000 7f3cea600640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067280 tx=0x7f3ce4067910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e805ecde19f2213 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.091+0000 7f3ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:04:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.091+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce408b060 con 0x55a91db918e0 2024-06-13T07:04:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.092+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3ce4086450 con 0x55a91db918e0 2024-06-13T07:04:50.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.092+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd0031b90 con 0x55a91db918e0 2024-06-13T07:04:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.097+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3ce4086650 con 0x55a91db918e0 2024-06-13T07:04:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.097+0000 7f3cef466740 10 client.26629 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.097+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a91db97080 con 0x55a91db918e0 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0031b90 con 0x55a91db918e0 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f3ce408f890 con 0x55a91db918e0 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 10 client.26629 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 20 client.26629 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db98bd0 0x55a91db9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 4 client.26629 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.099+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91daf2b50 con 0x55a91db98bd0 2024-06-13T07:04:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.100+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db98bd0 0x55a91db9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.101+0000 7f3ccb400640 10 client.26629 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.101+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db98bd0 0x55a91db9b050 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16e5934218e17cf server_cookie=eee05fab5dd4313d in_seq=0 out_seq=0 2024-06-13T07:04:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.103+0000 7f3ccb400640 1 -- 172.21.15.7:0/3809582815 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a91daf2b50 con 0x55a91db98bd0 2024-06-13T07:04:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.103+0000 7f3ccb400640 10 client.26629 handle_command_reply: tid=0 2024-06-13T07:04:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.103+0000 7f3cef466740 1 client.26629 shutdown 2024-06-13T07:04:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.104+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 msgr2=0x55a91db93cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.104+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067280 tx=0x7f3ce4067910 comp rx=0 tx=0).stop 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.104+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 shutdown_connections 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.104+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db98bd0 0x55a91db9b050 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.105+0000 7f3ccb400640 0 client.26629 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.105+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db918e0 0x55a91db93cd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.105+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.105+0000 7f3cef466740 1 --2- 172.21.15.7:0/3809582815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.105+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 >> 172.21.15.7:0/3809582815 conn(0x55a91db49fb0 msgr2=0x55a91db7c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.106+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 shutdown_connections 2024-06-13T07:04:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.107+0000 7f3cef466740 1 -- 172.21.15.7:0/3809582815 wait complete. 2024-06-13T07:04:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.107+0000 7f3cef466740 20 client.26629 trim_cache size 0 max 16384 2024-06-13T07:04:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.144+0000 7f3cef466740 1 Processor -- start 2024-06-13T07:04:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.144+0000 7f3cef466740 1 -- start start 2024-06-13T07:04:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 0x55a91db94720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc158860 con 0x55a91db81de0 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc156130 con 0x55a91db827d0 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3cef466740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc153f80 con 0x55a91db49fb0 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 0x55a91db94720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 0x55a91db94720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54160/0 (socket says 172.21.15.7:54160) 2024-06-13T07:04:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.145+0000 7f3ce9c00640 1 -- 172.21.15.7:0/621638107 learned_addr learned my addr 172.21.15.7:0/621638107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.146+0000 7f3ce9200640 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 -- 172.21.15.7:0/621638107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 msgr2=0x55a91db94720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 0x55a91db94720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 -- 172.21.15.7:0/621638107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 msgr2=0x55a91db8c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 -- 172.21.15.7:0/621638107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91db86fe0 con 0x55a91db81de0 2024-06-13T07:04:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.147+0000 7f3ce9200640 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005c40 tx=0x7f3cd0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e599adb002510286 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.149+0000 7f3cd6000640 1 -- 172.21.15.7:0/621638107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd003f070 con 0x55a91db81de0 2024-06-13T07:04:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.149+0000 7f3cd6000640 1 -- 172.21.15.7:0/621638107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd004d070 con 0x55a91db81de0 2024-06-13T07:04:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.149+0000 7f3cd6000640 1 -- 172.21.15.7:0/621638107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0035040 con 0x55a91db81de0 2024-06-13T07:04:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 -- 172.21.15.7:0/621638107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 msgr2=0x55a91db8c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005c40 tx=0x7f3cd0031400 comp rx=0 tx=0).stop 2024-06-13T07:04:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 -- 172.21.15.7:0/621638107 shutdown_connections 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db827d0 0x55a91db8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91db8c370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 --2- 172.21.15.7:0/621638107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db49fb0 0x55a91db94720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.151+0000 7f3cef466740 1 -- 172.21.15.7:0/621638107 >> 172.21.15.7:0/621638107 conn(0x55a91db835b0 msgr2=0x55a91db8adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.152+0000 7f3cef466740 1 -- 172.21.15.7:0/621638107 shutdown_connections 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.152+0000 7f3cef466740 1 -- 172.21.15.7:0/621638107 wait complete. 2024-06-13T07:04:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.152+0000 7f3cef466740 1 -- start start 2024-06-13T07:04:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.153+0000 7f3cef466740 1 Processor -- start 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91dbff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 0x55a91dbffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1545f0 con 0x55a91db81de0 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc155a60 con 0x55a91db49fb0 2024-06-13T07:04:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc154cc0 con 0x55a91db827d0 2024-06-13T07:04:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3cef466740 10 client.0 _ll_register_callbacks cb 0x7f3ce407d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3ce9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:58526/0 (socket says 172.21.15.7:58526) 2024-06-13T07:04:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.154+0000 7f3ce9c00640 1 -- 172.21.15.7:0/1705887372 learned_addr learned my addr 172.21.15.7:0/1705887372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3cea600640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 0x55a91dbffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3ce9c00640 1 -- 172.21.15.7:0/1705887372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 msgr2=0x55a91dbffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 0x55a91dbffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3ce9c00640 1 -- 172.21.15.7:0/1705887372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 msgr2=0x55a91dbff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91dbff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3ce9c00640 1 -- 172.21.15.7:0/1705887372 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91dc00330 con 0x55a91db49fb0 2024-06-13T07:04:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.155+0000 7f3cea600640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 0x55a91dbffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.156+0000 7f3ce9c00640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80cc940 tx=0x7f3cd80d28f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d82c573d305a62 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.156+0000 7f3ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.156+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd809c140 con 0x55a91db49fb0 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.156+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3cd8010550 con 0x55a91db49fb0 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.156+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd004d070 con 0x55a91db49fb0 2024-06-13T07:04:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.157+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3cd809bcd0 con 0x55a91db49fb0 2024-06-13T07:04:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.157+0000 7f3cef466740 10 client.26696 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.157+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a91dbac6c0 con 0x55a91db49fb0 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.161+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd004d070 con 0x55a91db49fb0 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.161+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f3cd80d2b60 con 0x55a91db49fb0 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 10 client.26696 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 20 client.26696 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db918e0 0x55a91dc167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 4 client.26696 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:04:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a91daecea0 con 0x55a91db918e0 2024-06-13T07:04:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.162+0000 7f3ce9200640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db918e0 0x55a91dc167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.163+0000 7f3ce9200640 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db918e0 0x55a91dc167d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a356ae0c9241fb4 server_cookie=b8821824e7f8b425 in_seq=0 out_seq=0 2024-06-13T07:04:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.163+0000 7f3ccb400640 10 client.26696 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.292+0000 7f3ccb400640 1 -- 172.21.15.7:0/1705887372 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a91daecea0 con 0x55a91db918e0 2024-06-13T07:04:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.292+0000 7f3ccb400640 10 client.26696 handle_command_reply: tid=0 2024-06-13T07:04:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.292+0000 7f3cef466740 1 client.26696 shutdown 2024-06-13T07:04:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 msgr2=0x55a91dc01030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80cc940 tx=0x7f3cd80d28f0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 shutdown_connections 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a91db918e0 0x55a91dc167d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a91db827d0 0x55a91dbffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91db81de0 0x55a91dbff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 --2- 172.21.15.7:0/1705887372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a91db49fb0 0x55a91dc01030 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 >> 172.21.15.7:0/1705887372 conn(0x55a91db835b0 msgr2=0x55a91db83cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.293+0000 7f3ccb400640 0 client.26696 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.294+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 shutdown_connections 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.294+0000 7f3cef466740 1 -- 172.21.15.7:0/1705887372 wait complete. 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.294+0000 7f3cef466740 20 client.26696 trim_cache size 0 max 16384 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:04:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:04:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.297+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 msgr2=0x7f3ccc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.297+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067650 tx=0x7f3ce406e040 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.297+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 msgr2=0x7f3cdc143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.297+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b450 tx=0x7f3cd800b910 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.297+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 shutdown_connections 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f3ccc0a2c10 0x7f3ccc0a50d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cdc0a6710 0x7f3cdc14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cdc0a5d70 0x7f3cdc143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 --2- 172.21.15.7:0/432750605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cdc0a4c40 0x7f3cdc143700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 >> 172.21.15.7:0/432750605 conn(0x7f3cdc0a00d0 msgr2=0x7f3cdc0a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 shutdown_connections 2024-06-13T07:04:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:04:50.298+0000 7f3ceb000640 1 -- 172.21.15.7:0/432750605 wait complete. 2024-06-13T07:04:51.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:04:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:51 smithi189 ceph-mon[36616]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-06-13T07:04:51.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:51 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:51 smithi175 ceph-mon[36886]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-06-13T07:04:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:51 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:51.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:51 smithi007 ceph-mon[32077]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-06-13T07:04:51.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:51 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:04:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 530.0 s, 437.4 tps, lat 233.212 ms stddev 304.141 2024-06-13T07:04:53.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:53 smithi007 ceph-mon[32077]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-13T07:04:53.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:53.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:53.415 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:04:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:53 smithi189 ceph-mon[36616]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-13T07:04:53.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:53.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:53.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:04:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:53 smithi175 ceph-mon[36886]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 422 op/s 2024-06-13T07:04:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:04:53.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:04:53.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:04:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:55 smithi189 ceph-mon[36616]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-06-13T07:04:55.611 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:55 smithi007 ceph-mon[32077]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-06-13T07:04:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:55 smithi175 ceph-mon[36886]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-06-13T07:04:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.610+0000 7f3718400700 1 -- 172.21.15.7:0/2870002972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 msgr2=0x7f3710115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.610+0000 7f3718400700 1 --2- 172.21.15.7:0/2870002972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f3710115d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3704009990 tx=0x7f3704009ca0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:04:55.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.610+0000 7f3718400700 1 -- 172.21.15.7:0/2870002972 shutdown_connections 2024-06-13T07:04:55.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.610+0000 7f3718400700 1 --2- 172.21.15.7:0/2870002972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f3710115d80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.611+0000 7f3718400700 1 --2- 172.21.15.7:0/2870002972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f371010dee0 0x7f371010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.612+0000 7f3718400700 1 --2- 172.21.15.7:0/2870002972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3710109490 0x7f3710109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.612+0000 7f3718400700 1 -- 172.21.15.7:0/2870002972 >> 172.21.15.7:0/2870002972 conn(0x7f371006d9e0 msgr2=0x7f371006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.613+0000 7f3718400700 1 -- 172.21.15.7:0/2870002972 shutdown_connections 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.613+0000 7f3718400700 1 -- 172.21.15.7:0/2870002972 wait complete. 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 Processor -- start 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 -- start start 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f371011f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f371010dee0 0x7f371011a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3710118eb0 con 0x7f371010dee0 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3710118d30 con 0x7f3710109e50 2024-06-13T07:04:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.615+0000 7f3718400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3710119030 con 0x7f3710109490 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46328/0 (socket says 172.21.15.7:46328) 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 -- 172.21.15.7:0/2200490200 learned_addr learned my addr 172.21.15.7:0/2200490200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3716400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f371010dee0 0x7f371011a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715000700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f371011f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 -- 172.21.15.7:0/2200490200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 msgr2=0x7f371011f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f371011f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 -- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f371010dee0 msgr2=0x7f371011a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f371010dee0 0x7f371011a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.616+0000 7f3715a00700 1 -- 172.21.15.7:0/2200490200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371011abc0 con 0x7f3710109490 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.617+0000 7f3715a00700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f370800b840 tx=0x7f370800bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.617+0000 7f370e000700 1 -- 172.21.15.7:0/2200490200 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370800f840 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.617+0000 7f370e000700 1 -- 172.21.15.7:0/2200490200 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3708017400 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.617+0000 7f370e000700 1 -- 172.21.15.7:0/2200490200 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370800d630 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.617+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371011aeb0 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.618+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3710125090 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.623+0000 7f370e000700 1 -- 172.21.15.7:0/2200490200 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f37080175b0 con 0x7f3710109490 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 Processor -- start 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 -- start start 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f370e000700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 0x7f36f80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 0x558ebff72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 0x558ebff6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff6d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ebfe7ba10 con 0x558ebff6d240 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ebfebeb00 con 0x558ebff72b90 2024-06-13T07:04:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.624+0000 7f3722d03b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ebfe798f0 con 0x558ebff734b0 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f370e000700 1 -- 172.21.15.7:0/2200490200 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f37080cf110 con 0x7f3710109490 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 0x558ebff72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 0x558ebff72f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52978/0 (socket says 172.21.15.7:52978) 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715a00700 1 -- 172.21.15.7:0/4063879565 learned_addr learned my addr 172.21.15.7:0/4063879565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 0x558ebff6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3716400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 0x7f36f80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3716400700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff6d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 -- 172.21.15.7:0/4063879565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 msgr2=0x558ebff72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 0x558ebff72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 -- 172.21.15.7:0/4063879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 msgr2=0x558ebff6d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff6d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3715000700 1 -- 172.21.15.7:0/4063879565 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebfe66f90 con 0x558ebff734b0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.625+0000 7f3716400700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff6d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.626+0000 7f3715000700 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 0x558ebff6cd00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f371011bb10 tx=0x7f370000af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c68e59623d028dd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.626+0000 7f3716400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 0x7f36f80a52b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f370400a010 tx=0x7f370401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.626+0000 7f36fea00700 1 -- 172.21.15.7:0/4063879565 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000fd60 con 0x558ebff734b0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.626+0000 7f36fea00700 1 -- 172.21.15.7:0/4063879565 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700004500 con 0x558ebff734b0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.626+0000 7f36fea00700 1 -- 172.21.15.7:0/4063879565 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700005310 con 0x558ebff734b0 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 -- 172.21.15.7:0/4063879565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 msgr2=0x558ebff6cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 0x558ebff6cd00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f371011bb10 tx=0x7f370000af10 comp rx=0 tx=0).stop 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 -- 172.21.15.7:0/4063879565 shutdown_connections 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff6d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff72b90 0x558ebff72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 --2- 172.21.15.7:0/4063879565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff734b0 0x558ebff6cd00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 -- 172.21.15.7:0/4063879565 >> 172.21.15.7:0/4063879565 conn(0x558ebfe99170 msgr2=0x558ebff67ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 -- 172.21.15.7:0/4063879565 shutdown_connections 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.627+0000 7f3722d03b80 1 -- 172.21.15.7:0/4063879565 wait complete. 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.628+0000 7f3722d03b80 1 -- start start 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.628+0000 7f3722d03b80 1 Processor -- start 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.628+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.628+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff76520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 0x558ebff76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3722d03b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ebfe5af60 con 0x558ebff6d240 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3722d03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ebfebaac0 con 0x558ebff734b0 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3722d03b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ebff79e90 con 0x558ebff72b90 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3722d03b80 10 client.0 _ll_register_callbacks cb 0x558ebfe54380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3715a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3716400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 0x558ebff76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3716400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 0x558ebff76a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52990/0 (socket says 172.21.15.7:52990) 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3715000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff76520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.629+0000 7f3716400700 1 -- 172.21.15.7:0/1636335155 learned_addr learned my addr 172.21.15.7:0/1636335155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 -- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 msgr2=0x558ebff76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 0x558ebff76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 -- 172.21.15.7:0/1636335155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 msgr2=0x558ebff79220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff79220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 -- 172.21.15.7:0/1636335155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebfe0d8d0 con 0x558ebff72b90 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3715000700 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff76520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f370000a600 tx=0x7f3700004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4b4550063abc52 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f36f3400700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700026020 con 0x558ebff72b90 2024-06-13T07:04:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3700005380 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.630+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3700004500 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.631+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f370001c070 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.631+0000 7f3722d03b80 10 client.26641 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.631+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558ebff84610 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.631+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700004500 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.631+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f3700010430 con 0x558ebff72b90 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 10 client.26641 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 20 client.26641 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff86b40 0x558ebff88f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 4 client.26641 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ebfecc000 con 0x558ebff86b40 2024-06-13T07:04:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.632+0000 7f3715a00700 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff86b40 0x558ebff88f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.633+0000 7f3715a00700 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff86b40 0x558ebff88f30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1c253a65322129 server_cookie=3f3b5c0527402e40 in_seq=0 out_seq=0 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.633+0000 7f36f3400700 10 client.26641 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f36f3400700 1 -- 172.21.15.7:0/1636335155 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ebfecc000 con 0x558ebff86b40 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f36f3400700 10 client.26641 handle_command_reply: tid=0 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 client.26641 shutdown 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 msgr2=0x558ebff76520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff76520 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f370000a600 tx=0x7f3700004a30 comp rx=0 tx=0).stop 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 shutdown_connections 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff86b40 0x558ebff88f30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff734b0 0x558ebff76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff79220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1636335155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff76520 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 >> 172.21.15.7:0/1636335155 conn(0x558ebfe99170 msgr2=0x558ebff61790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.634+0000 7f36f3400700 0 client.26641 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.635+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 shutdown_connections 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.635+0000 7f3722d03b80 1 -- 172.21.15.7:0/1636335155 wait complete. 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.635+0000 7f3722d03b80 20 client.26641 trim_cache size 0 max 16384 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.662+0000 7f3722d03b80 1 Processor -- start 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.662+0000 7f3722d03b80 1 -- start start 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.662+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebfe99170 0x558ebff6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.662+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.662+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.663+0000 7f3722d03b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c8160 con 0x558ebff6d240 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.663+0000 7f3722d03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c59f0 con 0x558ebfe99170 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.663+0000 7f3722d03b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c37e0 con 0x558ebff72b90 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.664+0000 7f3716400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.664+0000 7f3716400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46346/0 (socket says 172.21.15.7:46346) 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.664+0000 7f3716400700 1 -- 172.21.15.7:0/1029079975 learned_addr learned my addr 172.21.15.7:0/1029079975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.664+0000 7f3715000700 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.665+0000 7f3716400700 1 -- 172.21.15.7:0/1029079975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebfe99170 msgr2=0x558ebff6cd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.665+0000 7f3716400700 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebfe99170 0x558ebff6cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.665+0000 7f3716400700 1 -- 172.21.15.7:0/1029079975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 msgr2=0x558ebff681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.666+0000 7f3716400700 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.666+0000 7f3716400700 1 -- 172.21.15.7:0/1029079975 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ebff762f0 con 0x558ebff72b90 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.666+0000 7f3715000700 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.667+0000 7f3716400700 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3704005b40 tx=0x7f370400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f788147c2fd98bf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.667+0000 7f36fe000700 1 -- 172.21.15.7:0/1029079975 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704003dd0 con 0x558ebff72b90 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.668+0000 7f36fe000700 1 -- 172.21.15.7:0/1029079975 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37040055f0 con 0x558ebff72b90 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.668+0000 7f36fe000700 1 -- 172.21.15.7:0/1029079975 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370402ab10 con 0x558ebff72b90 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 -- 172.21.15.7:0/1029079975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 msgr2=0x558ebff62130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3704005b40 tx=0x7f370400b540 comp rx=0 tx=0).stop 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 -- 172.21.15.7:0/1029079975 shutdown_connections 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebff72b90 0x558ebff62130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebfe99170 0x558ebff6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1029079975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff6d240 0x558ebff681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 -- 172.21.15.7:0/1029079975 >> 172.21.15.7:0/1029079975 conn(0x558ebff6dbd0 msgr2=0x558ebff67a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 -- 172.21.15.7:0/1029079975 shutdown_connections 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.669+0000 7f3722d03b80 1 -- 172.21.15.7:0/1029079975 wait complete. 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.670+0000 7f3722d03b80 1 -- start start 2024-06-13T07:04:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.670+0000 7f3722d03b80 1 Processor -- start 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 0x558ebff89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 0x558ebffef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c3eb0 con 0x558ebff72b90 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c5320 con 0x558ebff6d240 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37101c4580 con 0x558ebfe99170 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3722d03b80 10 client.0 _ll_register_callbacks cb 0x558ebfe54380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3715000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3715000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:53004/0 (socket says 172.21.15.7:53004) 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3715000700 1 -- 172.21.15.7:0/1685977578 learned_addr learned my addr 172.21.15.7:0/1685977578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:04:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.671+0000 7f3716400700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 0x558ebffef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715a00700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 0x558ebff89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715000700 1 -- 172.21.15.7:0/1685977578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 msgr2=0x558ebff89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715000700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 0x558ebff89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715000700 1 -- 172.21.15.7:0/1685977578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 msgr2=0x558ebffef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715000700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 0x558ebffef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715000700 1 -- 172.21.15.7:0/1685977578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ebffef9b0 con 0x558ebff6d240 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3716400700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 0x558ebffef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.672+0000 7f3715a00700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 0x558ebff89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.673+0000 7f3715000700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f370000f6a0 tx=0x7f3700004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d6ea562c0a764e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.673+0000 7f36f3400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700026040 con 0x558ebff6d240 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37040055f0 con 0x558ebff6d240 2024-06-13T07:04:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3700010430 con 0x558ebff6d240 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3700044050 con 0x558ebff6d240 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f3722d03b80 10 client.26711 fetch_fsmap learned FSMap version 38 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.674+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558ebfff0c80 con 0x558ebff6d240 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37040055f0 con 0x558ebff6d240 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f3700003680 con 0x558ebff6d240 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 10 client.26711 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 20 client.26711 populate_metadata read hostname 'smithi007' 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff734b0 0x558ebfff32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 4 client.26711 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ebfec6580 con 0x558ebff734b0 2024-06-13T07:04:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.675+0000 7f3715a00700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff734b0 0x558ebfff32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.676+0000 7f3715a00700 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff734b0 0x558ebfff32b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6ca6952ea7099a server_cookie=639220c50f03f653 in_seq=0 out_seq=0 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.676+0000 7f36f3400700 10 client.26711 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.676+0000 7f36f3400700 1 -- 172.21.15.7:0/1685977578 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ebfec6580 con 0x558ebff734b0 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.676+0000 7f36f3400700 10 client.26711 handle_command_reply: tid=0 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.676+0000 7f3722d03b80 1 client.26711 shutdown 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 msgr2=0x558ebff89650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f370000f6a0 tx=0x7f3700004c00 comp rx=0 tx=0).stop 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 shutdown_connections 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x558ebff734b0 0x558ebfff32b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ebff72b90 0x558ebffef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ebfe99170 0x558ebff89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 --2- 172.21.15.7:0/1685977578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ebff6d240 0x558ebff89650 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 >> 172.21.15.7:0/1685977578 conn(0x558ebff6dbd0 msgr2=0x558ebff73890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f36f3400700 0 client.26711 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.677+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 shutdown_connections 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.678+0000 7f3722d03b80 1 -- 172.21.15.7:0/1685977578 wait complete. 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.678+0000 7f3722d03b80 20 client.26711 trim_cache size 0 max 16384 2024-06-13T07:04:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.679+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 msgr2=0x7f36f80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.679+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 0x7f36f80a52b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f370400a010 tx=0x7f370401a040 comp rx=0 tx=0).stop 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.679+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 msgr2=0x7f371011f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.679+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f370800b840 tx=0x7f370800bb50 comp rx=0 tx=0).stop 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 shutdown_connections 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f371010dee0 0x7f371011a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3710109490 0x7f371011f1a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f36f80a2df0 0x7f36f80a52b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 --2- 172.21.15.7:0/2200490200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3710109e50 0x7f371011f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:04:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.681+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 >> 172.21.15.7:0/2200490200 conn(0x7f371006d9e0 msgr2=0x7f3710071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:04:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.682+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 shutdown_connections 2024-06-13T07:04:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:04:55.682+0000 7f3718400700 1 -- 172.21.15.7:0/2200490200 wait complete. 2024-06-13T07:04:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:57 smithi189 ceph-mon[36616]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-06-13T07:04:57.588 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:57 smithi007 ceph-mon[32077]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-06-13T07:04:57.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:57 smithi175 ceph-mon[36886]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-06-13T07:04:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 535.0 s, 480.8 tps, lat 213.972 ms stddev 314.418 2024-06-13T07:04:58.592 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:04:58.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:04:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:04:59.571 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:04:59 smithi007 ceph-mon[32077]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-13T07:04:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:04:59 smithi189 ceph-mon[36616]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-13T07:04:59.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:04:59 smithi175 ceph-mon[36886]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-06-13T07:05:01.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:01 smithi189 ceph-mon[36616]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-06-13T07:05:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:01 smithi175 ceph-mon[36886]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-06-13T07:05:01.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:01 smithi007 ceph-mon[32077]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.1 MiB/s wr, 552 op/s 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.292+0000 7f02a4c00700 1 -- 172.21.15.7:0/3897133271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a01143e0 msgr2=0x7f02a01167d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.292+0000 7f02a4c00700 1 --2- 172.21.15.7:0/3897133271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a01143e0 0x7f02a01167d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f029800b0a0 tx=0x7f029800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 -- 172.21.15.7:0/3897133271 shutdown_connections 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 --2- 172.21.15.7:0/3897133271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a01143e0 0x7f02a01167d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 --2- 172.21.15.7:0/3897133271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a010ae30 0x7f02a010b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 --2- 172.21.15.7:0/3897133271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010b750 0x7f02a0113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 -- 172.21.15.7:0/3897133271 >> 172.21.15.7:0/3897133271 conn(0x7f02a006d600 msgr2=0x7f02a006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.293+0000 7f02a4c00700 1 -- 172.21.15.7:0/3897133271 shutdown_connections 2024-06-13T07:05:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.294+0000 7f02a4c00700 1 -- 172.21.15.7:0/3897133271 wait complete. 2024-06-13T07:05:02.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.296+0000 7f02a4c00700 1 Processor -- start 2024-06-13T07:05:02.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.296+0000 7f02a4c00700 1 -- start start 2024-06-13T07:05:02.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010ae30 0x7f02a011a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 0x7f02a011eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0118e30 con 0x7f02a01143e0 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0118cb0 con 0x7f02a010ae30 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.297+0000 7f02a4c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0118fb0 con 0x7f02a010b750 2024-06-13T07:05:02.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.298+0000 7f029d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.298+0000 7f029d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56122/0 (socket says 172.21.15.7:56122) 2024-06-13T07:05:02.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.298+0000 7f029d400700 1 -- 172.21.15.7:0/1201045090 learned_addr learned my addr 172.21.15.7:0/1201045090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:02.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.298+0000 7f029de00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010ae30 0x7f02a011a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029d400700 1 -- 172.21.15.7:0/1201045090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010ae30 msgr2=0x7f02a011a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.298+0000 7f029e800700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 0x7f02a011eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029d400700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010ae30 0x7f02a011a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029d400700 1 -- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 msgr2=0x7f02a011eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029d400700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 0x7f02a011eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029d400700 1 -- 172.21.15.7:0/1201045090 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a011f5d0 con 0x7f02a010b750 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.299+0000 7f029e800700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 0x7f02a011eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.300+0000 7f029d400700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e9f0 tx=0x7f028c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.301+0000 7f0296a00700 1 -- 172.21.15.7:0/1201045090 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00cbd0 con 0x7f02a010b750 2024-06-13T07:05:02.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.301+0000 7f0296a00700 1 -- 172.21.15.7:0/1201045090 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f028c00cd70 con 0x7f02a010b750 2024-06-13T07:05:02.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.301+0000 7f0296a00700 1 -- 172.21.15.7:0/1201045090 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c018710 con 0x7f02a010b750 2024-06-13T07:05:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.303+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a011f8c0 con 0x7f02a010b750 2024-06-13T07:05:02.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.304+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a0125140 con 0x7f02a010b750 2024-06-13T07:05:02.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.311+0000 7f0296a00700 1 -- 172.21.15.7:0/1201045090 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f028c01f020 con 0x7f02a010b750 2024-06-13T07:05:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.313+0000 7f0296a00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 0x7f02840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.314+0000 7f0296a00700 1 -- 172.21.15.7:0/1201045090 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f028c0cf490 con 0x7f02a010b750 2024-06-13T07:05:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.317+0000 7f029de00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 0x7f02840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.318+0000 7f029de00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 0x7f02840a52e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02a011bbc0 tx=0x7f0290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.318+0000 7f02af66bb80 1 Processor -- start 2024-06-13T07:05:02.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 -- start start 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bcb4bfa90 con 0x557bcb5b0630 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bcb502b80 con 0x557bcb52cec0 2024-06-13T07:05:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.319+0000 7f02af66bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557bcb4bd970 con 0x557bcb5bb380 2024-06-13T07:05:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.320+0000 7f029e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.320+0000 7f029e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56128/0 (socket says 172.21.15.7:56128) 2024-06-13T07:05:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.320+0000 7f029e800700 1 -- 172.21.15.7:0/1554222802 learned_addr learned my addr 172.21.15.7:0/1554222802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.320+0000 7f029de00700 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.321+0000 7f029d400700 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.321+0000 7f029e800700 1 -- 172.21.15.7:0/1554222802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 msgr2=0x557bcb5b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.321+0000 7f029e800700 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.322+0000 7f029e800700 1 -- 172.21.15.7:0/1554222802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 msgr2=0x557bcb5b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.322+0000 7f029e800700 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.322+0000 7f029e800700 1 -- 172.21.15.7:0/1554222802 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bcb4ab010 con 0x557bcb5bb380 2024-06-13T07:05:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.322+0000 7f029e800700 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0298006070 tx=0x7f0298009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356517712a3853bb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.323+0000 7f028b400700 1 -- 172.21.15.7:0/1554222802 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029800bab0 con 0x557bcb5bb380 2024-06-13T07:05:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.323+0000 7f028b400700 1 -- 172.21.15.7:0/1554222802 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f029800bc50 con 0x557bcb5bb380 2024-06-13T07:05:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.323+0000 7f028b400700 1 -- 172.21.15.7:0/1554222802 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298005040 con 0x557bcb5bb380 2024-06-13T07:05:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.328+0000 7f02af66bb80 1 -- 172.21.15.7:0/1554222802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 msgr2=0x557bcb5bd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.328+0000 7f02af66bb80 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0298006070 tx=0x7f0298009470 comp rx=0 tx=0).stop 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 -- 172.21.15.7:0/1554222802 shutdown_connections 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5bd770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 --2- 172.21.15.7:0/1554222802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 -- 172.21.15.7:0/1554222802 >> 172.21.15.7:0/1554222802 conn(0x557bcb4dd1f0 msgr2=0x557bcb5ac530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.330+0000 7f02af66bb80 1 -- 172.21.15.7:0/1554222802 shutdown_connections 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.332+0000 7f02af66bb80 1 -- 172.21.15.7:0/1554222802 wait complete. 2024-06-13T07:05:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.333+0000 7f02af66bb80 1 -- start start 2024-06-13T07:05:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.333+0000 7f02af66bb80 1 Processor -- start 2024-06-13T07:05:02.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 0x557bcb5baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5bb380 0x557bcb5b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bcb49efe0 con 0x557bcb52cec0 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557bcb4feb40 con 0x557bcb5bb380 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557bcb5bdfe0 con 0x557bcb5b0630 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.336+0000 7f02af66bb80 10 client.0 _ll_register_callbacks cb 0x557bcb49c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.337+0000 7f029d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.337+0000 7f029d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56144/0 (socket says 172.21.15.7:56144) 2024-06-13T07:05:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.337+0000 7f029d400700 1 -- 172.21.15.7:0/3198897703 learned_addr learned my addr 172.21.15.7:0/3198897703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.338+0000 7f029de00700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 0x557bcb5baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 -- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5bb380 msgr2=0x557bcb5b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5bb380 0x557bcb5b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 -- 172.21.15.7:0/3198897703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 msgr2=0x557bcb5baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 0x557bcb5baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 -- 172.21.15.7:0/3198897703 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcb451950 con 0x557bcb5b0630 2024-06-13T07:05:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029de00700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 0x557bcb5baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f029d400700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f028c009230 tx=0x7f028c09be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b10b841211f8752 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.339+0000 7f027fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.343+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0b3020 con 0x557bcb5b0630 2024-06-13T07:05:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.344+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f028c0109a0 con 0x557bcb5b0630 2024-06-13T07:05:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.344+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f029800bc50 con 0x557bcb5b0630 2024-06-13T07:05:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.348+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f028c0b9050 con 0x557bcb5b0630 2024-06-13T07:05:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.348+0000 7f02af66bb80 10 client.26659 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.348+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557bcb5c8720 con 0x557bcb5b0630 2024-06-13T07:05:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f029800bc50 con 0x557bcb5b0630 2024-06-13T07:05:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f028c0b4b50 con 0x557bcb5b0630 2024-06-13T07:05:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 10 client.26659 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 10 client.26659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 20 client.26659 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb5cad60 0x557bcb5cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 4 client.26659 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.349+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bcb510080 con 0x557bcb5cad60 2024-06-13T07:05:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.352+0000 7f029de00700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb5cad60 0x557bcb5cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.353+0000 7f029de00700 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb5cad60 0x557bcb5cd190 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cef31c8e39ff716 server_cookie=1ebccd10ff835b18 in_seq=0 out_seq=0 2024-06-13T07:05:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.353+0000 7f027fe00700 10 client.26659 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.354+0000 7f027fe00700 1 -- 172.21.15.7:0/3198897703 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557bcb510080 con 0x557bcb5cad60 2024-06-13T07:05:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.354+0000 7f027fe00700 10 client.26659 handle_command_reply: tid=0 2024-06-13T07:05:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.354+0000 7f02af66bb80 1 client.26659 shutdown 2024-06-13T07:05:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.354+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 msgr2=0x557bcb5b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.354+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f028c009230 tx=0x7f028c09be90 comp rx=0 tx=0).stop 2024-06-13T07:05:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 shutdown_connections 2024-06-13T07:05:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb5cad60 0x557bcb5cd190 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5bb380 0x557bcb5b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb52cec0 0x557bcb5baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3198897703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5b0630 0x557bcb5b7db0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f027fe00700 0 client.26659 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 >> 172.21.15.7:0/3198897703 conn(0x557bcb4dd1f0 msgr2=0x557bcb5ab7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 shutdown_connections 2024-06-13T07:05:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 1 -- 172.21.15.7:0/3198897703 wait complete. 2024-06-13T07:05:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.356+0000 7f02af66bb80 20 client.26659 trim_cache size 0 max 16384 2024-06-13T07:05:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.407+0000 7f02af66bb80 1 Processor -- start 2024-06-13T07:05:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.407+0000 7f02af66bb80 1 -- start start 2024-06-13T07:05:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5b0630 0x557bcb5ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5bb380 0x557bcb5d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c8240 con 0x557bcb5bb380 2024-06-13T07:05:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c5ad0 con 0x557bcb5b0630 2024-06-13T07:05:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f02af66bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c38e0 con 0x557bcb52cec0 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f029de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f029de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56146/0 (socket says 172.21.15.7:56146) 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f029de00700 1 -- 172.21.15.7:0/2218471011 learned_addr learned my addr 172.21.15.7:0/2218471011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f029e800700 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5bb380 0x557bcb5d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.408+0000 7f029d400700 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5b0630 0x557bcb5ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 -- 172.21.15.7:0/2218471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5b0630 msgr2=0x557bcb5ba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5b0630 0x557bcb5ba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 -- 172.21.15.7:0/2218471011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5bb380 msgr2=0x557bcb5d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5bb380 0x557bcb5d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 -- 172.21.15.7:0/2218471011 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bcb5b1c50 con 0x557bcb52cec0 2024-06-13T07:05:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f029de00700 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x557bcb5cda00 tx=0x7f0290009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe55a43f57f42cd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.409+0000 7f028aa00700 1 -- 172.21.15.7:0/2218471011 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290010910 con 0x557bcb52cec0 2024-06-13T07:05:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.410+0000 7f028aa00700 1 -- 172.21.15.7:0/2218471011 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0290004020 con 0x557bcb52cec0 2024-06-13T07:05:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.410+0000 7f028aa00700 1 -- 172.21.15.7:0/2218471011 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290005020 con 0x557bcb52cec0 2024-06-13T07:05:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 -- 172.21.15.7:0/2218471011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 msgr2=0x557bcb5bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x557bcb5cda00 tx=0x7f0290009f90 comp rx=0 tx=0).stop 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 -- 172.21.15.7:0/2218471011 shutdown_connections 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5bb380 0x557bcb5d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb52cec0 0x557bcb5bc770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 --2- 172.21.15.7:0/2218471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb5b0630 0x557bcb5ba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 -- 172.21.15.7:0/2218471011 >> 172.21.15.7:0/2218471011 conn(0x557bcb4dd1f0 msgr2=0x557bcb5b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 -- 172.21.15.7:0/2218471011 shutdown_connections 2024-06-13T07:05:02.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.411+0000 7f02af66bb80 1 -- 172.21.15.7:0/2218471011 wait complete. 2024-06-13T07:05:02.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.412+0000 7f02af66bb80 1 -- start start 2024-06-13T07:05:02.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.412+0000 7f02af66bb80 1 Processor -- start 2024-06-13T07:05:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c3f90 con 0x557bcb5b0630 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c5400 con 0x557bcb52cec0 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.414+0000 7f02af66bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01c4660 con 0x557bcb5bb380 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.415+0000 7f02af66bb80 10 client.0 _ll_register_callbacks cb 0x557bcb49c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.415+0000 7f029e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.415+0000 7f029e800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56154/0 (socket says 172.21.15.7:56154) 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.415+0000 7f029e800700 1 -- 172.21.15.7:0/3774758387 learned_addr learned my addr 172.21.15.7:0/3774758387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.416+0000 7f029e800700 1 -- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 msgr2=0x557bcb5c3280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.416+0000 7f029e800700 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5c3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.416+0000 7f029e800700 1 -- 172.21.15.7:0/3774758387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 msgr2=0x557bcb5c37c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.416+0000 7f029e800700 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.416+0000 7f029e800700 1 -- 172.21.15.7:0/3774758387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcb5c3ec0 con 0x557bcb5bb380 2024-06-13T07:05:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.417+0000 7f029e800700 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0298021000 tx=0x7f0298009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4e5e3092daace1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.417+0000 7f027fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.417+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298029020 con 0x557bcb5bb380 2024-06-13T07:05:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.417+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f02980074a0 con 0x557bcb5bb380 2024-06-13T07:05:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.418+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0290004020 con 0x557bcb5bb380 2024-06-13T07:05:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.418+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0298004e60 con 0x557bcb5bb380 2024-06-13T07:05:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.418+0000 7f02af66bb80 10 client.26665 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.418+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557bcb636080 con 0x557bcb5bb380 2024-06-13T07:05:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.419+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0290004020 con 0x557bcb5bb380 2024-06-13T07:05:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.420+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f0298011070 con 0x557bcb5bb380 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.420+0000 7f02af66bb80 10 client.26665 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.420+0000 7f02af66bb80 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.420+0000 7f02af66bb80 20 client.26665 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.421+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb636310 0x557bcb638700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.421+0000 7f02af66bb80 4 client.26665 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.421+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557bcb50a600 con 0x557bcb636310 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.421+0000 7f029de00700 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb636310 0x557bcb638700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.422+0000 7f029de00700 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb636310 0x557bcb638700 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e890826b2ffb2e server_cookie=c93162a56dc49334 in_seq=0 out_seq=0 2024-06-13T07:05:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.422+0000 7f027fe00700 10 client.26665 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.423+0000 7f027fe00700 1 -- 172.21.15.7:0/3774758387 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557bcb50a600 con 0x557bcb636310 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.423+0000 7f027fe00700 10 client.26665 handle_command_reply: tid=0 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.423+0000 7f02af66bb80 1 client.26665 shutdown 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.423+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 msgr2=0x557bcb5be230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.423+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0298021000 tx=0x7f0298009e20 comp rx=0 tx=0).stop 2024-06-13T07:05:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 shutdown_connections 2024-06-13T07:05:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557bcb636310 0x557bcb638700 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557bcb5bb380 0x557bcb5be230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557bcb52cec0 0x557bcb5c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 --2- 172.21.15.7:0/3774758387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bcb5b0630 0x557bcb5c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 >> 172.21.15.7:0/3774758387 conn(0x557bcb4dd1f0 msgr2=0x557bcb5aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.424+0000 7f027fe00700 0 client.26665 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.425+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 shutdown_connections 2024-06-13T07:05:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.425+0000 7f02af66bb80 1 -- 172.21.15.7:0/3774758387 wait complete. 2024-06-13T07:05:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.426+0000 7f02af66bb80 20 client.26665 trim_cache size 0 max 16384 2024-06-13T07:05:02.427 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:05:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:05:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "fc64ad60-06e1-4bc3-9a5a-a91673d2c1ec", 2024-06-13T07:05:02.428 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:05:02.428 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:05:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.428+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 msgr2=0x7f02840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.428+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 0x7f02840a52e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02a011bbc0 tx=0x7f0290009260 comp rx=0 tx=0).stop 2024-06-13T07:05:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.428+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 msgr2=0x7f02a011ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.428+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e9f0 tx=0x7f028c00edb0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 shutdown_connections 2024-06-13T07:05:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02a01143e0 0x7f02a011eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f02840a2e20 0x7f02840a52e0 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02a011bbc0 tx=0x7f0290009260 comp rx=0 tx=0).stop 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02a010ae30 0x7f02a011a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 --2- 172.21.15.7:0/1201045090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02a010b750 0x7f02a011ab90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.429+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 >> 172.21.15.7:0/1201045090 conn(0x7f02a006d600 msgr2=0x7f02a0071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.430+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 shutdown_connections 2024-06-13T07:05:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:02.430+0000 7f02a4c00700 1 -- 172.21.15.7:0/1201045090 wait complete. 2024-06-13T07:05:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 540.0 s, 443.0 tps, lat 213.089 ms stddev 327.826 2024-06-13T07:05:03.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:05:03.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:03 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-06-13T07:05:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:05:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:05:03.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:03 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:05:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:03 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 -- 172.21.15.7:0/3573921728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410c3e0 msgr2=0x7f121410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 --2- 172.21.15.7:0/3573921728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410c3e0 0x7f121410e7d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009900 tx=0x7f11fc02a400 comp rx=0 tx=0).stop 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 -- 172.21.15.7:0/3573921728 shutdown_connections 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 --2- 172.21.15.7:0/3573921728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410ed60 0x7f1214111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 --2- 172.21.15.7:0/3573921728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410c3e0 0x7f121410e7d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 --2- 172.21.15.7:0/3573921728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 0x7f121410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.173+0000 7f1219c00640 1 -- 172.21.15.7:0/3573921728 >> 172.21.15.7:0/3573921728 conn(0x7f121406ebd0 msgr2=0x7f1214071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.174+0000 7f1219c00640 1 -- 172.21.15.7:0/3573921728 shutdown_connections 2024-06-13T07:05:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.174+0000 7f1219c00640 1 -- 172.21.15.7:0/3573921728 wait complete. 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.174+0000 7f1219c00640 1 Processor -- start 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 -- start start 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 0x7f121410b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410ed60 0x7f121410a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1214114dd0 con 0x7f1214109ab0 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1214114c50 con 0x7f121410ed60 2024-06-13T07:05:04.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1219c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1214114f50 con 0x7f121410c3e0 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f120a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1212a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 0x7f121410b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f120a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56172/0 (socket says 172.21.15.7:56172) 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f120a000640 1 -- 172.21.15.7:0/3641240524 learned_addr learned my addr 172.21.15.7:0/3641240524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.175+0000 7f1213400640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410ed60 0x7f121410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 -- 172.21.15.7:0/3641240524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410ed60 msgr2=0x7f121410a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410ed60 0x7f121410a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 -- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 msgr2=0x7f121410b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 0x7f121410b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 -- 172.21.15.7:0/3641240524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121410ae50 con 0x7f121410c3e0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.176+0000 7f120a000640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f11fc031440 tx=0x7f11fc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.177+0000 7f120be00640 1 -- 172.21.15.7:0/3641240524 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc03f030 con 0x7f121410c3e0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.177+0000 7f120be00640 1 -- 172.21.15.7:0/3641240524 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc003ce0 con 0x7f121410c3e0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.177+0000 7f120be00640 1 -- 172.21.15.7:0/3641240524 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc035070 con 0x7f121410c3e0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.177+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141aa550 con 0x7f121410c3e0 2024-06-13T07:05:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.177+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141aa9c0 con 0x7f121410c3e0 2024-06-13T07:05:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.180+0000 7f120be00640 1 -- 172.21.15.7:0/3641240524 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f11fc042030 con 0x7f121410c3e0 2024-06-13T07:05:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.181+0000 7f121c9e6740 1 Processor -- start 2024-06-13T07:05:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.181+0000 7f121c9e6740 1 -- start start 2024-06-13T07:05:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.181+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051a95020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.181+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051a9a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051a9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f121c9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620519f8420 con 0x562051a9a760 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f121c9e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56205199fc50 con 0x562051a94c00 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f121c9e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5620519f43e0 con 0x562051a955f0 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1213400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051a95020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1213400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051a95020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35104/0 (socket says 172.21.15.7:35104) 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1213400640 1 -- 172.21.15.7:0/2228471953 learned_addr learned my addr 172.21.15.7:0/2228471953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051a9a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f120be00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 0x7f11f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f120be00640 1 -- 172.21.15.7:0/3641240524 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f11fc0ef890 con 0x7f121410c3e0 2024-06-13T07:05:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f120a000640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051a9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 -- 172.21.15.7:0/2228471953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 msgr2=0x562051a95020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051a95020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1213400640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 0x7f11f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 -- 172.21.15.7:0/2228471953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 msgr2=0x562051a9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051a9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f1212a00640 1 -- 172.21.15.7:0/2228471953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56205170c210 con 0x562051a955f0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.182+0000 7f120a000640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051a9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.183+0000 7f1212a00640 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051a9a0f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1204005fd0 tx=0x7f12040107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44b6f4a37c4112a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.183+0000 7f1203e00640 1 -- 172.21.15.7:0/2228471953 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120401c680 con 0x562051a955f0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.183+0000 7f1203e00640 1 -- 172.21.15.7:0/2228471953 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400be20 con 0x562051a955f0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.183+0000 7f1203e00640 1 -- 172.21.15.7:0/2228471953 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204011ca0 con 0x562051a955f0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 -- 172.21.15.7:0/2228471953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 msgr2=0x562051a9a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051a9a0f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1204005fd0 tx=0x7f12040107a0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f1213400640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 0x7f11f80a5170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x562051a95e50 tx=0x7f120c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 -- 172.21.15.7:0/2228471953 shutdown_connections 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051a9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051a9a0f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2228471953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051a95020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.184+0000 7f121c9e6740 1 -- 172.21.15.7:0/2228471953 >> 172.21.15.7:0/2228471953 conn(0x562051a5cfb0 msgr2=0x562051a90680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.185+0000 7f121c9e6740 1 -- 172.21.15.7:0/2228471953 shutdown_connections 2024-06-13T07:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.186+0000 7f121c9e6740 1 -- 172.21.15.7:0/2228471953 wait complete. 2024-06-13T07:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.187+0000 7f121c9e6740 1 -- start start 2024-06-13T07:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.187+0000 7f121c9e6740 1 Processor -- start 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aa7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a9a760 0x562051aaed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562051a9d710 con 0x562051a955f0 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5620519b30b0 con 0x562051a94c00 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562051a9d890 con 0x562051a9a760 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.188+0000 7f121c9e6740 10 client.0 _ll_register_callbacks cb 0x56205199ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.189+0000 7f120a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.189+0000 7f120a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60494/0 (socket says 172.21.15.7:60494) 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.189+0000 7f120a000640 1 -- 172.21.15.7:0/3206422276 learned_addr learned my addr 172.21.15.7:0/3206422276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.190+0000 7f120a000640 1 -- 172.21.15.7:0/3206422276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a9a760 msgr2=0x562051aaed00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.190+0000 7f120a000640 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a9a760 0x562051aaed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.190+0000 7f120a000640 1 -- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 msgr2=0x562051aa7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.190+0000 7f120a000640 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aa7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.190+0000 7f120a000640 1 -- 172.21.15.7:0/3206422276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620517d8620 con 0x562051a955f0 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.191+0000 7f120a000640 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc0ba310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e125f57981afe88 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.191+0000 7f1200c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:05:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.193+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc049040 con 0x562051a955f0 2024-06-13T07:05:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.193+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f11fc0b8470 con 0x562051a955f0 2024-06-13T07:05:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.194+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120400be20 con 0x562051a955f0 2024-06-13T07:05:04.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.194+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f11fc0bbe00 con 0x562051a955f0 2024-06-13T07:05:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.195+0000 7f121c9e6740 10 client.16140 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.195+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562051aa19e0 con 0x562051a955f0 2024-06-13T07:05:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.198+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400be20 con 0x562051a955f0 2024-06-13T07:05:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.199+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f11fc03b460 con 0x562051a955f0 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.199+0000 7f121c9e6740 10 client.16140 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.199+0000 7f121c9e6740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.200+0000 7f121c9e6740 20 client.16140 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.200+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051aa3140 0x562051aa5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.200+0000 7f121c9e6740 4 client.16140 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.200+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562051a05b50 con 0x562051aa3140 2024-06-13T07:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.202+0000 7f1212a00640 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051aa3140 0x562051aa5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.202+0000 7f1212a00640 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051aa3140 0x562051aa5600 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c12e2522747bced server_cookie=490e6ecf80e18804 in_seq=0 out_seq=0 2024-06-13T07:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.202+0000 7f1200c00640 10 client.16140 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.205+0000 7f1200c00640 1 -- 172.21.15.7:0/3206422276 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562051a05b50 con 0x562051aa3140 2024-06-13T07:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.205+0000 7f1200c00640 10 client.16140 handle_command_reply: tid=0 2024-06-13T07:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.205+0000 7f121c9e6740 1 client.16140 shutdown 2024-06-13T07:05:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.206+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 msgr2=0x562051aa7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.206+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11fc000c00 tx=0x7f11fc0ba310 comp rx=0 tx=0).stop 2024-06-13T07:05:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 shutdown_connections 2024-06-13T07:05:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051aa3140 0x562051aa5600 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a9a760 0x562051aaed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a955f0 0x562051aa7cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 --2- 172.21.15.7:0/3206422276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aa7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f1200c00640 0 client.16140 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.208+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 >> 172.21.15.7:0/3206422276 conn(0x562051a5cfb0 msgr2=0x562051a8f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.209+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 shutdown_connections 2024-06-13T07:05:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.209+0000 7f121c9e6740 1 -- 172.21.15.7:0/3206422276 wait complete. 2024-06-13T07:05:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.210+0000 7f121c9e6740 20 client.16140 trim_cache size 0 max 16384 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.256+0000 7f121c9e6740 1 Processor -- start 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.256+0000 7f121c9e6740 1 -- start start 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a9a760 0x562051a93570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12141b3330 con 0x562051a94c00 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12141b0c00 con 0x562051a9a760 2024-06-13T07:05:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f121c9e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12141aea70 con 0x562051a955f0 2024-06-13T07:05:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f1212a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f1212a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60502/0 (socket says 172.21.15.7:60502) 2024-06-13T07:05:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f1212a00640 1 -- 172.21.15.7:0/2994147413 learned_addr learned my addr 172.21.15.7:0/2994147413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.257+0000 7f1213400640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a9a760 0x562051a93570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f1212a00640 1 -- 172.21.15.7:0/2994147413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 msgr2=0x562051aaecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f120a000640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f1212a00640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f1212a00640 1 -- 172.21.15.7:0/2994147413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a9a760 msgr2=0x562051a93570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f1212a00640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a9a760 0x562051a93570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f1212a00640 1 -- 172.21.15.7:0/2994147413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562051a96400 con 0x562051a94c00 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.258+0000 7f120a000640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.259+0000 7f1212a00640 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1204011010 tx=0x7f1204010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3446719d28ae097 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.260+0000 7f1203400640 1 -- 172.21.15.7:0/2994147413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120402b030 con 0x562051a94c00 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.260+0000 7f1203400640 1 -- 172.21.15.7:0/2994147413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400b810 con 0x562051a94c00 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.260+0000 7f1203400640 1 -- 172.21.15.7:0/2994147413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040048c0 con 0x562051a94c00 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 -- 172.21.15.7:0/2994147413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 msgr2=0x562051a90810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1204011010 tx=0x7f1204010fa0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 -- 172.21.15.7:0/2994147413 shutdown_connections 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a9a760 0x562051a93570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2994147413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a94c00 0x562051a90810 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.261+0000 7f121c9e6740 1 -- 172.21.15.7:0/2994147413 >> 172.21.15.7:0/2994147413 conn(0x562051a5cfb0 msgr2=0x562051a90100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.262+0000 7f121c9e6740 1 -- 172.21.15.7:0/2994147413 shutdown_connections 2024-06-13T07:05:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.263+0000 7f121c9e6740 1 -- 172.21.15.7:0/2994147413 wait complete. 2024-06-13T07:05:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.263+0000 7f121c9e6740 1 -- start start 2024-06-13T07:05:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 Processor -- start 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051aadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12141af0c0 con 0x562051a9a760 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12141b0530 con 0x562051a94c00 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12141af790 con 0x562051a955f0 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.264+0000 7f121c9e6740 10 client.0 _ll_register_callbacks cb 0x7f11fc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35134/0 (socket says 172.21.15.7:35134) 2024-06-13T07:05:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 -- 172.21.15.7:0/2979916343 learned_addr learned my addr 172.21.15.7:0/2979916343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f120a000640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1213400640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051aadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 -- 172.21.15.7:0/2979916343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 msgr2=0x562051aaac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 -- 172.21.15.7:0/2979916343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 msgr2=0x562051aadbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051aadbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.265+0000 7f1212a00640 1 -- 172.21.15.7:0/2979916343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562051aae1d0 con 0x562051a94c00 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.267+0000 7f1213400640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051aadbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.267+0000 7f1212a00640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f12040107e0 tx=0x7f1204010e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2734ab9b490a9751 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.267+0000 7f1200c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.267+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204034070 con 0x562051a94c00 2024-06-13T07:05:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.267+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120400b810 con 0x562051a94c00 2024-06-13T07:05:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.269+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1204005250 con 0x562051a94c00 2024-06-13T07:05:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.271+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1204040050 con 0x562051a94c00 2024-06-13T07:05:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.271+0000 7f121c9e6740 10 client.26735 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.271+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562051b264d0 con 0x562051a94c00 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400b810 con 0x562051a94c00 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f12040119a0 con 0x562051a94c00 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 10 client.26735 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 10 client.26735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 20 client.26735 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051b26c80 0x562051b290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 4 client.26735 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620519ffea0 con 0x562051b26c80 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.272+0000 7f120a000640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051b26c80 0x562051b290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.273+0000 7f1200c00640 10 client.26735 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.274+0000 7f120a000640 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051b26c80 0x562051b290a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4230d1ea1de0ee server_cookie=5ea6d9f9bdc757f4 in_seq=0 out_seq=0 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.305+0000 7f1200c00640 1 -- 172.21.15.7:0/2979916343 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5620519ffea0 con 0x562051b26c80 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.305+0000 7f1200c00640 10 client.26735 handle_command_reply: tid=0 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.305+0000 7f121c9e6740 1 client.26735 shutdown 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.305+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 msgr2=0x562051aaa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.305+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f12040107e0 tx=0x7f1204010e10 comp rx=0 tx=0).stop 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 shutdown_connections 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x562051b26c80 0x562051b290a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562051a9a760 0x562051aadbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562051a955f0 0x562051aaac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 --2- 172.21.15.7:0/2979916343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562051a94c00 0x562051aaa740 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 >> 172.21.15.7:0/2979916343 conn(0x562051a5cfb0 msgr2=0x562051a93570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.306+0000 7f1200c00640 0 client.26735 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.307+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 shutdown_connections 2024-06-13T07:05:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.307+0000 7f121c9e6740 1 -- 172.21.15.7:0/2979916343 wait complete. 2024-06-13T07:05:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.307+0000 7f121c9e6740 20 client.26735 trim_cache size 0 max 16384 2024-06-13T07:05:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:05:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:05:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:05:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 msgr2=0x7f11f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 0x7f11f80a5170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x562051a95e50 tx=0x7f120c004ab0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 msgr2=0x7f121410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f11fc031440 tx=0x7f11fc032ce0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 shutdown_connections 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f11f80a2cb0 0x7f11f80a5170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f121410ed60 0x7f121410a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f121410c3e0 0x7f121410a050 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 --2- 172.21.15.7:0/3641240524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1214109ab0 0x7f121410b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 >> 172.21.15.7:0/3641240524 conn(0x7f121406ebd0 msgr2=0x7f1214108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.308+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 shutdown_connections 2024-06-13T07:05:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:04.310+0000 7f1219c00640 1 -- 172.21.15.7:0/3641240524 wait complete. 2024-06-13T07:05:04.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:04 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:04 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:04 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:04 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:04 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.669 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:04 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:05:04.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:05:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:05:05.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:05 smithi189 ceph-mon[36616]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:05:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:05 smithi175 ceph-mon[36886]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:05:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:05 smithi007 ceph-mon[32077]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-06-13T07:05:07.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:07 smithi175 ceph-mon[36886]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.0 MiB/s wr, 495 op/s 2024-06-13T07:05:07.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:07 smithi007 ceph-mon[32077]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.0 MiB/s wr, 495 op/s 2024-06-13T07:05:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:07 smithi189 ceph-mon[36616]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.0 MiB/s wr, 495 op/s 2024-06-13T07:05:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 545.0 s, 469.8 tps, lat 221.893 ms stddev 263.775 2024-06-13T07:05:08.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4ef400700 1 --2- 172.21.15.7:0/24480951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:08.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 -- 172.21.15.7:0/24480951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0107840 msgr2=0x7fd4f010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 --2- 172.21.15.7:0/24480951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0107840 0x7fd4f010f3c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800b0d0 tx=0x7fd4e800b3e0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 -- 172.21.15.7:0/24480951 shutdown_connections 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 --2- 172.21.15.7:0/24480951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0107840 0x7fd4f010f3c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 --2- 172.21.15.7:0/24480951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0102740 0x7fd4f0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 --2- 172.21.15.7:0/24480951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 -- 172.21.15.7:0/24480951 >> 172.21.15.7:0/24480951 conn(0x7fd4f00795f0 msgr2=0x7fd4f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 -- 172.21.15.7:0/24480951 shutdown_connections 2024-06-13T07:05:08.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.740+0000 7fd4f7000700 1 -- 172.21.15.7:0/24480951 wait complete. 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 Processor -- start 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 -- start start 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0102740 0x7fd4f019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01128e0 con 0x7fd4f01030f0 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0112760 con 0x7fd4f0102740 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f7000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0112a60 con 0x7fd4f0107840 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4efe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0102740 0x7fd4f019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56190/0 (socket says 172.21.15.7:56190) 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.742+0000 7fd4f5000700 1 -- 172.21.15.7:0/3085696189 learned_addr learned my addr 172.21.15.7:0/3085696189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4ef400700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 -- 172.21.15.7:0/3085696189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0102740 msgr2=0x7fd4f019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0102740 0x7fd4f019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 -- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 msgr2=0x7fd4f019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 -- 172.21.15.7:0/3085696189 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f019fe20 con 0x7fd4f0107840 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800b820 tx=0x7fd4e8007ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4ecc00700 1 -- 172.21.15.7:0/3085696189 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8021020 con 0x7fd4f0107840 2024-06-13T07:05:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.743+0000 7fd4ecc00700 1 -- 172.21.15.7:0/3085696189 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e8004c50 con 0x7fd4f0107840 2024-06-13T07:05:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.744+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f01a00b0 con 0x7fd4f0107840 2024-06-13T07:05:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.744+0000 7fd4ecc00700 1 -- 172.21.15.7:0/3085696189 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e801d760 con 0x7fd4f0107840 2024-06-13T07:05:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.744+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a7990 con 0x7fd4f0107840 2024-06-13T07:05:08.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.747+0000 7fd4ecc00700 1 -- 172.21.15.7:0/3085696189 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd4e8024020 con 0x7fd4f0107840 2024-06-13T07:05:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.749+0000 7fd4ecc00700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 0x7fd4d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.749+0000 7fd501989b80 1 Processor -- start 2024-06-13T07:05:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.749+0000 7fd4ecc00700 1 -- 172.21.15.7:0/3085696189 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd4e800f020 con 0x7fd4f0107840 2024-06-13T07:05:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.749+0000 7fd4efe00700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 0x7fd4d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 -- start start 2024-06-13T07:05:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6abed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580b69cda10 con 0x5580b6abf240 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5580b6a10b00 con 0x5580b6ac54b0 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd501989b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580b69cb8f0 con 0x5580b6ac4b90 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6abed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6abed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35176/0 (socket says 172.21.15.7:35176) 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.750+0000 7fd4ef400700 1 -- 172.21.15.7:0/3630435091 learned_addr learned my addr 172.21.15.7:0/3630435091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 -- 172.21.15.7:0/3630435091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 msgr2=0x5580b6ac4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 -- 172.21.15.7:0/3630435091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 msgr2=0x5580b6abed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6abed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 -- 172.21.15.7:0/3630435091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b69b8f90 con 0x5580b6abf240 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4efe00700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 0x7fd4d40a5160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.751+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abf680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007c00 tx=0x7fd4e800cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8d0e0aa51f15e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.752+0000 7fd4e5600700 1 -- 172.21.15.7:0/3630435091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a57a0 con 0x5580b6abf240 2024-06-13T07:05:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.752+0000 7fd4e5600700 1 -- 172.21.15.7:0/3630435091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e809ac60 con 0x5580b6abf240 2024-06-13T07:05:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.752+0000 7fd4e5600700 1 -- 172.21.15.7:0/3630435091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a5310 con 0x5580b6abf240 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 -- 172.21.15.7:0/3630435091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 msgr2=0x5580b6abf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abf680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007c00 tx=0x7fd4e800cfa0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 -- 172.21.15.7:0/3630435091 shutdown_connections 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abf680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 --2- 172.21.15.7:0/3630435091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6abed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 -- 172.21.15.7:0/3630435091 >> 172.21.15.7:0/3630435091 conn(0x5580b69eb170 msgr2=0x5580b6ab9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 -- 172.21.15.7:0/3630435091 shutdown_connections 2024-06-13T07:05:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.753+0000 7fd501989b80 1 -- 172.21.15.7:0/3630435091 wait complete. 2024-06-13T07:05:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 -- start start 2024-06-13T07:05:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 Processor -- start 2024-06-13T07:05:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6acb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5580b69acf60 con 0x5580b6abf240 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5580b6a0cac0 con 0x5580b6ac54b0 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580b69cda10 con 0x5580b6ac4b90 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.754+0000 7fd501989b80 10 client.0 _ll_register_callbacks cb 0x5580b69a6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4efe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6acb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35184/0 (socket says 172.21.15.7:35184) 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 -- 172.21.15.7:0/2811335142 learned_addr learned my addr 172.21.15.7:0/2811335142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4ef400700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 -- 172.21.15.7:0/2811335142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 msgr2=0x5580b6ac8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 -- 172.21.15.7:0/2811335142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 msgr2=0x5580b6acb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6acb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 -- 172.21.15.7:0/2811335142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b695f8d0 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4f5000700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cddd0 tx=0x7fd4e8009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47303084bcc3236b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4da000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.755+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80ba070 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd4e80a5510 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e809ac60 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd4e80bb260 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 10 client.26747 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5580b6ade9e0 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e809ac60 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fd4e80d0070 con 0x5580b6ac54b0 2024-06-13T07:05:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 10 client.26747 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 20 client.26747 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6ae0dd0 0x5580b6ae31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 4 client.26747 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.756+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b6a1e000 con 0x5580b6ae0dd0 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.757+0000 7fd4efe00700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6ae0dd0 0x5580b6ae31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.757+0000 7fd4efe00700 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6ae0dd0 0x5580b6ae31c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5a2726be19bd0a server_cookie=f08d0c6eb625ec52 in_seq=0 out_seq=0 2024-06-13T07:05:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.757+0000 7fd4da000700 10 client.26747 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.758+0000 7fd4da000700 1 -- 172.21.15.7:0/2811335142 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5580b6a1e000 con 0x5580b6ae0dd0 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.758+0000 7fd4da000700 10 client.26747 handle_command_reply: tid=0 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.758+0000 7fd501989b80 1 client.26747 shutdown 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.759+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 msgr2=0x5580b6ac8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.759+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cddd0 tx=0x7fd4e8009d80 comp rx=0 tx=0).stop 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 shutdown_connections 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6ae0dd0 0x5580b6ae31c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd4da000700 0 client.26747 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac54b0 0x5580b6ac8a70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6acb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 --2- 172.21.15.7:0/2811335142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac4b90 0x5580b6ac8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 >> 172.21.15.7:0/2811335142 conn(0x5580b69eb170 msgr2=0x5580b6ab3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.760+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 shutdown_connections 2024-06-13T07:05:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.763+0000 7fd501989b80 1 -- 172.21.15.7:0/2811335142 wait complete. 2024-06-13T07:05:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.763+0000 7fd501989b80 20 client.26747 trim_cache size 0 max 16384 2024-06-13T07:05:08.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 Processor -- start 2024-06-13T07:05:08.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 -- start start 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ab4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01ae700 con 0x5580b6abf240 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01abf90 con 0x5580b6ac4b90 2024-06-13T07:05:08.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.791+0000 7fd501989b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d550 con 0x5580b6ac54b0 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ab4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35194/0 (socket says 172.21.15.7:35194) 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 -- 172.21.15.7:0/2701360164 learned_addr learned my addr 172.21.15.7:0/2701360164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ab4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:56224/0 (socket says 172.21.15.7:56224) 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4efe00700 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 -- 172.21.15.7:0/2701360164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 msgr2=0x5580b6ab4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ab4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 -- 172.21.15.7:0/2701360164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 msgr2=0x5580b6abed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.792+0000 7fd4ef400700 1 -- 172.21.15.7:0/2701360164 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b6ac6aa0 con 0x5580b6ac4b90 2024-06-13T07:05:08.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.793+0000 7fd4ef400700 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ea0 tx=0x7fd4dc00eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf20824982156fb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd4e4c00700 1 -- 172.21.15.7:0/2701360164 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc012db0 con 0x5580b6ac4b90 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd4e4c00700 1 -- 172.21.15.7:0/2701360164 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4dc004510 con 0x5580b6ac4b90 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd4e4c00700 1 -- 172.21.15.7:0/2701360164 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005350 con 0x5580b6ac4b90 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 -- 172.21.15.7:0/2701360164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 msgr2=0x5580b6aba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005ea0 tx=0x7fd4dc00eec0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 -- 172.21.15.7:0/2701360164 shutdown_connections 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ab4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6abf240 0x5580b6abed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 --2- 172.21.15.7:0/2701360164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6ac4b90 0x5580b6aba160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.794+0000 7fd501989b80 1 -- 172.21.15.7:0/2701360164 >> 172.21.15.7:0/2701360164 conn(0x5580b69eb170 msgr2=0x5580b6ac5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.795+0000 7fd501989b80 1 -- 172.21.15.7:0/2701360164 shutdown_connections 2024-06-13T07:05:08.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.795+0000 7fd501989b80 1 -- 172.21.15.7:0/2701360164 wait complete. 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.795+0000 7fd501989b80 1 -- start start 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 Processor -- start 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 0x5580b6ad14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ae89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01aa450 con 0x5580b6ac4b90 2024-06-13T07:05:08.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01ab8c0 con 0x5580b6abf240 2024-06-13T07:05:08.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01aab20 con 0x5580b6ac54b0 2024-06-13T07:05:08.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.796+0000 7fd501989b80 10 client.0 _ll_register_callbacks cb 0x5580b69a6380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.798+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.798+0000 7fd4efe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 0x5580b6ad14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.798+0000 7fd4f5000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ae89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.798+0000 7fd4ef400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60564/0 (socket says 172.21.15.7:60564) 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.798+0000 7fd4ef400700 1 -- 172.21.15.7:0/3947194830 learned_addr learned my addr 172.21.15.7:0/3947194830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.799+0000 7fd4ef400700 1 -- 172.21.15.7:0/3947194830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 msgr2=0x5580b6ae89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.799+0000 7fd4ef400700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ae89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.799+0000 7fd4ef400700 1 -- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 msgr2=0x5580b6ad14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.799+0000 7fd4ef400700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 0x5580b6ad14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.799+0000 7fd4ef400700 1 -- 172.21.15.7:0/3947194830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b6ae9260 con 0x5580b6ac4b90 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4ef400700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00efd0 tx=0x7fd4dc004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b98227893a9260 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4da000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:05:08.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4f5000700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ae89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:08.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4efe00700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 0x5580b6ad14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:08.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc02a060 con 0x5580b6ac4b90 2024-06-13T07:05:08.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.800+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fd4dc005020 con 0x5580b6ac4b90 2024-06-13T07:05:08.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.801+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4dc004510 con 0x5580b6ac4b90 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.801+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc031050 con 0x5580b6ac4b90 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.801+0000 7fd501989b80 10 client.16158 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.801+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5580b6b4bb00 con 0x5580b6ac4b90 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4dc004510 con 0x5580b6ac4b90 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fd4dc025c20 con 0x5580b6ac4b90 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 20 client.16158 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6b4c080 0x5580b6b4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 4 client.16158 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580b6a18580 con 0x5580b6b4c080 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.802+0000 7fd4efe00700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6b4c080 0x5580b6b4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.803+0000 7fd4efe00700 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6b4c080 0x5580b6b4e4a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553d692f1094d29e server_cookie=c89c76230db16d62 in_seq=0 out_seq=0 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd4da000700 10 client.16158 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd4da000700 1 -- 172.21.15.7:0/3947194830 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5580b6a18580 con 0x5580b6b4c080 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd4da000700 10 client.16158 handle_command_reply: tid=0 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 client.16158 shutdown 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 msgr2=0x5580b6ae8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00efd0 tx=0x7fd4dc004cb0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 shutdown_connections 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5580b6b4c080 0x5580b6b4e4a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b6ac54b0 0x5580b6ae89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5580b6abf240 0x5580b6ad14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 --2- 172.21.15.7:0/3947194830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580b6ac4b90 0x5580b6ae8450 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 >> 172.21.15.7:0/3947194830 conn(0x5580b69eb170 msgr2=0x5580b6ac3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.805+0000 7fd4da000700 0 client.16158 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.806+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 shutdown_connections 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.806+0000 7fd501989b80 1 -- 172.21.15.7:0/3947194830 wait complete. 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.806+0000 7fd501989b80 20 client.16158 trim_cache size 0 max 16384 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:05:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "fc64ad60-06e1-4bc3-9a5a-a91673d2c1ec": { 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "fc64ad60-06e1-4bc3-9a5a-a91673d2c1ec", 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:05:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 msgr2=0x7fd4d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 0x7fd4d40a5160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0009190 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 msgr2=0x7fd4f019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800b820 tx=0x7fd4e8007ae0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 shutdown_connections 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f0107840 0x7fd4f019f6e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fd4d40a2ca0 0x7fd4d40a5160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0102740 0x7fd4f019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 --2- 172.21.15.7:0/3085696189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f01030f0 0x7fd4f019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.807+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 >> 172.21.15.7:0/3085696189 conn(0x7fd4f00795f0 msgr2=0x7fd4f0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.808+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 shutdown_connections 2024-06-13T07:05:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:08.808+0000 7fd4f7000700 1 -- 172.21.15.7:0/3085696189 wait complete. 2024-06-13T07:05:09.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:09 smithi175 ceph-mon[36886]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 533 op/s 2024-06-13T07:05:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:09 smithi007 ceph-mon[32077]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 533 op/s 2024-06-13T07:05:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:09 smithi189 ceph-mon[36616]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 533 op/s 2024-06-13T07:05:10.848 INFO:tasks.cephfs.filesystem:scrub status for tag:fc64ad60-06e1-4bc3-9a5a-a91673d2c1ec - {'path': '/', 'tag': 'fc64ad60-06e1-4bc3-9a5a-a91673d2c1ec', 'options': 'recursive,force'} 2024-06-13T07:05:11.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:11 smithi175 ceph-mon[36886]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.4 MiB/s wr, 586 op/s 2024-06-13T07:05:11.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:11 smithi007 ceph-mon[32077]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.4 MiB/s wr, 586 op/s 2024-06-13T07:05:11.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:11 smithi189 ceph-mon[36616]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.4 MiB/s wr, 586 op/s 2024-06-13T07:05:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 550.0 s, 542.2 tps, lat 188.798 ms stddev 219.491 2024-06-13T07:05:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:13 smithi175 ceph-mon[36886]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.5 MiB/s wr, 494 op/s 2024-06-13T07:05:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:13 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:05:13.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:13 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:05:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:13 smithi007 ceph-mon[32077]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.5 MiB/s wr, 494 op/s 2024-06-13T07:05:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:13 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:05:13.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:13 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:05:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:13 smithi189 ceph-mon[36616]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.5 MiB/s wr, 494 op/s 2024-06-13T07:05:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:13 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:05:13.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:13 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:05:14.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:14 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:05:14.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:14 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:05:14.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:14 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:05:15.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:15 smithi007 ceph-mon[32077]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 571 op/s 2024-06-13T07:05:15.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:15 smithi189 ceph-mon[36616]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 571 op/s 2024-06-13T07:05:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:15 smithi175 ceph-mon[36886]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 571 op/s 2024-06-13T07:05:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.580+0000 7f9677400640 1 -- 172.21.15.7:0/47294550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9670100e20 msgr2=0x7f96701012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.581+0000 7f9677400640 1 --2- 172.21.15.7:0/47294550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9670100e20 0x7f96701012a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9660009900 tx=0x7f966002a410 comp rx=0 tx=0).stop 2024-06-13T07:05:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.582+0000 7f9677400640 1 -- 172.21.15.7:0/47294550 shutdown_connections 2024-06-13T07:05:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.582+0000 7f9677400640 1 --2- 172.21.15.7:0/47294550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96701017e0 0x7f9670108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.583+0000 7f9677400640 1 --2- 172.21.15.7:0/47294550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9670100e20 0x7f96701012a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.583+0000 7f9677400640 1 --2- 172.21.15.7:0/47294550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96700ffc30 0x7f9670100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.583+0000 7f9677400640 1 -- 172.21.15.7:0/47294550 >> 172.21.15.7:0/47294550 conn(0x7f96700fb1c0 msgr2=0x7f96700fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.583+0000 7f9677400640 1 -- 172.21.15.7:0/47294550 shutdown_connections 2024-06-13T07:05:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.583+0000 7f9677400640 1 -- 172.21.15.7:0/47294550 wait complete. 2024-06-13T07:05:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.584+0000 7f9677400640 1 Processor -- start 2024-06-13T07:05:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.585+0000 7f9677400640 1 -- start start 2024-06-13T07:05:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.587+0000 7f9677400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.587+0000 7f9677400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 0x7f967010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.587+0000 7f9674a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.587+0000 7f9674a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46526/0 (socket says 172.21.15.7:46526) 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9677400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96701017e0 0x7f96701ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9677400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9670113cd0 con 0x7f9670100e20 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 -- 172.21.15.7:0/2485040542 learned_addr learned my addr 172.21.15.7:0/2485040542 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9670113b50 con 0x7f96701017e0 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9670113e50 con 0x7f96700ffc30 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 -- 172.21.15.7:0/2485040542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96701017e0 msgr2=0x7f96701ae3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96701017e0 0x7f96701ae3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 -- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 msgr2=0x7f967010eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f966fe00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 0x7f967010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 0x7f967010eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.588+0000 7f9674a00640 1 -- 172.21.15.7:0/2485040542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9670111c90 con 0x7f96700ffc30 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.589+0000 7f966fe00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 0x7f967010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.589+0000 7f9674a00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f966400d770 tx=0x7f966400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.589+0000 7f966d600640 1 -- 172.21.15.7:0/2485040542 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664017070 con 0x7f96700ffc30 2024-06-13T07:05:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.590+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96701ae940 con 0x7f96700ffc30 2024-06-13T07:05:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.590+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96701aeee0 con 0x7f96700ffc30 2024-06-13T07:05:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.590+0000 7f966d600640 1 -- 172.21.15.7:0/2485040542 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9664009c30 con 0x7f96700ffc30 2024-06-13T07:05:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.590+0000 7f966d600640 1 -- 172.21.15.7:0/2485040542 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664003dd0 con 0x7f96700ffc30 2024-06-13T07:05:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.599+0000 7f967a164740 1 Processor -- start 2024-06-13T07:05:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.599+0000 7f967a164740 1 -- start start 2024-06-13T07:05:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.599+0000 7f967a164740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092e4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.599+0000 7f967a164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092e9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.599+0000 7f967a164740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f967a164740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef09236420 con 0x55ef092e3c10 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f967a164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef091ddc50 con 0x55ef092e4600 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f967a164740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef092323e0 con 0x55ef092e9770 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f9675400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f966fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f966fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46536/0 (socket says 172.21.15.7:46536) 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f966fe00640 1 -- 172.21.15.7:0/2519893870 learned_addr learned my addr 172.21.15.7:0/2519893870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.600+0000 7f9675400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092e4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34394/0 (socket says 172.21.15.7:34394) 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.601+0000 7f966fe00640 1 -- 172.21.15.7:0/2519893870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 msgr2=0x55ef092e9100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.601+0000 7f966fe00640 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092e9100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.601+0000 7f966fe00640 1 -- 172.21.15.7:0/2519893870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 msgr2=0x55ef092e4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.601+0000 7f966fe00640 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092e4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.601+0000 7f966fe00640 1 -- 172.21.15.7:0/2519893870 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef08f4a210 con 0x55ef092e9770 2024-06-13T07:05:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.604+0000 7f966d600640 1 -- 172.21.15.7:0/2485040542 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9664010460 con 0x7f96700ffc30 2024-06-13T07:05:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.604+0000 7f966fe00640 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9660005ab0 tx=0x7f9660004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec986a5862b5300 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.604+0000 7f965dc00640 1 -- 172.21.15.7:0/2519893870 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966003f070 con 0x55ef092e9770 2024-06-13T07:05:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.605+0000 7f965dc00640 1 -- 172.21.15.7:0/2519893870 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660005520 con 0x55ef092e9770 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.605+0000 7f965dc00640 1 -- 172.21.15.7:0/2519893870 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966003f070 con 0x55ef092e9770 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 -- 172.21.15.7:0/2519893870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 msgr2=0x55ef092ebc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9660005ab0 tx=0x7f9660004170 comp rx=0 tx=0).stop 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 -- 172.21.15.7:0/2519893870 shutdown_connections 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092ebc00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092e9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 --2- 172.21.15.7:0/2519893870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092e4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 -- 172.21.15.7:0/2519893870 >> 172.21.15.7:0/2519893870 conn(0x55ef0929afb0 msgr2=0x55ef092ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.606+0000 7f967a164740 1 -- 172.21.15.7:0/2519893870 shutdown_connections 2024-06-13T07:05:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.608+0000 7f966d600640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 0x7f96580a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.608+0000 7f966d600640 1 -- 172.21.15.7:0/2485040542 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f96640cecd0 con 0x7f96700ffc30 2024-06-13T07:05:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.609+0000 7f966fe00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 0x7f96580a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.609+0000 7f967a164740 1 -- 172.21.15.7:0/2519893870 wait complete. 2024-06-13T07:05:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.610+0000 7f967a164740 1 -- start start 2024-06-13T07:05:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.611+0000 7f967a164740 1 Processor -- start 2024-06-13T07:05:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.613+0000 7f967a164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092ede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.613+0000 7f967a164740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.613+0000 7f967a164740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 0x55ef092f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.613+0000 7f966fe00640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 0x7f96580a50a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9660000c00 tx=0x7f966002a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.614+0000 7f9674a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.614+0000 7f9674a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092ede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34374/0 (socket says 172.21.15.7:34374) 2024-06-13T07:05:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.614+0000 7f966fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.615+0000 7f966fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46538/0 (socket says 172.21.15.7:46538) 2024-06-13T07:05:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.615+0000 7f9674a00640 1 -- 172.21.15.7:0/4257570914 learned_addr learned my addr 172.21.15.7:0/4257570914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.615+0000 7f9675400640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 0x55ef092f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.616+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef092ec6c0 con 0x55ef092e9770 2024-06-13T07:05:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.616+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ef091f10b0 con 0x55ef092e3c10 2024-06-13T07:05:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.616+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef092ec840 con 0x55ef092e4600 2024-06-13T07:05:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.617+0000 7f966fe00640 1 -- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 msgr2=0x55ef092ede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.617+0000 7f966fe00640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092ede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.617+0000 7f966fe00640 1 -- 172.21.15.7:0/4257570914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 msgr2=0x55ef092f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.617+0000 7f966fe00640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 0x55ef092f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.617+0000 7f966fe00640 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef09016620 con 0x55ef092e4600 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.618+0000 7f9675400640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 0x55ef092f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.618+0000 7f967a164740 10 client.0 _ll_register_callbacks cb 0x55ef091dcf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.618+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9660005520 con 0x55ef092e4600 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.619+0000 7f966fe00640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9660009900 tx=0x7f9660004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e8d7460579d50b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.619+0000 7f9652000640 10 client.26698 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.620+0000 7f9652000640 1 -- 172.21.15.7:0/4257570914 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660064050 con 0x55ef092e4600 2024-06-13T07:05:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.620+0000 7f9652000640 1 -- 172.21.15.7:0/4257570914 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9660004360 con 0x55ef092e4600 2024-06-13T07:05:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.620+0000 7f9652000640 1 -- 172.21.15.7:0/4257570914 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f966005a070 con 0x55ef092e4600 2024-06-13T07:05:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.620+0000 7f967a164740 10 client.26698 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.621+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55ef092ec6f0 con 0x55ef092e4600 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.621+0000 7f9652000640 1 -- 172.21.15.7:0/4257570914 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f96600316b0 con 0x55ef092e4600 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.621+0000 7f967a164740 10 client.26698 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.621+0000 7f967a164740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.621+0000 7f967a164740 20 client.26698 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.622+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef092f9ed0 0x55ef092fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.622+0000 7f967a164740 4 client.26698 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.622+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef09243b50 con 0x55ef092f9ed0 2024-06-13T07:05:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.622+0000 7f9674a00640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef092f9ed0 0x55ef092fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.623+0000 7f9674a00640 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef092f9ed0 0x55ef092fc390 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01857243ba34dd0 server_cookie=fa303f17023ed46a in_seq=0 out_seq=0 2024-06-13T07:05:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.623+0000 7f9652000640 10 client.26698 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.624+0000 7f9652000640 1 -- 172.21.15.7:0/4257570914 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef09243b50 con 0x55ef092f9ed0 2024-06-13T07:05:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.625+0000 7f9652000640 10 client.26698 handle_command_reply: tid=0 2024-06-13T07:05:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.625+0000 7f967a164740 1 client.26698 shutdown 2024-06-13T07:05:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.626+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 msgr2=0x55ef092ee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.626+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9660009900 tx=0x7f9660004170 comp rx=0 tx=0).stop 2024-06-13T07:05:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 shutdown_connections 2024-06-13T07:05:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef092f9ed0 0x55ef092fc390 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e9770 0x55ef092f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e4600 0x55ef092ee3b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 --2- 172.21.15.7:0/4257570914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 >> 172.21.15.7:0/4257570914 conn(0x55ef0929afb0 msgr2=0x55ef092cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.627+0000 7f9652000640 0 client.26698 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.628+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 shutdown_connections 2024-06-13T07:05:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.628+0000 7f967a164740 1 -- 172.21.15.7:0/4257570914 wait complete. 2024-06-13T07:05:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.628+0000 7f967a164740 20 client.26698 trim_cache size 0 max 16384 2024-06-13T07:05:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 Processor -- start 2024-06-13T07:05:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 -- start start 2024-06-13T07:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9654012280 con 0x55ef092e3c10 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f965400fb50 con 0x55ef092e4600 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.655+0000 7f967a164740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f965400d9a0 con 0x55ef092e9770 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f9674a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:34384/0 (socket says 172.21.15.7:34384) 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 -- 172.21.15.7:0/1733267029 learned_addr learned my addr 172.21.15.7:0/1733267029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f9675400640 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 -- 172.21.15.7:0/1733267029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 msgr2=0x55ef092d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 -- 172.21.15.7:0/1733267029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 msgr2=0x55ef092d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f966fe00640 1 -- 172.21.15.7:0/1733267029 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef092cf330 con 0x55ef092e4600 2024-06-13T07:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.656+0000 7f9675400640 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.657+0000 7f966fe00640 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9660031440 tx=0x7f9660055040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7905f07185c359b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.657+0000 7f965d200640 1 -- 172.21.15.7:0/1733267029 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660064070 con 0x55ef092e4600 2024-06-13T07:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.657+0000 7f965d200640 1 -- 172.21.15.7:0/1733267029 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660005520 con 0x55ef092e4600 2024-06-13T07:05:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.657+0000 7f965d200640 1 -- 172.21.15.7:0/1733267029 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966003f070 con 0x55ef092e4600 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- 172.21.15.7:0/1733267029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 msgr2=0x55ef092f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9660031440 tx=0x7f9660055040 comp rx=0 tx=0).stop 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- 172.21.15.7:0/1733267029 shutdown_connections 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e4600 0x55ef092f53b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 --2- 172.21.15.7:0/1733267029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e3c10 0x55ef092d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- 172.21.15.7:0/1733267029 >> 172.21.15.7:0/1733267029 conn(0x55ef0929afb0 msgr2=0x55ef092e5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- 172.21.15.7:0/1733267029 shutdown_connections 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- 172.21.15.7:0/1733267029 wait complete. 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.658+0000 7f967a164740 1 -- start start 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 Processor -- start 2024-06-13T07:05:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e4600 0x55ef092f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f965400e010 con 0x55ef092e4600 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f965400f480 con 0x55ef092e3c10 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f965400e6e0 con 0x55ef092e9770 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.659+0000 7f967a164740 10 client.0 _ll_register_callbacks cb 0x7f96680094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:46560/0 (socket says 172.21.15.7:46560) 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 -- 172.21.15.7:0/2929041153 learned_addr learned my addr 172.21.15.7:0/2929041153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9674a00640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 -- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 msgr2=0x55ef092f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 -- 172.21.15.7:0/2929041153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e4600 msgr2=0x55ef092f1c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f966fe00640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e4600 0x55ef092f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e4600 0x55ef092f1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.660+0000 7f9675400640 1 -- 172.21.15.7:0/2929041153 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef092f2800 con 0x55ef092e9770 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f9675400640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9668013910 tx=0x7f9668013dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d17c388b1ef171a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f9652000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668011c10 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9660005520 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9668011db0 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9668003970 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f967a164740 10 client.26704 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.661+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ef09363ed0 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.662+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660005520 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.662+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f966800f040 con 0x55ef092e9770 2024-06-13T07:05:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 10 client.26704 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 20 client.26704 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef09364880 0x55ef09366ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 4 client.26704 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.663+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef0923dea0 con 0x55ef09364880 2024-06-13T07:05:17.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:17 smithi007 ceph-mon[32077]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-13T07:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.664+0000 7f9674a00640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef09364880 0x55ef09366ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.664+0000 7f9674a00640 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef09364880 0x55ef09366ca0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3617ab583f868bde server_cookie=87b7574e0cae7d4d in_seq=0 out_seq=0 2024-06-13T07:05:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.664+0000 7f9652000640 10 client.26704 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.686+0000 7f9652000640 1 -- 172.21.15.7:0/2929041153 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef0923dea0 con 0x55ef09364880 2024-06-13T07:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.686+0000 7f9652000640 10 client.26704 handle_command_reply: tid=0 2024-06-13T07:05:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.686+0000 7f967a164740 1 client.26704 shutdown 2024-06-13T07:05:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 msgr2=0x55ef092f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9668013910 tx=0x7f9668013dd0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 shutdown_connections 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ef09364880 0x55ef09366ca0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef092e9770 0x55ef092f2160 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef092e4600 0x55ef092f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f9652000640 0 client.26704 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 --2- 172.21.15.7:0/2929041153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ef092e3c10 0x55ef092f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.687+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 >> 172.21.15.7:0/2929041153 conn(0x55ef0929afb0 msgr2=0x55ef092ede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.688+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 shutdown_connections 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.688+0000 7f967a164740 1 -- 172.21.15.7:0/2929041153 wait complete. 2024-06-13T07:05:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.688+0000 7f967a164740 20 client.26704 trim_cache size 0 max 16384 2024-06-13T07:05:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:05:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:05:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:05:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 msgr2=0x7f96580a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 0x7f96580a50a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9660000c00 tx=0x7f966002a410 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 msgr2=0x7f967010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f966400d770 tx=0x7f966400dc30 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 shutdown_connections 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f96580a2be0 0x7f96580a50a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96701017e0 0x7f96701ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670100e20 0x7f967010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 --2- 172.21.15.7:0/2485040542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96700ffc30 0x7f967010e620 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 >> 172.21.15.7:0/2485040542 conn(0x7f96700fb1c0 msgr2=0x7f9670104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 shutdown_connections 2024-06-13T07:05:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:17.689+0000 7f9677400640 1 -- 172.21.15.7:0/2485040542 wait complete. 2024-06-13T07:05:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:17 smithi189 ceph-mon[36616]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-13T07:05:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:17 smithi175 ceph-mon[36886]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.8 MiB/s wr, 486 op/s 2024-06-13T07:05:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 555.0 s, 689.7 tps, lat 138.964 ms stddev 175.095 2024-06-13T07:05:19.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:19 smithi007 ceph-mon[32077]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 496 op/s 2024-06-13T07:05:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:19 smithi189 ceph-mon[36616]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 496 op/s 2024-06-13T07:05:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:19 smithi175 ceph-mon[36886]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 496 op/s 2024-06-13T07:05:21.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:21 smithi007 ceph-mon[32077]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 567 op/s 2024-06-13T07:05:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:21 smithi189 ceph-mon[36616]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 567 op/s 2024-06-13T07:05:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:21 smithi175 ceph-mon[36886]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.0 MiB/s wr, 567 op/s 2024-06-13T07:05:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 560.0 s, 497.4 tps, lat 201.260 ms stddev 322.884 2024-06-13T07:05:23.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:23 smithi007 ceph-mon[32077]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-13T07:05:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:23 smithi189 ceph-mon[36616]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-13T07:05:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:23 smithi175 ceph-mon[36886]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-06-13T07:05:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:25 smithi189 ceph-mon[36616]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-06-13T07:05:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:25 smithi175 ceph-mon[36886]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-06-13T07:05:25.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:25 smithi007 ceph-mon[32077]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 550 op/s 2024-06-13T07:05:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:27 smithi189 ceph-mon[36616]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-06-13T07:05:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:27 smithi175 ceph-mon[36886]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-06-13T07:05:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 565.0 s, 502.6 tps, lat 203.105 ms stddev 247.550 2024-06-13T07:05:27.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:27 smithi007 ceph-mon[32077]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-06-13T07:05:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:29 smithi189 ceph-mon[36616]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-06-13T07:05:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:29 smithi175 ceph-mon[36886]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-06-13T07:05:29.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:29 smithi007 ceph-mon[32077]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-06-13T07:05:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.871+0000 7f0381600640 1 -- 172.21.15.7:0/2865039206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c103c30 msgr2=0x7f037c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.871+0000 7f0381600640 1 --2- 172.21.15.7:0/2865039206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c103c30 0x7f037c104050 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f036c009860 tx=0x7f036c02a340 comp rx=0 tx=0).stop 2024-06-13T07:05:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 -- 172.21.15.7:0/2865039206 shutdown_connections 2024-06-13T07:05:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 --2- 172.21.15.7:0/2865039206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 --2- 172.21.15.7:0/2865039206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c104e20 0x7f037c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 --2- 172.21.15.7:0/2865039206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c103c30 0x7f037c104050 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 -- 172.21.15.7:0/2865039206 >> 172.21.15.7:0/2865039206 conn(0x7f037c0ff1c0 msgr2=0x7f037c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.874+0000 7f0381600640 1 -- 172.21.15.7:0/2865039206 shutdown_connections 2024-06-13T07:05:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.875+0000 7f0381600640 1 -- 172.21.15.7:0/2865039206 wait complete. 2024-06-13T07:05:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.876+0000 7f0381600640 1 Processor -- start 2024-06-13T07:05:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.876+0000 7f0381600640 1 -- start start 2024-06-13T07:05:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.877+0000 7f0381600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f0381600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 0x7f037c06c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f0381600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f0381600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10f970 con 0x7f037c1057e0 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f0381600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10f7f0 con 0x7f037c104e20 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f0381600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10faf0 con 0x7f037c103c30 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f037aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f037aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34280/0 (socket says 172.21.15.7:34280) 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f037aa00640 1 -- 172.21.15.7:0/2142620012 learned_addr learned my addr 172.21.15.7:0/2142620012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.878+0000 7f037b400640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 0x7f037c06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 -- 172.21.15.7:0/2142620012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 msgr2=0x7f037c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 -- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 msgr2=0x7f037c073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037a000640 1 -- 172.21.15.7:0/2142620012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c1b7070 con 0x7f037c104e20 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.879+0000 7f037b400640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.880+0000 7f037aa00640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:05:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.880+0000 7f037a000640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 0x7f037c06c410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0368009fd0 tx=0x7f036800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.881+0000 7f0373400640 1 -- 172.21.15.7:0/2142620012 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036800ccc0 con 0x7f037c104e20 2024-06-13T07:05:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.881+0000 7f0373400640 1 -- 172.21.15.7:0/2142620012 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036800ee70 con 0x7f037c104e20 2024-06-13T07:05:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.881+0000 7f0373400640 1 -- 172.21.15.7:0/2142620012 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368004a00 con 0x7f037c104e20 2024-06-13T07:05:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.881+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c1b72a0 con 0x7f037c104e20 2024-06-13T07:05:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.881+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c1b7820 con 0x7f037c104e20 2024-06-13T07:05:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.895+0000 7f038439d740 1 Processor -- start 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.898+0000 7f0373400640 1 -- 172.21.15.7:0/2142620012 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0368004ba0 con 0x7f037c104e20 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.898+0000 7f038439d740 1 -- start start 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.899+0000 7f038439d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 0x55e2f6330020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.899+0000 7f038439d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f63350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.899+0000 7f038439d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 0x55e2f6337bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.901+0000 7f037b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 0x55e2f6330020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.901+0000 7f037aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f63350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.901+0000 7f037aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f63350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52262/0 (socket says 172.21.15.7:52262) 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.901+0000 7f037b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 0x55e2f6330020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34290/0 (socket says 172.21.15.7:34290) 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.901+0000 7f037aa00640 1 -- 172.21.15.7:0/523245742 learned_addr learned my addr 172.21.15.7:0/523245742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.902+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e2f6293420 con 0x55e2f6335760 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.902+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e2f623ac50 con 0x55e2f63305f0 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.902+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e2f628f3e0 con 0x55e2f632fc00 2024-06-13T07:05:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.902+0000 7f037a000640 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 0x55e2f6337bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.903+0000 7f037a000640 1 -- 172.21.15.7:0/523245742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 msgr2=0x55e2f6330020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.903+0000 7f037a000640 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 0x55e2f6330020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.904+0000 7f037a000640 1 -- 172.21.15.7:0/523245742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 msgr2=0x55e2f63350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.905+0000 7f037a000640 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f63350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.906+0000 7f037a000640 1 -- 172.21.15.7:0/523245742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f5fa7210 con 0x55e2f6335760 2024-06-13T07:05:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.906+0000 7f0373400640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 0x7f03600a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.906+0000 7f037a000640 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 0x55e2f6337bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03680ccb00 tx=0x7f03680d79f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf6a8a2b79897e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.907+0000 7f037b400640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 0x7f03600a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.907+0000 7f0367400640 1 -- 172.21.15.7:0/523245742 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680e3070 con 0x55e2f6335760 2024-06-13T07:05:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.908+0000 7f0367400640 1 -- 172.21.15.7:0/523245742 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036809bf90 con 0x55e2f6335760 2024-06-13T07:05:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.908+0000 7f0367400640 1 -- 172.21.15.7:0/523245742 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680e3070 con 0x55e2f6335760 2024-06-13T07:05:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.908+0000 7f0373400640 1 -- 172.21.15.7:0/2142620012 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f03680cf370 con 0x7f037c104e20 2024-06-13T07:05:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.909+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 msgr2=0x55e2f6337bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.909+0000 7f038439d740 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 0x55e2f6337bf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03680ccb00 tx=0x7f03680d79f0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.910+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 shutdown_connections 2024-06-13T07:05:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.910+0000 7f038439d740 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f6335760 0x55e2f6337bf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.910+0000 7f038439d740 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f63350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.910+0000 7f038439d740 1 --2- 172.21.15.7:0/523245742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f632fc00 0x55e2f6330020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.910+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 >> 172.21.15.7:0/523245742 conn(0x55e2f62f7fb0 msgr2=0x55e2f632b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.911+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 shutdown_connections 2024-06-13T07:05:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.911+0000 7f038439d740 1 -- 172.21.15.7:0/523245742 wait complete. 2024-06-13T07:05:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.912+0000 7f038439d740 1 -- start start 2024-06-13T07:05:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.912+0000 7f038439d740 1 Processor -- start 2024-06-13T07:05:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.912+0000 7f037b400640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 0x7f03600a52b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e2f6330e50 tx=0x7f037400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f038439d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f038439d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f633a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f038439d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f6335760 0x55e2f63414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f037aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f037a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f633a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f037aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56984/0 (socket says 172.21.15.7:56984) 2024-06-13T07:05:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f037a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f633a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52264/0 (socket says 172.21.15.7:52264) 2024-06-13T07:05:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e2f6338710 con 0x55e2f632fc00 2024-06-13T07:05:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.914+0000 7f037aa00640 1 -- 172.21.15.7:0/4160657868 learned_addr learned my addr 172.21.15.7:0/4160657868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e2f624e0b0 con 0x55e2f63305f0 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e2f6338890 con 0x55e2f6335760 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f038439d740 10 client.0 _ll_register_callbacks cb 0x55e2f6239f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f037aa00640 1 -- 172.21.15.7:0/4160657868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f6335760 msgr2=0x55e2f63414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f037aa00640 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f6335760 0x55e2f63414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f037aa00640 1 -- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 msgr2=0x55e2f633a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f037aa00640 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f633a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.915+0000 7f037aa00640 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f6073620 con 0x55e2f632fc00 2024-06-13T07:05:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.920+0000 7f037aa00640 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f036c009830 tx=0x7f036c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366c5ef9afee1d64 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.920+0000 7f035be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.921+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c037040 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.921+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036809bf90 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.922+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f036c005030 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.923+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f036c04b050 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.924+0000 7f038439d740 10 client.16179 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.924+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2f6344910 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036809bf90 con 0x55e2f632fc00 2024-06-13T07:05:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f036c0035f0 con 0x55e2f632fc00 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 10 client.16179 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 20 client.16179 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f63460a0 0x55e2f63484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 4 client.16179 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.925+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2f62a0b50 con 0x55e2f63460a0 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.928+0000 7f037a000640 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f63460a0 0x55e2f63484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.930+0000 7f037a000640 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f63460a0 0x55e2f63484f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53cd068423d22ed server_cookie=5f6d370578bb2092 in_seq=0 out_seq=0 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f035be00640 10 client.16179 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f035be00640 1 -- 172.21.15.7:0/4160657868 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2f62a0b50 con 0x55e2f63460a0 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f035be00640 10 client.16179 handle_command_reply: tid=0 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f038439d740 1 client.16179 shutdown 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 msgr2=0x55e2f6339f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.932+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f036c009830 tx=0x7f036c0041a0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 shutdown_connections 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f63460a0 0x55e2f63484f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f6335760 0x55e2f63414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f633a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 --2- 172.21.15.7:0/4160657868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f6339f60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f035be00640 0 client.16179 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.933+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 >> 172.21.15.7:0/4160657868 conn(0x55e2f62f7fb0 msgr2=0x55e2f632a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.934+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 shutdown_connections 2024-06-13T07:05:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.934+0000 7f038439d740 1 -- 172.21.15.7:0/4160657868 wait complete. 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.934+0000 7f038439d740 20 client.16179 trim_cache size 0 max 16384 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.974+0000 7f038439d740 1 Processor -- start 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.974+0000 7f038439d740 1 -- start start 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f63414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f6339580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1c43d0 con 0x55e2f632fc00 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1c1ca0 con 0x55e2f63305f0 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f038439d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1bfaf0 con 0x55e2f62f7fb0 2024-06-13T07:05:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:34298/0 (socket says 172.21.15.7:34298) 2024-06-13T07:05:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 -- 172.21.15.7:0/1297515579 learned_addr learned my addr 172.21.15.7:0/1297515579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037a000640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f63414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 -- 172.21.15.7:0/1297515579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 msgr2=0x55e2f6339580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037b400640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f6339580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f6339580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.975+0000 7f037aa00640 1 -- 172.21.15.7:0/1297515579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 msgr2=0x55e2f63414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f037aa00640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f63414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f037aa00640 1 -- 172.21.15.7:0/1297515579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f63309d0 con 0x55e2f62f7fb0 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f037a000640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f63414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f037b400640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f6339580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f037aa00640 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f036c003c00 tx=0x7f036c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475890f671c95a89 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f0366a00640 1 -- 172.21.15.7:0/1297515579 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c037040 con 0x55e2f62f7fb0 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f0366a00640 1 -- 172.21.15.7:0/1297515579 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c009450 con 0x55e2f62f7fb0 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.976+0000 7f0366a00640 1 -- 172.21.15.7:0/1297515579 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0051a0 con 0x55e2f62f7fb0 2024-06-13T07:05:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 -- 172.21.15.7:0/1297515579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 msgr2=0x55e2f632eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f036c003c00 tx=0x7f036c005850 comp rx=0 tx=0).stop 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 -- 172.21.15.7:0/1297515579 shutdown_connections 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f63305f0 0x55e2f6339580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f63414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 --2- 172.21.15.7:0/1297515579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f62f7fb0 0x55e2f632eaf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 -- 172.21.15.7:0/1297515579 >> 172.21.15.7:0/1297515579 conn(0x55e2f63311f0 msgr2=0x55e2f632e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.977+0000 7f038439d740 1 -- 172.21.15.7:0/1297515579 shutdown_connections 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 -- 172.21.15.7:0/1297515579 wait complete. 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 -- start start 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 Processor -- start 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f633e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f63305f0 0x55e2f63aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1c0160 con 0x55e2f632fc00 2024-06-13T07:05:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1c15d0 con 0x55e2f62f7fb0 2024-06-13T07:05:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f037c1c0830 con 0x55e2f63305f0 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.978+0000 7f038439d740 10 client.0 _ll_register_callbacks cb 0x7f0368005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f633e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f63305f0 0x55e2f63aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52294/0 (socket says 172.21.15.7:52294) 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 -- 172.21.15.7:0/3938770270 learned_addr learned my addr 172.21.15.7:0/3938770270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 -- 172.21.15.7:0/3938770270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f63305f0 msgr2=0x55e2f63aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f63305f0 0x55e2f63aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 -- 172.21.15.7:0/3938770270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 msgr2=0x55e2f633e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f633e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.979+0000 7f037aa00640 1 -- 172.21.15.7:0/3938770270 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f63af380 con 0x55e2f62f7fb0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f037aa00640 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f036c0058c0 tx=0x7f036c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60cf4aad0f23c88e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f035be00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c037040 con 0x55e2f62f7fb0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f036c005240 con 0x55e2f62f7fb0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c009450 con 0x55e2f62f7fb0 2024-06-13T07:05:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f036c068050 con 0x55e2f62f7fb0 2024-06-13T07:05:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f038439d740 10 client.26780 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2f63b0600 con 0x55e2f62f7fb0 2024-06-13T07:05:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.980+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036c009450 con 0x55e2f62f7fb0 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f036c03d570 con 0x55e2f62f7fb0 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 10 client.26780 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 10 client.26780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 20 client.26780 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f6335760 0x55e2f63b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 4 client.26780 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2f629aea0 con 0x55e2f6335760 2024-06-13T07:05:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.981+0000 7f037a000640 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f6335760 0x55e2f63b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.982+0000 7f037a000640 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f6335760 0x55e2f63b2f70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4160919fd0700043 server_cookie=7304079f4f1290d4 in_seq=0 out_seq=0 2024-06-13T07:05:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.982+0000 7f035be00640 10 client.26780 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.996+0000 7f035be00640 1 -- 172.21.15.7:0/3938770270 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e2f629aea0 con 0x55e2f6335760 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.996+0000 7f035be00640 10 client.26780 handle_command_reply: tid=0 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.996+0000 7f038439d740 1 client.26780 shutdown 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 msgr2=0x55e2f633e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f036c0058c0 tx=0x7f036c003820 comp rx=0 tx=0).stop 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 shutdown_connections 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e2f6335760 0x55e2f63b2f70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e2f63305f0 0x55e2f63aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2f632fc00 0x55e2f633e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 --2- 172.21.15.7:0/3938770270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e2f62f7fb0 0x55e2f633e460 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 >> 172.21.15.7:0/3938770270 conn(0x55e2f63311f0 msgr2=0x55e2f632ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f035be00640 0 client.26780 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 shutdown_connections 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 1 -- 172.21.15.7:0/3938770270 wait complete. 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.997+0000 7f038439d740 20 client.26780 trim_cache size 0 max 16384 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:05:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 msgr2=0x7f03600a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 0x7f03600a52b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e2f6330e50 tx=0x7f037400d040 comp rx=0 tx=0).stop 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 msgr2=0x7f037c06c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 0x7f037c06c410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0368009fd0 tx=0x7f036800ec20 comp rx=0 tx=0).stop 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 shutdown_connections 2024-06-13T07:05:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f03600a2df0 0x7f03600a52b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1057e0 0x7f037c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f037c104e20 0x7f037c06c410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 --2- 172.21.15.7:0/2142620012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f037c103c30 0x7f037c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 >> 172.21.15.7:0/2142620012 conn(0x7f037c0ff1c0 msgr2=0x7f037c077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 shutdown_connections 2024-06-13T07:05:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:30.998+0000 7f0381600640 1 -- 172.21.15.7:0/2142620012 wait complete. 2024-06-13T07:05:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:31 smithi189 ceph-mon[36616]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-06-13T07:05:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:31 smithi175 ceph-mon[36886]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-06-13T07:05:31.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:31 smithi007 ceph-mon[32077]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-06-13T07:05:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 570.0 s, 573.8 tps, lat 179.770 ms stddev 224.350 2024-06-13T07:05:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:33 smithi189 ceph-mon[36616]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-06-13T07:05:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:33 smithi175 ceph-mon[36886]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-06-13T07:05:33.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:33 smithi007 ceph-mon[32077]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-06-13T07:05:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:35 smithi189 ceph-mon[36616]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 MiB/s wr, 505 op/s 2024-06-13T07:05:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:35 smithi175 ceph-mon[36886]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 MiB/s wr, 505 op/s 2024-06-13T07:05:35.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:35 smithi007 ceph-mon[32077]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 MiB/s wr, 505 op/s 2024-06-13T07:05:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:37 smithi189 ceph-mon[36616]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-06-13T07:05:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:37 smithi175 ceph-mon[36886]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-06-13T07:05:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 575.0 s, 582.6 tps, lat 169.640 ms stddev 190.746 2024-06-13T07:05:37.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:37 smithi007 ceph-mon[32077]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-06-13T07:05:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:39 smithi189 ceph-mon[36616]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-06-13T07:05:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:39 smithi175 ceph-mon[36886]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-06-13T07:05:39.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:39 smithi007 ceph-mon[32077]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 MiB/s wr, 504 op/s 2024-06-13T07:05:40.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:05:41.634 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:05:41.824 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:41 smithi007 ceph-mon[32077]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-06-13T07:05:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:41 smithi189 ceph-mon[36616]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-06-13T07:05:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:41 smithi175 ceph-mon[36886]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 MiB/s wr, 509 op/s 2024-06-13T07:05:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 580.0 s, 486.2 tps, lat 185.626 ms stddev 212.352 2024-06-13T07:05:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:43 smithi189 ceph-mon[36616]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-06-13T07:05:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:43 smithi175 ceph-mon[36886]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-06-13T07:05:43.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:43 smithi007 ceph-mon[32077]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 MiB/s wr, 484 op/s 2024-06-13T07:05:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.164+0000 7f228ac00640 1 -- 172.21.15.7:0/372890585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22841057b0 msgr2=0x7f228410c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.164+0000 7f228ac00640 1 --2- 172.21.15.7:0/372890585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22841057b0 0x7f228410c040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 -- 172.21.15.7:0/372890585 shutdown_connections 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 --2- 172.21.15.7:0/372890585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22841057b0 0x7f228410c040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 --2- 172.21.15.7:0/372890585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f2284105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 --2- 172.21.15.7:0/372890585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284103c00 0x7f2284104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 -- 172.21.15.7:0/372890585 >> 172.21.15.7:0/372890585 conn(0x7f22840ff190 msgr2=0x7f22841015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 -- 172.21.15.7:0/372890585 shutdown_connections 2024-06-13T07:05:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.165+0000 7f228ac00640 1 -- 172.21.15.7:0/372890585 wait complete. 2024-06-13T07:05:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 Processor -- start 2024-06-13T07:05:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 -- start start 2024-06-13T07:05:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 0x7f2284199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22841057b0 0x7f22841a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f228410f8d0 con 0x7f22841057b0 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f228410f750 con 0x7f2284103c00 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f228ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f228410fa50 con 0x7f2284104df0 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2283400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2283e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 0x7f2284199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2288c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22841057b0 0x7f22841a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2283400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41950/0 (socket says 172.21.15.7:41950) 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2283400640 1 -- 172.21.15.7:0/1861469114 learned_addr learned my addr 172.21.15.7:0/1861469114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.166+0000 7f2283e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 0x7f2284199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57880/0 (socket says 172.21.15.7:57880) 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 -- 172.21.15.7:0/1861469114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 msgr2=0x7f2284199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 0x7f2284199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 -- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22841057b0 msgr2=0x7f22841a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22841057b0 0x7f22841a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 -- 172.21.15.7:0/1861469114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22841a1a30 con 0x7f2284104df0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.167+0000 7f2283400640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f227000e9f0 tx=0x7f227000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.168+0000 7f2280c00640 1 -- 172.21.15.7:0/1861469114 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000cd00 con 0x7f2284104df0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.168+0000 7f2280c00640 1 -- 172.21.15.7:0/1861469114 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270005500 con 0x7f2284104df0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.168+0000 7f2280c00640 1 -- 172.21.15.7:0/1861469114 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270010690 con 0x7f2284104df0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.168+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22841a1d20 con 0x7f2284104df0 2024-06-13T07:05:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.169+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a21c0 con 0x7f2284104df0 2024-06-13T07:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 Processor -- start 2024-06-13T07:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f2280c00640 1 -- 172.21.15.7:0/1861469114 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2270004f20 con 0x7f2284104df0 2024-06-13T07:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 -- start start 2024-06-13T07:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 0x560d0bb2f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ff70 0x560d0bb32400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d0ba85420 con 0x560d0bb2ae00 2024-06-13T07:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d0ba2cc50 con 0x560d0bb2a410 2024-06-13T07:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.174+0000 7f228da74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d0ba813e0 con 0x560d0bb2ff70 2024-06-13T07:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57884/0 (socket says 172.21.15.7:57884) 2024-06-13T07:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 -- 172.21.15.7:0/629857476 learned_addr learned my addr 172.21.15.7:0/629857476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 -- 172.21.15.7:0/629857476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ff70 msgr2=0x560d0bb32400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2283e00640 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 0x560d0bb2f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ff70 0x560d0bb32400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.175+0000 7f2288c00640 1 -- 172.21.15.7:0/629857476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 msgr2=0x560d0bb2f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.176+0000 7f2288c00640 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 0x560d0bb2f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.176+0000 7f2288c00640 1 -- 172.21.15.7:0/629857476 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0b799210 con 0x560d0bb2a410 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.176+0000 7f2283e00640 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 0x560d0bb2f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.176+0000 7f2280c00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.177+0000 7f2288c00640 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90314b82e7e63c51 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.178+0000 7f2280c00640 1 -- 172.21.15.7:0/1861469114 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f22700cf580 con 0x7f2284104df0 2024-06-13T07:05:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.178+0000 7f2279200640 1 -- 172.21.15.7:0/629857476 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c031650 con 0x560d0bb2a410 2024-06-13T07:05:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.179+0000 7f2279200640 1 -- 172.21.15.7:0/629857476 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c036030 con 0x560d0bb2a410 2024-06-13T07:05:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.179+0000 7f2279200640 1 -- 172.21.15.7:0/629857476 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c0047b0 con 0x560d0bb2a410 2024-06-13T07:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.180+0000 7f2283e00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.182+0000 7f228da74740 1 -- 172.21.15.7:0/629857476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 msgr2=0x560d0bb2a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.182+0000 7f228da74740 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c005fa0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.182+0000 7f2283e00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 0x7f22680a51c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x560d0bb2b200 tx=0x7f227400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.185+0000 7f228da74740 1 -- 172.21.15.7:0/629857476 shutdown_connections 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.185+0000 7f228da74740 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ff70 0x560d0bb32400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.185+0000 7f228da74740 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ae00 0x560d0bb2f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.185+0000 7f228da74740 1 --2- 172.21.15.7:0/629857476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb2a830 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.185+0000 7f228da74740 1 -- 172.21.15.7:0/629857476 >> 172.21.15.7:0/629857476 conn(0x560d0bae9fb0 msgr2=0x560d0bb25e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.186+0000 7f228da74740 1 -- 172.21.15.7:0/629857476 shutdown_connections 2024-06-13T07:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.186+0000 7f228da74740 1 -- 172.21.15.7:0/629857476 wait complete. 2024-06-13T07:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.186+0000 7f228da74740 1 -- start start 2024-06-13T07:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 Processor -- start 2024-06-13T07:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb34b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d0bb32d40 con 0x560d0bb2a410 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560d0ba400b0 con 0x560d0bb2ff70 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560d0bb32ec0 con 0x560d0bb2ae00 2024-06-13T07:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.187+0000 7f228da74740 10 client.0 _ll_register_callbacks cb 0x560d0ba2bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44884/0 (socket says 172.21.15.7:44884) 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 -- 172.21.15.7:0/3245826829 learned_addr learned my addr 172.21.15.7:0/3245826829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 -- 172.21.15.7:0/3245826829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 msgr2=0x560d0bb34b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb34b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 -- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 msgr2=0x560d0bb3bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.188+0000 7f2283e00640 1 -- 172.21.15.7:0/3245826829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0b865620 con 0x560d0bb2a410 2024-06-13T07:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.189+0000 7f2283e00640 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2274004650 tx=0x7f22740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3af96beab0cb11 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.189+0000 7f226d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.189+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227401f620 con 0x560d0bb2a410 2024-06-13T07:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.190+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227c036030 con 0x560d0bb2a410 2024-06-13T07:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.190+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f227401f7c0 con 0x560d0bb2a410 2024-06-13T07:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.190+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f227401f9c0 con 0x560d0bb2a410 2024-06-13T07:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.190+0000 7f228da74740 10 client.16194 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.190+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560d0bb3efa0 con 0x560d0bb2a410 2024-06-13T07:05:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.191+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227c036030 con 0x560d0bb2a410 2024-06-13T07:05:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.192+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f227401e3b0 con 0x560d0bb2a410 2024-06-13T07:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.197+0000 7f228da74740 10 client.16194 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.197+0000 7f228da74740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.197+0000 7f228da74740 20 client.16194 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.199+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb406c0 0x560d0bb42b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.199+0000 7f228da74740 4 client.16194 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.199+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d0ba92b50 con 0x560d0bb406c0 2024-06-13T07:05:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.200+0000 7f2283400640 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb406c0 0x560d0bb42b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.201+0000 7f2283400640 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb406c0 0x560d0bb42b60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b655e156a73e91f server_cookie=3f41611dbd9930db in_seq=0 out_seq=0 2024-06-13T07:05:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.201+0000 7f226d600640 10 client.16194 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.202+0000 7f226d600640 1 -- 172.21.15.7:0/3245826829 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d0ba92b50 con 0x560d0bb406c0 2024-06-13T07:05:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.202+0000 7f226d600640 10 client.16194 handle_command_reply: tid=0 2024-06-13T07:05:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.202+0000 7f228da74740 1 client.16194 shutdown 2024-06-13T07:05:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 msgr2=0x560d0bb34620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2274004650 tx=0x7f22740046c0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 shutdown_connections 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb406c0 0x560d0bb42b60 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb34b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 --2- 172.21.15.7:0/3245826829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb34620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 >> 172.21.15.7:0/3245826829 conn(0x560d0bae9fb0 msgr2=0x560d0bb1c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f226d600640 0 client.16194 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 shutdown_connections 2024-06-13T07:05:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.203+0000 7f228da74740 1 -- 172.21.15.7:0/3245826829 wait complete. 2024-06-13T07:05:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.206+0000 7f228da74740 20 client.16194 trim_cache size 0 max 16384 2024-06-13T07:05:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 Processor -- start 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 -- start start 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb3bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ff70 0x560d0bb29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22841aab40 con 0x560d0bb2ff70 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a8410 con 0x560d0bb2a410 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f228da74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a6200 con 0x560d0bb2ae00 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.252+0000 7f2283e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:57904/0 (socket says 172.21.15.7:57904) 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 -- 172.21.15.7:0/820759851 learned_addr learned my addr 172.21.15.7:0/820759851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283400640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb3bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2288c00640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ff70 0x560d0bb29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 -- 172.21.15.7:0/820759851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 msgr2=0x560d0bb3bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb3bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 -- 172.21.15.7:0/820759851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ff70 msgr2=0x560d0bb29570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ff70 0x560d0bb29570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 -- 172.21.15.7:0/820759851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0bb3edc0 con 0x560d0bb2a410 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.253+0000 7f2283e00640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2274003a30 tx=0x7f2274004650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3be31dde8821bcc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.254+0000 7f226fe00640 1 -- 172.21.15.7:0/820759851 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227401f9c0 con 0x560d0bb2a410 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.254+0000 7f2283400640 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb3bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.256+0000 7f226fe00640 1 -- 172.21.15.7:0/820759851 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227401f420 con 0x560d0bb2a410 2024-06-13T07:05:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.256+0000 7f226fe00640 1 -- 172.21.15.7:0/820759851 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227402f690 con 0x560d0bb2a410 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 -- 172.21.15.7:0/820759851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 msgr2=0x560d0bb3cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2274003a30 tx=0x7f2274004650 comp rx=0 tx=0).stop 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 -- 172.21.15.7:0/820759851 shutdown_connections 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2ff70 0x560d0bb29570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb3bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 --2- 172.21.15.7:0/820759851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2a410 0x560d0bb3cc60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 -- 172.21.15.7:0/820759851 >> 172.21.15.7:0/820759851 conn(0x560d0bae9fb0 msgr2=0x560d0bb28e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 -- 172.21.15.7:0/820759851 shutdown_connections 2024-06-13T07:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.257+0000 7f228da74740 1 -- 172.21.15.7:0/820759851 wait complete. 2024-06-13T07:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.258+0000 7f228da74740 1 -- start start 2024-06-13T07:05:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.258+0000 7f228da74740 1 Processor -- start 2024-06-13T07:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb37780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a68d0 con 0x560d0bb2a410 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a7d40 con 0x560d0bb2ff70 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22841a6fa0 con 0x560d0bb2ae00 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f228da74740 10 client.0 _ll_register_callbacks cb 0x7f2274004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f2283400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f2283400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb37cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41976/0 (socket says 172.21.15.7:41976) 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f2283400640 1 -- 172.21.15.7:0/270519659 learned_addr learned my addr 172.21.15.7:0/270519659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f2283e00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb37780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.259+0000 7f2288c00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.260+0000 7f2288c00640 1 -- 172.21.15.7:0/270519659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 msgr2=0x560d0bb37cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.260+0000 7f2288c00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb37cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.260+0000 7f2288c00640 1 -- 172.21.15.7:0/270519659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 msgr2=0x560d0bb37780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.260+0000 7f2288c00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb37780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.260+0000 7f2288c00640 1 -- 172.21.15.7:0/270519659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0bb3b2d0 con 0x560d0bb2ff70 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.261+0000 7f2288c00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3ace0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8d0b56e396a050 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.261+0000 7f226d600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.261+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c051070 con 0x560d0bb2ff70 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.261+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f227c004040 con 0x560d0bb2ff70 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.261+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227401f420 con 0x560d0bb2ff70 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.262+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f227c05d050 con 0x560d0bb2ff70 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.262+0000 7f228da74740 10 client.26795 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.262+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560d0bb20480 con 0x560d0bb2ff70 2024-06-13T07:05:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.262+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227401f420 con 0x560d0bb2ff70 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f227c043070 con 0x560d0bb2ff70 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 10 client.26795 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 20 client.26795 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb20a50 0x560d0bb22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 4 client.26795 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.263+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d0ba8cea0 con 0x560d0bb20a50 2024-06-13T07:05:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.264+0000 7f2283e00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb20a50 0x560d0bb22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.268+0000 7f226d600640 10 client.26795 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.268+0000 7f2283e00640 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb20a50 0x560d0bb22e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8c2fc170f83032 server_cookie=8a0f5025be72314 in_seq=0 out_seq=0 2024-06-13T07:05:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.405+0000 7f226d600640 1 -- 172.21.15.7:0/270519659 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d0ba8cea0 con 0x560d0bb20a50 2024-06-13T07:05:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.405+0000 7f226d600640 10 client.26795 handle_command_reply: tid=0 2024-06-13T07:05:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.405+0000 7f228da74740 1 client.26795 shutdown 2024-06-13T07:05:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.405+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 msgr2=0x560d0bb3ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.405+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3ace0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c003ba0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 shutdown_connections 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560d0bb20a50 0x560d0bb22e70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560d0bb2ff70 0x560d0bb3ace0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0bb2ae00 0x560d0bb37cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 --2- 172.21.15.7:0/270519659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d0bb2a410 0x560d0bb37780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 >> 172.21.15.7:0/270519659 conn(0x560d0bae9fb0 msgr2=0x560d0bb33c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f226d600640 0 client.26795 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 shutdown_connections 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 1 -- 172.21.15.7:0/270519659 wait complete. 2024-06-13T07:05:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.406+0000 7f228da74740 20 client.26795 trim_cache size 0 max 16384 2024-06-13T07:05:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:05:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:05:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:05:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 msgr2=0x7f22680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 0x7f22680a51c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x560d0bb2b200 tx=0x7f227400b410 comp rx=0 tx=0).stop 2024-06-13T07:05:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 msgr2=0x7f228419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f227000e9f0 tx=0x7f227000edf0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 shutdown_connections 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22841057b0 0x7f22841a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2284104df0 0x7f228419a350 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 --2- 172.21.15.7:0/1861469114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2284103c00 0x7f2284199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.411+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 >> 172.21.15.7:0/1861469114 conn(0x7f22840ff190 msgr2=0x7f2284108090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.412+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 shutdown_connections 2024-06-13T07:05:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:44.412+0000 7f228ac00640 1 -- 172.21.15.7:0/1861469114 wait complete. 2024-06-13T07:05:45.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:45 smithi007 ceph-mon[32077]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:45 smithi189 ceph-mon[36616]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:45 smithi175 ceph-mon[36886]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:46.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.353+0000 7fbaff400700 1 -- 172.21.15.7:0/1055692164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109db0 msgr2=0x7fbaf81150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.353+0000 7fbaff400700 1 --2- 172.21.15.7:0/1055692164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109db0 0x7fbaf81150a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009990 tx=0x7fbaec009ca0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.353+0000 7fbaff400700 1 -- 172.21.15.7:0/1055692164 shutdown_connections 2024-06-13T07:05:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.354+0000 7fbaff400700 1 --2- 172.21.15.7:0/1055692164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf81155e0 0x7fbaf81179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.354+0000 7fbaff400700 1 --2- 172.21.15.7:0/1055692164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf8109490 0x7fbaf8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.357+0000 7fbaff400700 1 --2- 172.21.15.7:0/1055692164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109db0 0x7fbaf81150a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.357+0000 7fbaff400700 1 -- 172.21.15.7:0/1055692164 >> 172.21.15.7:0/1055692164 conn(0x7fbaf806d600 msgr2=0x7fbaf806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.357+0000 7fbaff400700 1 -- 172.21.15.7:0/1055692164 shutdown_connections 2024-06-13T07:05:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.358+0000 7fbaff400700 1 -- 172.21.15.7:0/1055692164 wait complete. 2024-06-13T07:05:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.359+0000 7fbaff400700 1 Processor -- start 2024-06-13T07:05:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.359+0000 7fbaff400700 1 -- start start 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.360+0000 7fbaff400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109490 0x7fbaf81a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaff400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaff400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf81155e0 0x7fbaf81a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaff400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf811b0a0 con 0x7fbaf81155e0 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaff400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf811af20 con 0x7fbaf8109490 2024-06-13T07:05:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaff400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf811b220 con 0x7fbaf8109db0 2024-06-13T07:05:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42000/0 (socket says 172.21.15.7:42000) 2024-06-13T07:05:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbafca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109490 0x7fbaf81a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 -- 172.21.15.7:0/4166043650 learned_addr learned my addr 172.21.15.7:0/4166043650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 -- 172.21.15.7:0/4166043650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109490 msgr2=0x7fbaf81a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109490 0x7fbaf81a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 -- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf81155e0 msgr2=0x7fbaf81a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf81155e0 0x7fbaf81a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.361+0000 7fbaf7e00700 1 -- 172.21.15.7:0/4166043650 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf81a86c0 con 0x7fbaf8109db0 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b2e0 tx=0x7fbaec00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaf5600700 1 -- 172.21.15.7:0/4166043650 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003890 con 0x7fbaf8109db0 2024-06-13T07:05:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaf5600700 1 -- 172.21.15.7:0/4166043650 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec00f460 con 0x7fbaf8109db0 2024-06-13T07:05:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaf5600700 1 -- 172.21.15.7:0/4166043650 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003890 con 0x7fbaf8109db0 2024-06-13T07:05:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81a8950 con 0x7fbaf8109db0 2024-06-13T07:05:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.362+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81b0230 con 0x7fbaf8109db0 2024-06-13T07:05:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.368+0000 7fbb09d6fb80 1 Processor -- start 2024-06-13T07:05:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 -- start start 2024-06-13T07:05:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c7bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c75d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c76680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563318b84a10 con 0x563318c76240 2024-06-13T07:05:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563318bc7b00 con 0x563318c7c4b0 2024-06-13T07:05:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563318b828f0 con 0x563318c7bb90 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.369+0000 7fbaf5600700 1 -- 172.21.15.7:0/4166043650 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbaec009d10 con 0x7fbaf8109db0 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.370+0000 7fbafd400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c7bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.370+0000 7fbafd400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c7bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42016/0 (socket says 172.21.15.7:42016) 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.370+0000 7fbafd400700 1 -- 172.21.15.7:0/3364470974 learned_addr learned my addr 172.21.15.7:0/3364470974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.370+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.371+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3364470974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 msgr2=0x563318c7bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.371+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c7bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.371+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3364470974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 msgr2=0x563318c75d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.371+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c75d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.371+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3364470974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563318b6ff90 con 0x563318c76240 2024-06-13T07:05:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.372+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c76680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0cd430 tx=0x7fbaec0d3a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0db373b4ca46e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.372+0000 7fbaf5600700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 0x7fbae00a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.372+0000 7fbae6000700 1 -- 172.21.15.7:0/3364470974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0ea420 con 0x563318c76240 2024-06-13T07:05:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.373+0000 7fbafd400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 0x7fbae00a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.373+0000 7fbae6000700 1 -- 172.21.15.7:0/3364470974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec0e5030 con 0x563318c76240 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.373+0000 7fbae6000700 1 -- 172.21.15.7:0/3364470974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0ea420 con 0x563318c76240 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.373+0000 7fbaf5600700 1 -- 172.21.15.7:0/4166043650 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbaec01e070 con 0x7fbaf8109db0 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3364470974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 msgr2=0x563318c76680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c76680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0cd430 tx=0x7fbaec0d3a30 comp rx=0 tx=0).stop 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3364470974 shutdown_connections 2024-06-13T07:05:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c76680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c7bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3364470974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c75d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.374+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3364470974 >> 172.21.15.7:0/3364470974 conn(0x563318ba2170 msgr2=0x563318c70ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.375+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3364470974 shutdown_connections 2024-06-13T07:05:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.375+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3364470974 wait complete. 2024-06-13T07:05:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.376+0000 7fbb09d6fb80 1 -- start start 2024-06-13T07:05:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.376+0000 7fbb09d6fb80 1 Processor -- start 2024-06-13T07:05:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 0x563318c82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c7bb90 0x563318c7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563318b63f60 con 0x563318c7bb90 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563318bc3ac0 con 0x563318c7c4b0 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563318c82c40 con 0x563318c76240 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.377+0000 7fbb09d6fb80 10 client.0 _ll_register_callbacks cb 0x563318b5d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.378+0000 7fbafca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 0x563318c82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.378+0000 7fbafca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 0x563318c82180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:42032/0 (socket says 172.21.15.7:42032) 2024-06-13T07:05:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.378+0000 7fbafca00700 1 -- 172.21.15.7:0/2749347479 learned_addr learned my addr 172.21.15.7:0/2749347479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.378+0000 7fbafd400700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 -- 172.21.15.7:0/2749347479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 msgr2=0x563318c82180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 0x563318c82180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 -- 172.21.15.7:0/2749347479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c7bb90 msgr2=0x563318c7f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c7bb90 0x563318c7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 -- 172.21.15.7:0/2749347479 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563318b168d0 con 0x563318c7c4b0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 0x7fbae00a55a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009c10 tx=0x7fbaf000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbafd400700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c7fa30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009be0 tx=0x7fbaf0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644a328a75512aa7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.379+0000 7fbadaa00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0027060 con 0x563318c7c4b0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbaf0015b30 con 0x563318c7c4b0 2024-06-13T07:05:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec0e5030 con 0x563318c7c4b0 2024-06-13T07:05:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbaf0015d30 con 0x563318c7c4b0 2024-06-13T07:05:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbb09d6fb80 10 client.26804 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.380+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563318c8d350 con 0x563318c7c4b0 2024-06-13T07:05:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec0e5030 con 0x563318c7c4b0 2024-06-13T07:05:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbaf0022c20 con 0x563318c7c4b0 2024-06-13T07:05:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 10 client.26804 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 20 client.26804 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318c8fa80 0x563318c91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 4 client.26804 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.381+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563318bd5000 con 0x563318c8fa80 2024-06-13T07:05:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.382+0000 7fbafca00700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318c8fa80 0x563318c91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.382+0000 7fbafca00700 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318c8fa80 0x563318c91f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffc802cb495df89 server_cookie=505bf6a2c9cc0d25 in_seq=0 out_seq=0 2024-06-13T07:05:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.382+0000 7fbadaa00700 10 client.26804 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.383+0000 7fbadaa00700 1 -- 172.21.15.7:0/2749347479 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563318bd5000 con 0x563318c8fa80 2024-06-13T07:05:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.383+0000 7fbadaa00700 10 client.26804 handle_command_reply: tid=0 2024-06-13T07:05:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.383+0000 7fbb09d6fb80 1 client.26804 shutdown 2024-06-13T07:05:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 msgr2=0x563318c7fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c7fa30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009be0 tx=0x7fbaf0009920 comp rx=0 tx=0).stop 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 shutdown_connections 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318c8fa80 0x563318c91f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c7fa30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c76240 0x563318c82180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/2749347479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c7bb90 0x563318c7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 >> 172.21.15.7:0/2749347479 conn(0x563318ba2170 msgr2=0x563318c6a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.384+0000 7fbadaa00700 0 client.26804 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.385+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 shutdown_connections 2024-06-13T07:05:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.386+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/2749347479 wait complete. 2024-06-13T07:05:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.386+0000 7fbb09d6fb80 20 client.26804 trim_cache size 0 max 16384 2024-06-13T07:05:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 Processor -- start 2024-06-13T07:05:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 -- start start 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c75d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b70b0 con 0x563318c76240 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b4940 con 0x563318c7c4b0 2024-06-13T07:05:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.423+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b2830 con 0x563318c7bb90 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbafd400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbafca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c75d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbafca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c75d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44968/0 (socket says 172.21.15.7:44968) 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbafca00700 1 -- 172.21.15.7:0/300312983 learned_addr learned my addr 172.21.15.7:0/300312983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 -- 172.21.15.7:0/300312983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 msgr2=0x563318c6b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c6b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 -- 172.21.15.7:0/300312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 msgr2=0x563318c75d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c75d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 -- 172.21.15.7:0/300312983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563318c803d0 con 0x563318c7bb90 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.424+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c711a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0cd430 tx=0x7fbaec0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74a89166fb237cae server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.425+0000 7fbae5600700 1 -- 172.21.15.7:0/300312983 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec09bbe0 con 0x563318c7bb90 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.425+0000 7fbae5600700 1 -- 172.21.15.7:0/300312983 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec0e5020 con 0x563318c7bb90 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.425+0000 7fbae5600700 1 -- 172.21.15.7:0/300312983 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec09bdc0 con 0x563318c7bb90 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.425+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/300312983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 msgr2=0x563318c711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.425+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c711a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0cd430 tx=0x7fbaec0094b0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/300312983 shutdown_connections 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7c4b0 0x563318c6b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c75d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/300312983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7bb90 0x563318c711a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/300312983 >> 172.21.15.7:0/300312983 conn(0x563318ba2170 msgr2=0x563318c70a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/300312983 shutdown_connections 2024-06-13T07:05:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/300312983 wait complete. 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 -- start start 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.426+0000 7fbb09d6fb80 1 Processor -- start 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.427+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c87e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 0x563318c883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7c4b0 0x563318c84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b2e00 con 0x563318c76240 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b4270 con 0x563318c7bb90 2024-06-13T07:05:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b34d0 con 0x563318c7c4b0 2024-06-13T07:05:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbb09d6fb80 10 client.0 _ll_register_callbacks cb 0x563318b5d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbafca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c87e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbaf7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 0x563318c883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbafca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c87e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44984/0 (socket says 172.21.15.7:44984) 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.428+0000 7fbafca00700 1 -- 172.21.15.7:0/3748292279 learned_addr learned my addr 172.21.15.7:0/3748292279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3748292279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7c4b0 msgr2=0x563318c84e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7c4b0 0x563318c84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3748292279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 msgr2=0x563318c87e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c87e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 -- 172.21.15.7:0/3748292279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563318c88a40 con 0x563318c7bb90 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbaf7e00700 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 0x563318c883d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x563318c6b7d0 tx=0x7fbaec005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c01acdc33fe1ee server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbadaa00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec09c050 con 0x563318c7bb90 2024-06-13T07:05:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.429+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec0e5020 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.430+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbaec09a320 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.430+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbaec098520 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.430+0000 7fbb09d6fb80 10 client.26810 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.430+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563318c88a40 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec0e5020 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbaec0cd4e0 con 0x563318c7bb90 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 10 client.26810 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 20 client.26810 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318cfaf30 0x563318cfd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 4 client.26810 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:05:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563318bcf580 con 0x563318cfaf30 2024-06-13T07:05:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.431+0000 7fbafca00700 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318cfaf30 0x563318cfd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.432+0000 7fbadaa00700 10 client.26810 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbafca00700 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318cfaf30 0x563318cfd350 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a593a9686dc7e5 server_cookie=145636d6fefc35a9 in_seq=0 out_seq=0 2024-06-13T07:05:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbadaa00700 1 -- 172.21.15.7:0/3748292279 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563318bcf580 con 0x563318cfaf30 2024-06-13T07:05:46.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbadaa00700 10 client.26810 handle_command_reply: tid=0 2024-06-13T07:05:46.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 client.26810 shutdown 2024-06-13T07:05:46.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 msgr2=0x563318c883d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 0x563318c883d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x563318c6b7d0 tx=0x7fbaec005bc0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 shutdown_connections 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563318cfaf30 0x563318cfd350 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563318c7c4b0 0x563318c84e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563318c76240 0x563318c87e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.433+0000 7fbb09d6fb80 1 --2- 172.21.15.7:0/3748292279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563318c7bb90 0x563318c883d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.434+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 >> 172.21.15.7:0/3748292279 conn(0x563318ba2170 msgr2=0x563318c7c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.434+0000 7fbadaa00700 0 client.26810 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.434+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 shutdown_connections 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.434+0000 7fbb09d6fb80 1 -- 172.21.15.7:0/3748292279 wait complete. 2024-06-13T07:05:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.434+0000 7fbb09d6fb80 20 client.26810 trim_cache size 0 max 16384 2024-06-13T07:05:46.435 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:05:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:05:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:05:46.435 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:05:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 msgr2=0x7fbae00a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 0x7fbae00a55a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009c10 tx=0x7fbaf000d040 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 msgr2=0x7fbaf81a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b2e0 tx=0x7fbaec00bab0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 shutdown_connections 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf81155e0 0x7fbaf81a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbae00a30e0 0x7fbae00a55a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf8109490 0x7fbaf81a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 --2- 172.21.15.7:0/4166043650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109db0 0x7fbaf81a3bf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.435+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 >> 172.21.15.7:0/4166043650 conn(0x7fbaf806d600 msgr2=0x7fbaf810ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.436+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 shutdown_connections 2024-06-13T07:05:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:46.436+0000 7fbaff400700 1 -- 172.21.15.7:0/4166043650 wait complete. 2024-06-13T07:05:47.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:47 smithi007 ceph-mon[32077]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:47 smithi189 ceph-mon[36616]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:47 smithi175 ceph-mon[36886]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-06-13T07:05:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 585.0 s, 488.8 tps, lat 224.108 ms stddev 264.881 2024-06-13T07:05:47.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:05:47.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:05:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:05:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:49 smithi189 ceph-mon[36616]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 MiB/s wr, 566 op/s 2024-06-13T07:05:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:49 smithi175 ceph-mon[36886]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 MiB/s wr, 566 op/s 2024-06-13T07:05:49.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:49 smithi007 ceph-mon[32077]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.7 MiB/s wr, 566 op/s 2024-06-13T07:05:51.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:51 smithi007 ceph-mon[32077]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:05:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:51 smithi189 ceph-mon[36616]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:05:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:51 smithi175 ceph-mon[36886]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 MiB/s wr, 572 op/s 2024-06-13T07:05:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 590.0 s, 400.0 tps, lat 240.640 ms stddev 305.132 2024-06-13T07:05:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 -- 172.21.15.7:0/293113131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a0074fe0 msgr2=0x7f71a010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 --2- 172.21.15.7:0/293113131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a0074fe0 0x7f71a010e9b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f71900098f0 tx=0x7f7190009c00 comp rx=0 tx=0).stop 2024-06-13T07:05:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 -- 172.21.15.7:0/293113131 shutdown_connections 2024-06-13T07:05:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 --2- 172.21.15.7:0/293113131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a010eef0 0x7f71a01112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 --2- 172.21.15.7:0/293113131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a00746c0 0x7f71a0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 --2- 172.21.15.7:0/293113131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a0074fe0 0x7f71a010e9b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.881+0000 7f71a6c00700 1 -- 172.21.15.7:0/293113131 >> 172.21.15.7:0/293113131 conn(0x7f71a00795f0 msgr2=0x7f71a0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.882+0000 7f71a6c00700 1 -- 172.21.15.7:0/293113131 shutdown_connections 2024-06-13T07:05:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.882+0000 7f71a6c00700 1 -- 172.21.15.7:0/293113131 wait complete. 2024-06-13T07:05:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.883+0000 7f71a6c00700 1 Processor -- start 2024-06-13T07:05:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.883+0000 7f71a6c00700 1 -- start start 2024-06-13T07:05:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00746c0 0x7f71a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 0x7f71a019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 0x7f71a019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0114a30 con 0x7f71a00746c0 2024-06-13T07:05:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71a01148b0 con 0x7f71a0074fe0 2024-06-13T07:05:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.884+0000 7f71a6c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0114bb0 con 0x7f71a010eef0 2024-06-13T07:05:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.885+0000 7f719f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 0x7f71a019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 0x7f71a019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f71a4c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 0x7f71a019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51354/0 (socket says 172.21.15.7:51354) 2024-06-13T07:05:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f71a4c00700 1 -- 172.21.15.7:0/2122577314 learned_addr learned my addr 172.21.15.7:0/2122577314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f719f400700 1 -- 172.21.15.7:0/2122577314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 msgr2=0x7f71a019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f719f400700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 0x7f71a019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f719f400700 1 -- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00746c0 msgr2=0x7f71a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f719f400700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00746c0 0x7f71a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.886+0000 7f719f400700 1 -- 172.21.15.7:0/2122577314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a019fe10 con 0x7f71a0074fe0 2024-06-13T07:05:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.889+0000 7f719f400700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 0x7f71a019b340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7190000c00 tx=0x7f7190009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.890+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190005270 con 0x7f71a0074fe0 2024-06-13T07:05:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.890+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719000ba40 con 0x7f71a0074fe0 2024-06-13T07:05:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.890+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190003c70 con 0x7f71a0074fe0 2024-06-13T07:05:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.891+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a01a00a0 con 0x7f71a0074fe0 2024-06-13T07:05:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.891+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01a7980 con 0x7f71a0074fe0 2024-06-13T07:05:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.893+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a0069180 con 0x7f71a0074fe0 2024-06-13T07:05:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.896+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f719002c440 con 0x7f71a0074fe0 2024-06-13T07:05:52.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.899+0000 7f719cc00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 0x7f71840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:52.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.899+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f71900d02f0 con 0x7f71a0074fe0 2024-06-13T07:05:52.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.899+0000 7f719fe00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 0x7f71840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:52.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.903+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7190013030 con 0x7f71a0074fe0 2024-06-13T07:05:52.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:52.910+0000 7f719fe00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 0x7f71840a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f718c005950 tx=0x7f718c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:53.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.369+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71a01a7b50 con 0x7f71a0074fe0 2024-06-13T07:05:53.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.370+0000 7f719cc00700 1 -- 172.21.15.7:0/2122577314 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11868 (secure 0 0 0) 0x7f719009d040 con 0x7f71a0074fe0 2024-06-13T07:05:53.372 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:05:53.372 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:04:39.673793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 msgr2=0x7f71840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 0x7f71840a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f718c005950 tx=0x7f718c0058e0 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 msgr2=0x7f71a019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 0x7f71a019b340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7190000c00 tx=0x7f7190009480 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 shutdown_connections 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a010eef0 0x7f71a019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00746c0 0x7f71a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f71840a2ca0 0x7f71840a5160 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 --2- 172.21.15.7:0/2122577314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71a0074fe0 0x7f71a019b340 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.373+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 >> 172.21.15.7:0/2122577314 conn(0x7f71a00795f0 msgr2=0x7f71a00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:53.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.374+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 shutdown_connections 2024-06-13T07:05:53.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:53.374+0000 7f71a6c00700 1 -- 172.21.15.7:0/2122577314 wait complete. 2024-06-13T07:05:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-06-13T07:05:53.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:53 smithi007 ceph-mon[32077]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-06-13T07:05:53.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:53 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2122577314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:05:53.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:53 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:53 smithi189 ceph-mon[36616]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-06-13T07:05:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:53 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2122577314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:05:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:53 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:53 smithi175 ceph-mon[36886]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 553 op/s 2024-06-13T07:05:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:53 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2122577314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:05:53.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:53 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:05:55.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:05:55.830 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:05:55.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:55 smithi189 ceph-mon[36616]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-06-13T07:05:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:55 smithi175 ceph-mon[36886]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-06-13T07:05:55.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:55 smithi007 ceph-mon[32077]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2024-06-13T07:05:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:56 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:56 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:56.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:56 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:05:56.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:56 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:56.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:56 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:56.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:56 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:05:56.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:56 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:05:56.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:56 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:05:56.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:56 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:05:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.660+0000 7f73a6600640 1 -- 172.21.15.7:0/2900958338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0075980 msgr2=0x7f73a0073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.660+0000 7f73a6600640 1 --2- 172.21.15.7:0/2900958338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0075980 0x7f73a0073dd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7398009ca0 tx=0x7f739802a950 comp rx=0 tx=0).stop 2024-06-13T07:05:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 -- 172.21.15.7:0/2900958338 shutdown_connections 2024-06-13T07:05:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 --2- 172.21.15.7:0/2900958338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0109890 0x7f73a010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 --2- 172.21.15.7:0/2900958338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a00743a0 0x7f73a0074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 --2- 172.21.15.7:0/2900958338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a0075980 0x7f73a0073dd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 -- 172.21.15.7:0/2900958338 >> 172.21.15.7:0/2900958338 conn(0x7f73a00fcf90 msgr2=0x7f73a00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 -- 172.21.15.7:0/2900958338 shutdown_connections 2024-06-13T07:05:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.663+0000 7f73a6600640 1 -- 172.21.15.7:0/2900958338 wait complete. 2024-06-13T07:05:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.664+0000 7f73a6600640 1 Processor -- start 2024-06-13T07:05:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.665+0000 7f73a6600640 1 -- start start 2024-06-13T07:05:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.665+0000 7f73a6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.665+0000 7f73a6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 0x7f73a0070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.665+0000 7f73a6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a0109890 0x7f73a01a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f739f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f739f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51378/0 (socket says 172.21.15.7:51378) 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010f8a0 con 0x7f73a0075980 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f739f400640 1 -- 172.21.15.7:0/2893089035 learned_addr learned my addr 172.21.15.7:0/2893089035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010f720 con 0x7f73a0109890 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010fa20 con 0x7f73a00743a0 2024-06-13T07:05:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f739fe00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a0109890 0x7f73a01a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.666+0000 7f739ea00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 0x7f73a0070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739ea00640 1 -- 172.21.15.7:0/2893089035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 msgr2=0x7f73a0070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739ea00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739ea00640 1 -- 172.21.15.7:0/2893089035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a0109890 msgr2=0x7f73a01a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739ea00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a0109890 0x7f73a01a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739ea00640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a0073890 con 0x7f73a0075980 2024-06-13T07:05:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.667+0000 7f739f400640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.669+0000 7f739ea00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 0x7f73a0070760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9f0 tx=0x7f738800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.670+0000 7f7397e00640 1 -- 172.21.15.7:0/2893089035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738800cd80 con 0x7f73a0075980 2024-06-13T07:05:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.670+0000 7f7397e00640 1 -- 172.21.15.7:0/2893089035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73880035f0 con 0x7f73a0075980 2024-06-13T07:05:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.670+0000 7f7397e00640 1 -- 172.21.15.7:0/2893089035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388010690 con 0x7f73a0075980 2024-06-13T07:05:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.670+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01aa4f0 con 0x7f73a0075980 2024-06-13T07:05:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.671+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01aa9c0 con 0x7f73a0075980 2024-06-13T07:05:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.676+0000 7f73a9407740 1 Processor -- start 2024-06-13T07:05:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.676+0000 7f73a9407740 1 -- start start 2024-06-13T07:05:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5bde70 0x56076d5c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56076d520420 con 0x56076d5b8d00 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56076d4c7c50 con 0x56076d5b8310 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f73a9407740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56076d51c3e0 con 0x56076d5bde70 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35576/0 (socket says 172.21.15.7:35576) 2024-06-13T07:05:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 -- 172.21.15.7:0/3990778061 learned_addr learned my addr 172.21.15.7:0/3990778061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 -- 172.21.15.7:0/3990778061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5bde70 msgr2=0x56076d5c0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5bde70 0x56076d5c0300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 -- 172.21.15.7:0/3990778061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 msgr2=0x56076d5bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.677+0000 7f739fe00640 1 -- 172.21.15.7:0/3990778061 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076d234210 con 0x56076d5b8310 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.678+0000 7f739fe00640 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f739000ac10 tx=0x7f739000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d0084e5662ec45 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.678+0000 7f7394c00640 1 -- 172.21.15.7:0/3990778061 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390005220 con 0x56076d5b8310 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.678+0000 7f7394c00640 1 -- 172.21.15.7:0/3990778061 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390004c80 con 0x56076d5b8310 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.678+0000 7f7394c00640 1 -- 172.21.15.7:0/3990778061 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390018870 con 0x56076d5b8310 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.679+0000 7f73a9407740 1 -- 172.21.15.7:0/3990778061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 msgr2=0x56076d5b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.679+0000 7f73a9407740 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f739000ac10 tx=0x7f739000f790 comp rx=0 tx=0).stop 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 -- 172.21.15.7:0/3990778061 shutdown_connections 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5bde70 0x56076d5c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 --2- 172.21.15.7:0/3990778061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5b8730 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 -- 172.21.15.7:0/3990778061 >> 172.21.15.7:0/3990778061 conn(0x56076d584fb0 msgr2=0x56076d5c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f7397e00640 1 -- 172.21.15.7:0/2893089035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f73880037a0 con 0x7f73a0075980 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.680+0000 7f73a9407740 1 -- 172.21.15.7:0/3990778061 shutdown_connections 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.681+0000 7f73a9407740 1 -- 172.21.15.7:0/3990778061 wait complete. 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.681+0000 7f73a9407740 1 -- start start 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.682+0000 7f73a9407740 1 Processor -- start 2024-06-13T07:05:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8d00 0x56076d5d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56076d5d6720 con 0x56076d5dd660 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56076d4db0b0 con 0x56076d5b8d00 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56076d5d68a0 con 0x56076d5b8310 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f73a9407740 10 client.0 _ll_register_callbacks cb 0x56076d4c6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f739ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f739ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57938/0 (socket says 172.21.15.7:57938) 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.683+0000 7f739ea00640 1 -- 172.21.15.7:0/906077989 learned_addr learned my addr 172.21.15.7:0/906077989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.684+0000 7f739ea00640 1 -- 172.21.15.7:0/906077989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 msgr2=0x56076d5d80f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.684+0000 7f739ea00640 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5d80f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.684+0000 7f739ea00640 1 -- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8d00 msgr2=0x56076d5d8630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.684+0000 7f739ea00640 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8d00 0x56076d5d8630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.684+0000 7f739ea00640 1 -- 172.21.15.7:0/906077989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076d300620 con 0x56076d5dd660 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.685+0000 7f739ea00640 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73880ccd80 tx=0x7f73880d5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc603d5ed33cbc97 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.685+0000 7f738d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.685+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73880e2060 con 0x56076d5dd660 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.685+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f73880d7790 con 0x56076d5dd660 2024-06-13T07:05:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.686+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7390004c80 con 0x56076d5dd660 2024-06-13T07:05:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.694+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f73880e9050 con 0x56076d5dd660 2024-06-13T07:05:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.694+0000 7f7397e00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 0x7f73840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.694+0000 7f7397e00640 1 -- 172.21.15.7:0/2893089035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f73880cf6b0 con 0x7f73a0075980 2024-06-13T07:05:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.695+0000 7f739fe00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 0x7f73840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.695+0000 7f73a9407740 10 client.16218 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.696+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56076d5c51a0 con 0x56076d5dd660 2024-06-13T07:05:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.696+0000 7f739fe00640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 0x7f73840a5170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f739000b010 tx=0x7f7390004840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.696+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390004c80 con 0x56076d5dd660 2024-06-13T07:05:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.697+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f73880981e0 con 0x56076d5dd660 2024-06-13T07:05:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 10 client.16218 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 20 client.16218 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d5c68e0 0x56076d5c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 4 client.16218 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56076d52db50 con 0x56076d5c68e0 2024-06-13T07:05:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.698+0000 7f739f400640 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d5c68e0 0x56076d5c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.699+0000 7f739f400640 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d5c68e0 0x56076d5c8d40 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914d1d12f62d8114 server_cookie=3d579e8411726206 in_seq=0 out_seq=0 2024-06-13T07:05:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.699+0000 7f738d600640 10 client.16218 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.700+0000 7f738d600640 1 -- 172.21.15.7:0/906077989 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56076d52db50 con 0x56076d5c68e0 2024-06-13T07:05:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.700+0000 7f738d600640 10 client.16218 handle_command_reply: tid=0 2024-06-13T07:05:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.700+0000 7f73a9407740 1 client.16218 shutdown 2024-06-13T07:05:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 msgr2=0x56076d5dfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73880ccd80 tx=0x7f73880d5f10 comp rx=0 tx=0).stop 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 shutdown_connections 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d5c68e0 0x56076d5c8d40 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5dd660 0x56076d5dfa50 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73880ccd80 tx=0x7f73880d5f10 comp rx=0 tx=0).stop 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8d00 0x56076d5d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 --2- 172.21.15.7:0/906077989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 >> 172.21.15.7:0/906077989 conn(0x56076d584fb0 msgr2=0x56076d5b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.702+0000 7f738d600640 0 client.16218 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.703+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 shutdown_connections 2024-06-13T07:05:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.703+0000 7f73a9407740 1 -- 172.21.15.7:0/906077989 wait complete. 2024-06-13T07:05:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.703+0000 7f73a9407740 20 client.16218 trim_cache size 0 max 16384 2024-06-13T07:05:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.737+0000 7f73a9407740 1 Processor -- start 2024-06-13T07:05:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 -- start start 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 0x56076d5d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d52b0 con 0x56076d5b8d00 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d2b80 con 0x56076d5ba600 2024-06-13T07:05:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.738+0000 7f73a9407740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d0a90 con 0x56076d5b8310 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.739+0000 7f739fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 0x56076d5d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.739+0000 7f739ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.739+0000 7f739ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57952/0 (socket says 172.21.15.7:57952) 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.739+0000 7f739ea00640 1 -- 172.21.15.7:0/73328336 learned_addr learned my addr 172.21.15.7:0/73328336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.739+0000 7f739f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739ea00640 1 -- 172.21.15.7:0/73328336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 msgr2=0x56076d5bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739ea00640 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5bd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739ea00640 1 -- 172.21.15.7:0/73328336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 msgr2=0x56076d5d80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739ea00640 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 0x56076d5d80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739ea00640 1 -- 172.21.15.7:0/73328336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076d5b9590 con 0x56076d5b8d00 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739f400640 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.740+0000 7f739fe00640 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 0x56076d5d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.741+0000 7f739ea00640 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9c0 tx=0x7f73880ccdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8f1d611d68d8a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.742+0000 7f738fe00640 1 -- 172.21.15.7:0/73328336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388097e10 con 0x56076d5b8d00 2024-06-13T07:05:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.742+0000 7f738fe00640 1 -- 172.21.15.7:0/73328336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738809bb70 con 0x56076d5b8d00 2024-06-13T07:05:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.742+0000 7f738fe00640 1 -- 172.21.15.7:0/73328336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73880a47e0 con 0x56076d5b8d00 2024-06-13T07:05:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.743+0000 7f73a9407740 1 -- 172.21.15.7:0/73328336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 msgr2=0x56076d5ba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.743+0000 7f73a9407740 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9c0 tx=0x7f73880ccdb0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.744+0000 7f73a9407740 1 -- 172.21.15.7:0/73328336 shutdown_connections 2024-06-13T07:05:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.744+0000 7f73a9407740 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5ba600 0x56076d5d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.744+0000 7f73a9407740 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5ba0c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.744+0000 7f73a9407740 1 --2- 172.21.15.7:0/73328336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5b8310 0x56076d5bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.744+0000 7f73a9407740 1 -- 172.21.15.7:0/73328336 >> 172.21.15.7:0/73328336 conn(0x56076d584fb0 msgr2=0x56076d5c0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.746+0000 7f73a9407740 1 -- 172.21.15.7:0/73328336 shutdown_connections 2024-06-13T07:05:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.748+0000 7f73a9407740 1 -- 172.21.15.7:0/73328336 wait complete. 2024-06-13T07:05:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.749+0000 7f73a9407740 1 -- start start 2024-06-13T07:05:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.749+0000 7f73a9407740 1 Processor -- start 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.755+0000 7f73a9407740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d1040 con 0x56076d5b8d00 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d24b0 con 0x56076d5b8310 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73a01d1710 con 0x56076d5ba600 2024-06-13T07:05:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f73a9407740 10 client.0 _ll_register_callbacks cb 0x7f73880d2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f739fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f739fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51386/0 (socket says 172.21.15.7:51386) 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.756+0000 7f739fe00640 1 -- 172.21.15.7:0/271520244 learned_addr learned my addr 172.21.15.7:0/271520244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739f400640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739ea00640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 -- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 msgr2=0x56076d5e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 -- 172.21.15.7:0/271520244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 msgr2=0x56076d5e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 -- 172.21.15.7:0/271520244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076d5e80e0 con 0x56076d5ba600 2024-06-13T07:05:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739ea00640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f739fe00640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x56076d5b9560 tx=0x7f739000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcaa8bf60eef0b98 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.757+0000 7f738d600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.758+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390034020 con 0x56076d5ba600 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.758+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7390003b40 con 0x56076d5ba600 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.758+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738809bb70 con 0x56076d5ba600 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.758+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f739003f050 con 0x56076d5ba600 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.759+0000 7f73a9407740 10 client.26755 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.759+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56076d63d6a0 con 0x56076d5ba600 2024-06-13T07:05:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.759+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738809bb70 con 0x56076d5ba600 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f7390018440 con 0x56076d5ba600 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 10 client.26755 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 20 client.26755 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d63df60 0x56076d640380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 4 client.26755 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:05:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56076d527ea0 con 0x56076d63df60 2024-06-13T07:05:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.760+0000 7f739f400640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d63df60 0x56076d640380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.761+0000 7f739f400640 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d63df60 0x56076d640380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c89c56b5ff2213 server_cookie=b70bcdf8f3d2bded in_seq=0 out_seq=0 2024-06-13T07:05:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.761+0000 7f738d600640 10 client.26755 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.787+0000 7f738d600640 1 -- 172.21.15.7:0/271520244 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56076d527ea0 con 0x56076d63df60 2024-06-13T07:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.787+0000 7f738d600640 10 client.26755 handle_command_reply: tid=0 2024-06-13T07:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.787+0000 7f73a9407740 1 client.26755 shutdown 2024-06-13T07:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 msgr2=0x56076d5e7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x56076d5b9560 tx=0x7f739000f790 comp rx=0 tx=0).stop 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 shutdown_connections 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x56076d63df60 0x56076d640380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076d5ba600 0x56076d5e7af0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076d5b8d00 0x56076d5e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 --2- 172.21.15.7:0/271520244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076d5b8310 0x56076d5e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f738d600640 0 client.26755 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.788+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 >> 172.21.15.7:0/271520244 conn(0x56076d584fb0 msgr2=0x56076d5bd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.791+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 shutdown_connections 2024-06-13T07:05:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.791+0000 7f73a9407740 1 -- 172.21.15.7:0/271520244 wait complete. 2024-06-13T07:05:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.791+0000 7f73a9407740 20 client.26755 trim_cache size 0 max 16384 2024-06-13T07:05:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:05:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:05:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:05:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.792+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 msgr2=0x7f73840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.792+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 0x7f73840a5170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f739000b010 tx=0x7f7390004840 comp rx=0 tx=0).stop 2024-06-13T07:05:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.792+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 msgr2=0x7f73a0070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.792+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 0x7f73a0070760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9f0 tx=0x7f738800eeb0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 shutdown_connections 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73840a2cb0 0x7f73840a5170 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73a0109890 0x7f73a01a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a0075980 0x7f73a0070760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 --2- 172.21.15.7:0/2893089035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73a00743a0 0x7f73a0070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 >> 172.21.15.7:0/2893089035 conn(0x7f73a00fcf90 msgr2=0x7f73a0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 shutdown_connections 2024-06-13T07:05:57.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:05:57.793+0000 7f73a6600640 1 -- 172.21.15.7:0/2893089035 wait complete. 2024-06-13T07:05:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:57 smithi189 ceph-mon[36616]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-06-13T07:05:57.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:57 smithi175 ceph-mon[36886]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-06-13T07:05:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 595.0 s, 451.4 tps, lat 215.717 ms stddev 308.971 2024-06-13T07:05:57.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:57 smithi007 ceph-mon[32077]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 519 op/s 2024-06-13T07:05:59.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 -- 172.21.15.7:0/2928156626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc104ed0 msgr2=0x7fa2fc105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 --2- 172.21.15.7:0/2928156626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc104ed0 0x7fa2fc105360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009990 tx=0x7fa2f0009ca0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 -- 172.21.15.7:0/2928156626 shutdown_connections 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 --2- 172.21.15.7:0/2928156626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc104ed0 0x7fa2fc105360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 --2- 172.21.15.7:0/2928156626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 --2- 172.21.15.7:0/2928156626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2fc06a6e0 0x7fa2fc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 -- 172.21.15.7:0/2928156626 >> 172.21.15.7:0/2928156626 conn(0x7fa2fc078340 msgr2=0x7fa2fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 -- 172.21.15.7:0/2928156626 shutdown_connections 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.088+0000 7fa304400700 1 -- 172.21.15.7:0/2928156626 wait complete. 2024-06-13T07:05:59.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.090+0000 7fa304400700 1 Processor -- start 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 -- start start 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc06a6e0 0x7fa2fc073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2fc104ed0 0x7fa2fc0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc118e40 con 0x7fa2fc104ed0 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.091+0000 7fa304400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc118cc0 con 0x7fa2fc06a6e0 2024-06-13T07:05:59.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51406/0 (socket says 172.21.15.7:51406) 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 -- 172.21.15.7:0/3204414060 learned_addr learned my addr 172.21.15.7:0/3204414060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc118fc0 con 0x7fa2fc1073c0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa301a00700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc06a6e0 0x7fa2fc073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 -- 172.21.15.7:0/3204414060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc06a6e0 msgr2=0x7fa2fc073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc06a6e0 0x7fa2fc073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 -- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2fc104ed0 msgr2=0x7fa2fc0740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2fc104ed0 0x7fa2fc0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.092+0000 7fa302400700 1 -- 172.21.15.7:0/3204414060 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc06d6b0 con 0x7fa2fc1073c0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.093+0000 7fa302400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009fb0 tx=0x7fa2f00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.094+0000 7fa2fa000700 1 -- 172.21.15.7:0/3204414060 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0048070 con 0x7fa2fc1073c0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.094+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc06d940 con 0x7fa2fc1073c0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.094+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc0718b0 con 0x7fa2fc1073c0 2024-06-13T07:05:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.095+0000 7fa2fa000700 1 -- 172.21.15.7:0/3204414060 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000f940 con 0x7fa2fc1073c0 2024-06-13T07:05:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.095+0000 7fa2fa000700 1 -- 172.21.15.7:0/3204414060 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0048070 con 0x7fa2fc1073c0 2024-06-13T07:05:59.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.104+0000 7fa2fa000700 1 -- 172.21.15.7:0/3204414060 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa2f0005080 con 0x7fa2fc1073c0 2024-06-13T07:05:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.107+0000 7fa30ee76b80 1 Processor -- start 2024-06-13T07:05:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.107+0000 7fa30ee76b80 1 -- start start 2024-06-13T07:05:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adf9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604adea8a10 con 0x5604adfa04b0 2024-06-13T07:05:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5604adeebb00 con 0x5604adf9fb90 2024-06-13T07:05:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5604adea68f0 con 0x5604adf9a240 2024-06-13T07:05:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa2fa000700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 0x7fa2e40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35608/0 (socket says 172.21.15.7:35608) 2024-06-13T07:05:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.108+0000 7fa301a00700 1 -- 172.21.15.7:0/432376632 learned_addr learned my addr 172.21.15.7:0/432376632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa302400700 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adf9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa301a00700 1 -- 172.21.15.7:0/432376632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 msgr2=0x5604adf9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa301a00700 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adf9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa301a00700 1 -- 172.21.15.7:0/432376632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 msgr2=0x5604adf99d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa301a00700 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.109+0000 7fa301a00700 1 -- 172.21.15.7:0/432376632 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ade93f90 con 0x5604adf9fb90 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.110+0000 7fa301a00700 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400b730 tx=0x7fa2f400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f42670c83efdfaa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.110+0000 7fa301a00700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 0x7fa2e40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.111+0000 7fa2fa000700 1 -- 172.21.15.7:0/3204414060 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa2f000f4c0 con 0x7fa2fc1073c0 2024-06-13T07:05:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.115+0000 7fa2eaa00700 1 -- 172.21.15.7:0/432376632 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4016c80 con 0x5604adf9fb90 2024-06-13T07:05:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.115+0000 7fa2eaa00700 1 -- 172.21.15.7:0/432376632 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f4016e20 con 0x5604adf9fb90 2024-06-13T07:05:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.115+0000 7fa2eaa00700 1 -- 172.21.15.7:0/432376632 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4008830 con 0x5604adf9fb90 2024-06-13T07:05:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.116+0000 7fa30ee76b80 1 -- 172.21.15.7:0/432376632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 msgr2=0x5604adf9ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.116+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400b730 tx=0x7fa2f400ba40 comp rx=0 tx=0).stop 2024-06-13T07:05:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.116+0000 7fa301a00700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 0x7fa2e40a5360 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2f401d000 tx=0x7fa2f4008bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 -- 172.21.15.7:0/432376632 shutdown_connections 2024-06-13T07:05:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adf9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9fb90 0x5604adf9ff70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/432376632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf99d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 -- 172.21.15.7:0/432376632 >> 172.21.15.7:0/432376632 conn(0x5604adec6170 msgr2=0x5604adf94ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.117+0000 7fa30ee76b80 1 -- 172.21.15.7:0/432376632 shutdown_connections 2024-06-13T07:05:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.118+0000 7fa30ee76b80 1 -- 172.21.15.7:0/432376632 wait complete. 2024-06-13T07:05:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.118+0000 7fa30ee76b80 1 -- start start 2024-06-13T07:05:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.119+0000 7fa30ee76b80 1 Processor -- start 2024-06-13T07:05:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.120+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adfa61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.121+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adf9fb90 0x5604adfa34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.121+0000 7fa301a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adfa61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa301a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adfa61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51432/0 (socket says 172.21.15.7:51432) 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa301a00700 1 -- 172.21.15.7:0/3240754959 learned_addr learned my addr 172.21.15.7:0/3240754959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 0x5604adfa3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa301000700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adf9fb90 0x5604adfa34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604ade87f60 con 0x5604adf9fb90 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa302400700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 0x5604adfa3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5604adee7ac0 con 0x5604adfa04b0 2024-06-13T07:05:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.122+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5604adfa6e90 con 0x5604adf9a240 2024-06-13T07:05:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.123+0000 7fa30ee76b80 10 client.0 _ll_register_callbacks cb 0x5604ade81380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 -- 172.21.15.7:0/3240754959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 msgr2=0x5604adfa61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adfa61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 -- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adf9fb90 msgr2=0x5604adfa34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adf9fb90 0x5604adfa34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604ade3a8d0 con 0x5604adfa04b0 2024-06-13T07:05:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa302400700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 0x5604adfa3a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0043040 tx=0x7fa2f0043f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba8be20b814f849 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.124+0000 7fa2df400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:05:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.127+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00be570 con 0x5604adfa04b0 2024-06-13T07:05:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.127+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa2f00bee70 con 0x5604adfa04b0 2024-06-13T07:05:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.127+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f4016e20 con 0x5604adfa04b0 2024-06-13T07:05:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa2f00bf310 con 0x5604adfa04b0 2024-06-13T07:05:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa30ee76b80 10 client.26837 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5604adfb1650 con 0x5604adfa04b0 2024-06-13T07:05:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f4016e20 con 0x5604adfa04b0 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fa2f00b9390 con 0x5604adfa04b0 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa30ee76b80 10 client.26837 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa30ee76b80 10 client.26837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.132+0000 7fa30ee76b80 20 client.26837 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.133+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604adfb3a20 0x5604adfb5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.133+0000 7fa30ee76b80 4 client.26837 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:05:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.133+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604adef9000 con 0x5604adfb3a20 2024-06-13T07:05:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.133+0000 7fa301a00700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604adfb3a20 0x5604adfb5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.138+0000 7fa301a00700 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604adfb3a20 0x5604adfb5e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8261ed55ccc3ac7f server_cookie=2a55fb87bc91ba64 in_seq=0 out_seq=0 2024-06-13T07:05:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.138+0000 7fa2df400700 10 client.26837 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:05:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.140+0000 7fa2df400700 1 -- 172.21.15.7:0/3240754959 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa2f4016940 con 0x5604adfb3a20 2024-06-13T07:05:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.140+0000 7fa2df400700 10 client.26837 handle_command_reply: tid=0 2024-06-13T07:05:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.140+0000 7fa30ee76b80 1 client.26837 shutdown 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 msgr2=0x5604adfa3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 0x5604adfa3a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0043040 tx=0x7fa2f0043f90 comp rx=0 tx=0).stop 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 shutdown_connections 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604adfb3a20 0x5604adfb5e40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adfa04b0 0x5604adfa3a30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa2df400700 0 client.26837 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9a240 0x5604adfa61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/3240754959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adf9fb90 0x5604adfa34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 >> 172.21.15.7:0/3240754959 conn(0x5604adec6170 msgr2=0x5604adf8e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.141+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 shutdown_connections 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.142+0000 7fa30ee76b80 1 -- 172.21.15.7:0/3240754959 wait complete. 2024-06-13T07:05:59.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.142+0000 7fa30ee76b80 20 client.26837 trim_cache size 0 max 16384 2024-06-13T07:05:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.176+0000 7fa30ee76b80 1 Processor -- start 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 -- start start 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adf951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e00112f0 con 0x5604adfa04b0 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000eb80 con 0x5604adf9a240 2024-06-13T07:05:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.177+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000c9d0 con 0x5604adf9fb90 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adf951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adf951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35638/0 (socket says 172.21.15.7:35638) 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51436/0 (socket says 172.21.15.7:51436) 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301a00700 1 -- 172.21.15.7:0/1467871075 learned_addr learned my addr 172.21.15.7:0/1467871075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 -- 172.21.15.7:0/1467871075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 msgr2=0x5604adf951b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adf951b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 -- 172.21.15.7:0/1467871075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 msgr2=0x5604adf8f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.178+0000 7fa301000700 1 -- 172.21.15.7:0/1467871075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604adfa1a60 con 0x5604adf9fb90 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.179+0000 7fa301000700 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5604adfa4ac0 tx=0x7fa2ec007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138419943d46b819 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.179+0000 7fa2ea000700 1 -- 172.21.15.7:0/1467871075 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec010a50 con 0x5604adf9fb90 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.179+0000 7fa2ea000700 1 -- 172.21.15.7:0/1467871075 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ec010bf0 con 0x5604adf9fb90 2024-06-13T07:05:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.179+0000 7fa2ea000700 1 -- 172.21.15.7:0/1467871075 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0195f0 con 0x5604adf9fb90 2024-06-13T07:05:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.180+0000 7fa30ee76b80 1 -- 172.21.15.7:0/1467871075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 msgr2=0x5604adfb5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.180+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5604adfa4ac0 tx=0x7fa2ec007e00 comp rx=0 tx=0).stop 2024-06-13T07:05:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.182+0000 7fa30ee76b80 1 -- 172.21.15.7:0/1467871075 shutdown_connections 2024-06-13T07:05:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.182+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604adf8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.183+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adf951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.183+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/1467871075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfb5e10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.184+0000 7fa30ee76b80 1 -- 172.21.15.7:0/1467871075 >> 172.21.15.7:0/1467871075 conn(0x5604adec6170 msgr2=0x5604adf9eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.185+0000 7fa30ee76b80 1 -- 172.21.15.7:0/1467871075 shutdown_connections 2024-06-13T07:05:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.185+0000 7fa30ee76b80 1 -- 172.21.15.7:0/1467871075 wait complete. 2024-06-13T07:05:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.186+0000 7fa30ee76b80 1 -- start start 2024-06-13T07:05:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.186+0000 7fa30ee76b80 1 Processor -- start 2024-06-13T07:05:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adfa9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfa97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604ae01c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adfa9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa301000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfa97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa301a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adfa9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:35648/0 (socket says 172.21.15.7:35648) 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000d040 con 0x5604adfa04b0 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa301a00700 1 -- 172.21.15.7:0/2693988697 learned_addr learned my addr 172.21.15.7:0/2693988697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000e4b0 con 0x5604adf9a240 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000d710 con 0x5604adf9fb90 2024-06-13T07:05:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.187+0000 7fa30ee76b80 10 client.0 _ll_register_callbacks cb 0x5604ade81380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 -- 172.21.15.7:0/2693988697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 msgr2=0x5604adfa9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adfa9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 -- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 msgr2=0x5604ae01c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604ae01c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604adfaa060 con 0x5604adf9fb90 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa301000700 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfa97c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5604adfa4c50 tx=0x7fa2ec011c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c0cf885df14d8c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.188+0000 7fa2df400700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:05:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.189+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec01f040 con 0x5604adf9fb90 2024-06-13T07:05:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.189+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ec010bf0 con 0x5604adf9fb90 2024-06-13T07:05:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.189+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa2ec010850 con 0x5604adf9fb90 2024-06-13T07:05:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.190+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa2ec032050 con 0x5604adf9fb90 2024-06-13T07:05:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.190+0000 7fa30ee76b80 10 client.26773 fetch_fsmap learned FSMap version 38 2024-06-13T07:05:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.190+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5604ae01dd10 con 0x5604adf9fb90 2024-06-13T07:05:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.190+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ec010bf0 con 0x5604adf9fb90 2024-06-13T07:05:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.190+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fa2ec022030 con 0x5604adf9fb90 2024-06-13T07:05:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:05:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 10 client.26773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:05:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 20 client.26773 populate_metadata read hostname 'smithi007' 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604ae01e300 0x5604ae020720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 4 client.26773 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.191+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604adef3580 con 0x5604ae01e300 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.192+0000 7fa301a00700 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604ae01e300 0x5604ae020720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.193+0000 7fa301a00700 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604ae01e300 0x5604ae020720 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5da1b823d1fb0d0c server_cookie=c672a96d89db5e33 in_seq=0 out_seq=0 2024-06-13T07:05:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.193+0000 7fa2df400700 10 client.26773 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:05:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa2df400700 1 -- 172.21.15.7:0/2693988697 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604adef3580 con 0x5604ae01e300 2024-06-13T07:05:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa2df400700 10 client.26773 handle_command_reply: tid=0 2024-06-13T07:05:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 client.26773 shutdown 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 msgr2=0x5604adfa97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfa97c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5604adfa4c50 tx=0x7fa2ec011c10 comp rx=0 tx=0).stop 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 shutdown_connections 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604adfa04b0 0x5604ae01c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.194+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x5604ae01e300 0x5604ae020720 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5604adf9a240 0x5604adfa9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa30ee76b80 1 --2- 172.21.15.7:0/2693988697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604adf9fb90 0x5604adfa97c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 >> 172.21.15.7:0/2693988697 conn(0x5604adec6170 msgr2=0x5604adf98cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa2df400700 0 client.26773 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 shutdown_connections 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.195+0000 7fa30ee76b80 1 -- 172.21.15.7:0/2693988697 wait complete. 2024-06-13T07:05:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.196+0000 7fa30ee76b80 20 client.26773 trim_cache size 0 max 16384 2024-06-13T07:05:59.197 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:05:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.201+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 msgr2=0x7fa2e40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.201+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 0x7fa2e40a5360 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2f401d000 tx=0x7fa2f4008bc0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.201+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 msgr2=0x7fa2fc06ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:05:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.201+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009fb0 tx=0x7fa2f00057c0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 shutdown_connections 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2fc1073c0 0x7fa2fc06ce40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa2e40a2ea0 0x7fa2e40a5360 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2fc06a6e0 0x7fa2fc073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 --2- 172.21.15.7:0/3204414060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2fc104ed0 0x7fa2fc0740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 >> 172.21.15.7:0/3204414060 conn(0x7fa2fc078340 msgr2=0x7fa2fc103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 shutdown_connections 2024-06-13T07:05:59.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:05:59.202+0000 7fa304400700 1 -- 172.21.15.7:0/3204414060 wait complete. 2024-06-13T07:05:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:05:59 smithi189 ceph-mon[36616]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-06-13T07:05:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:05:59 smithi175 ceph-mon[36886]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-06-13T07:05:59.873 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:05:59 smithi007 ceph-mon[32077]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-06-13T07:06:00.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:06:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:06:01.562 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:01 smithi007 ceph-mon[32077]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-06-13T07:06:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:01 smithi189 ceph-mon[36616]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-06-13T07:06:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:01 smithi175 ceph-mon[36886]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2024-06-13T07:06:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 600.0 s, 429.8 tps, lat 240.637 ms stddev 378.339 2024-06-13T07:06:03.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:03 smithi007 ceph-mon[32077]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-06-13T07:06:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:03 smithi189 ceph-mon[36616]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-06-13T07:06:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:03 smithi175 ceph-mon[36886]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 496 op/s 2024-06-13T07:06:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 -- 172.21.15.7:0/3030214362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa01155e0 msgr2=0x7faaa01179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 --2- 172.21.15.7:0/3030214362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa01155e0 0x7faaa01179d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faa98009f90 tx=0x7faa9800b330 comp rx=0 tx=0).stop 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 -- 172.21.15.7:0/3030214362 shutdown_connections 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 --2- 172.21.15.7:0/3030214362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa01155e0 0x7faaa01179d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 --2- 172.21.15.7:0/3030214362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa0109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 --2- 172.21.15.7:0/3030214362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa0109db0 0x7faaa01150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.805+0000 7faaa4e00700 1 -- 172.21.15.7:0/3030214362 >> 172.21.15.7:0/3030214362 conn(0x7faaa006d600 msgr2=0x7faaa006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.806+0000 7faaa4e00700 1 -- 172.21.15.7:0/3030214362 shutdown_connections 2024-06-13T07:06:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.806+0000 7faaa4e00700 1 -- 172.21.15.7:0/3030214362 wait complete. 2024-06-13T07:06:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.808+0000 7faaa4e00700 1 Processor -- start 2024-06-13T07:06:04.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.808+0000 7faaa4e00700 1 -- start start 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0109db0 0x7faaa010b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa01155e0 0x7faaa010b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faaa011b040 con 0x7faaa01155e0 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaa011aec0 con 0x7faaa0109db0 2024-06-13T07:06:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faaa4e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaa011b1c0 con 0x7faaa0109490 2024-06-13T07:06:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faa9de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faa9de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52490/0 (socket says 172.21.15.7:52490) 2024-06-13T07:06:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.809+0000 7faa9de00700 1 -- 172.21.15.7:0/4262641704 learned_addr learned my addr 172.21.15.7:0/4262641704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9d400700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0109db0 0x7faaa010b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9de00700 1 -- 172.21.15.7:0/4262641704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0109db0 msgr2=0x7faaa010b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9de00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0109db0 0x7faaa010b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9de00700 1 -- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa01155e0 msgr2=0x7faaa010b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9de00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa01155e0 0x7faaa010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.810+0000 7faa9de00700 1 -- 172.21.15.7:0/4262641704 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa01a11f0 con 0x7faaa0109490 2024-06-13T07:06:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faa9de00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa9000d980 tx=0x7faa9000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faa96a00700 1 -- 172.21.15.7:0/4262641704 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90016070 con 0x7faaa0109490 2024-06-13T07:06:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faa96a00700 1 -- 172.21.15.7:0/4262641704 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9000de60 con 0x7faaa0109490 2024-06-13T07:06:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faa96a00700 1 -- 172.21.15.7:0/4262641704 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9000f620 con 0x7faaa0109490 2024-06-13T07:06:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa01a14e0 con 0x7faaa0109490 2024-06-13T07:06:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.811+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa01a19d0 con 0x7faaa0109490 2024-06-13T07:06:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.820+0000 7faaaf6bbb80 1 Processor -- start 2024-06-13T07:06:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.820+0000 7faa96a00700 1 -- 172.21.15.7:0/4262641704 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7faa90019070 con 0x7faaa0109490 2024-06-13T07:06:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.822+0000 7faaaf6bbb80 1 -- start start 2024-06-13T07:06:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfc9240 0x55962cfc9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55962ced7a10 con 0x55962cfcf4b0 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55962cf1ab00 con 0x55962cfceb90 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55962ced58f0 con 0x55962cfc9240 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49094/0 (socket says 172.21.15.7:49094) 2024-06-13T07:06:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9e800700 1 -- 172.21.15.7:0/1269903165 learned_addr learned my addr 172.21.15.7:0/1269903165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9d400700 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfc9240 0x55962cfc9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9e800700 1 -- 172.21.15.7:0/1269903165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfc9240 msgr2=0x55962cfc9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.823+0000 7faa9e800700 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfc9240 0x55962cfc9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa9e800700 1 -- 172.21.15.7:0/1269903165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 msgr2=0x55962cfc8d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa9e800700 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfc8d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa9e800700 1 -- 172.21.15.7:0/1269903165 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962cec2f90 con 0x55962cfceb90 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa9e800700 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faa98009c50 tx=0x7faa98009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13042d86a2adf06 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa8b400700 1 -- 172.21.15.7:0/1269903165 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9800cd30 con 0x55962cfceb90 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa8b400700 1 -- 172.21.15.7:0/1269903165 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa98007420 con 0x55962cfceb90 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.824+0000 7faa8b400700 1 -- 172.21.15.7:0/1269903165 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9802e680 con 0x55962cfceb90 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.825+0000 7faa96a00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 0x7faa840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.825+0000 7faa96a00700 1 -- 172.21.15.7:0/4262641704 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7faa900cee80 con 0x7faaa0109490 2024-06-13T07:06:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.825+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/1269903165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 msgr2=0x55962cfcef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.825+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faa98009c50 tx=0x7faa98009c80 comp rx=0 tx=0).stop 2024-06-13T07:06:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/1269903165 shutdown_connections 2024-06-13T07:06:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfc9240 0x55962cfc9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfcef70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/1269903165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfc8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/1269903165 >> 172.21.15.7:0/1269903165 conn(0x55962cef5170 msgr2=0x55962cfc3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faa9de00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 0x7faa840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.827+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/1269903165 shutdown_connections 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.828+0000 7faa9de00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 0x7faa840a51b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faa9000b2d0 tx=0x7faa90005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.829+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/1269903165 wait complete. 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.829+0000 7faaaf6bbb80 1 -- start start 2024-06-13T07:06:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.830+0000 7faaaf6bbb80 1 Processor -- start 2024-06-13T07:06:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.831+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfd5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.831+0000 7faa9e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfd5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.832+0000 7faa9e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfd5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37198/0 (socket says 172.21.15.7:37198) 2024-06-13T07:06:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.832+0000 7faa9e800700 1 -- 172.21.15.7:0/3511365121 learned_addr learned my addr 172.21.15.7:0/3511365121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.832+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfd2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfd2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55962ceb6f60 con 0x55962cfc9240 2024-06-13T07:06:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55962cf16ac0 con 0x55962cfceb90 2024-06-13T07:06:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faa9d400700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfd2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55962cfd5e30 con 0x55962cfcf4b0 2024-06-13T07:06:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.833+0000 7faaaf6bbb80 10 client.0 _ll_register_callbacks cb 0x55962ceb0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.835+0000 7faa9e800700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfd2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.835+0000 7faa9d400700 1 -- 172.21.15.7:0/3511365121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 msgr2=0x55962cfd2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa9d400700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfd2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa9d400700 1 -- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 msgr2=0x55962cfd5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa9d400700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfd5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa9d400700 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962ce698d0 con 0x55962cfceb90 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa9d400700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfd2510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005fd0 tx=0x7faa8c011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f99e8a5a48bfd1e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.836+0000 7faa7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.837+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004a30 con 0x55962cfceb90 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.837+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7faa8c004bd0 con 0x55962cfceb90 2024-06-13T07:06:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.837+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa98007420 con 0x55962cfceb90 2024-06-13T07:06:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.839+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faa8c00f810 con 0x55962cfceb90 2024-06-13T07:06:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.839+0000 7faaaf6bbb80 10 client.26855 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.839+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55962cfe0590 con 0x55962cfceb90 2024-06-13T07:06:04.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.841+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa98007420 con 0x55962cfceb90 2024-06-13T07:06:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.841+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7faa8c005510 con 0x55962cfceb90 2024-06-13T07:06:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.841+0000 7faaaf6bbb80 10 client.26855 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.841+0000 7faaaf6bbb80 10 client.26855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.842+0000 7faaaf6bbb80 20 client.26855 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.842+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962cfe2b20 0x55962cfe4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.842+0000 7faaaf6bbb80 4 client.26855 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.842+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55962cf28000 con 0x55962cfe2b20 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.842+0000 7faa9e800700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962cfe2b20 0x55962cfe4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.844+0000 7faa9e800700 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962cfe2b20 0x55962cfe4f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13c1600baa0bfbbd server_cookie=117bdb81ef214d2a in_seq=0 out_seq=0 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.844+0000 7faa7fe00700 10 client.26855 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.845+0000 7faa7fe00700 1 -- 172.21.15.7:0/3511365121 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55962cf28000 con 0x55962cfe2b20 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.845+0000 7faa7fe00700 10 client.26855 handle_command_reply: tid=0 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.845+0000 7faaaf6bbb80 1 client.26855 shutdown 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 msgr2=0x55962cfd2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfd2510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005fd0 tx=0x7faa8c011ae0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 shutdown_connections 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962cfe2b20 0x55962cfe4f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfd2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfd5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/3511365121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfd2510 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faa7fe00700 0 client.26855 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 >> 172.21.15.7:0/3511365121 conn(0x55962cef5170 msgr2=0x55962cfbd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 shutdown_connections 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/3511365121 wait complete. 2024-06-13T07:06:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.846+0000 7faaaf6bbb80 20 client.26855 trim_cache size 0 max 16384 2024-06-13T07:06:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.883+0000 7faaaf6bbb80 1 Processor -- start 2024-06-13T07:06:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.883+0000 7faaaf6bbb80 1 -- start start 2024-06-13T07:06:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfbe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfc4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b7180 con 0x55962cfc9240 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b4a10 con 0x55962cfceb90 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faaaf6bbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b2860 con 0x55962cfcf4b0 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faa9e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faa9e800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37208/0 (socket says 172.21.15.7:37208) 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faa9e800700 1 -- 172.21.15.7:0/2761552924 learned_addr learned my addr 172.21.15.7:0/2761552924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faa9e800700 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.884+0000 7faa9d400700 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfbe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.885+0000 7faa9e800700 1 -- 172.21.15.7:0/2761552924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 msgr2=0x55962cfc4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.885+0000 7faa9e800700 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfc4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.885+0000 7faa9e800700 1 -- 172.21.15.7:0/2761552924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 msgr2=0x55962cfbe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.885+0000 7faa9e800700 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfbe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.885+0000 7faa9e800700 1 -- 172.21.15.7:0/2761552924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962cfd0a80 con 0x55962cfc9240 2024-06-13T07:06:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.886+0000 7faa9e800700 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa98009c50 tx=0x7faa98007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d12b4d2da2a81a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.886+0000 7faa8aa00700 1 -- 172.21.15.7:0/2761552924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98003a00 con 0x55962cfc9240 2024-06-13T07:06:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.886+0000 7faa8aa00700 1 -- 172.21.15.7:0/2761552924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980078c0 con 0x55962cfc9240 2024-06-13T07:06:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.886+0000 7faa8aa00700 1 -- 172.21.15.7:0/2761552924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98003a00 con 0x55962cfc9240 2024-06-13T07:06:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.887+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/2761552924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 msgr2=0x55962cfc8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.887+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faa98009c50 tx=0x7faa98007df0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/2761552924 shutdown_connections 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfcf4b0 0x55962cfc4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfc9240 0x55962cfc8d00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/2761552924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfceb90 0x55962cfbe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/2761552924 >> 172.21.15.7:0/2761552924 conn(0x55962cef5170 msgr2=0x55962cfcd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.888+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/2761552924 shutdown_connections 2024-06-13T07:06:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.890+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/2761552924 wait complete. 2024-06-13T07:06:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.891+0000 7faaaf6bbb80 1 -- start start 2024-06-13T07:06:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.892+0000 7faaaf6bbb80 1 Processor -- start 2024-06-13T07:06:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.893+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.893+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfceb90 0x55962cfdb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faaaf6bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfe9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faa9e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faa9e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49124/0 (socket says 172.21.15.7:49124) 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faa9e800700 1 -- 172.21.15.7:0/829885325 learned_addr learned my addr 172.21.15.7:0/829885325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faa9d400700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfceb90 0x55962cfdb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b2ed0 con 0x55962cfcf4b0 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b4340 con 0x55962cfc9240 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01b35a0 con 0x55962cfceb90 2024-06-13T07:06:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faaaf6bbb80 10 client.0 _ll_register_callbacks cb 0x55962ceb0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.894+0000 7faa9e800700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfe9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 -- 172.21.15.7:0/829885325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfceb90 msgr2=0x55962cfdb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfceb90 0x55962cfdb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 -- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 msgr2=0x55962cfe9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfe9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962cfea770 con 0x55962cfc9240 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa9e800700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa9803a040 tx=0x7faa98003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de571bcf586cad09 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.895+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98046030 con 0x55962cfc9240 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa980078c0 con 0x55962cfc9240 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7faa9800f070 con 0x55962cfc9240 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7faa980074a0 con 0x55962cfc9240 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faaaf6bbb80 10 client.26861 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55962d04d340 con 0x55962cfc9240 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.896+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980078c0 con 0x55962cfc9240 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7faa98011070 con 0x55962cfc9240 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 10 client.26861 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 10 client.26861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 20 client.26861 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962d04d570 0x55962d04f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 4 client.26861 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:06:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.897+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55962cf22580 con 0x55962d04d570 2024-06-13T07:06:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.898+0000 7faa9d400700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962d04d570 0x55962d04f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:04.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.899+0000 7faa7fe00700 10 client.26861 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:06:04.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.899+0000 7faa9d400700 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962d04d570 0x55962d04f960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=709e2934fccfc415 server_cookie=f0f83ae92dea0ba9 in_seq=0 out_seq=0 2024-06-13T07:06:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.900+0000 7faa7fe00700 1 -- 172.21.15.7:0/829885325 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55962cf22580 con 0x55962d04d570 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.900+0000 7faa7fe00700 10 client.26861 handle_command_reply: tid=0 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.900+0000 7faaaf6bbb80 1 client.26861 shutdown 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 msgr2=0x55962cfdb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa9803a040 tx=0x7faa98003e70 comp rx=0 tx=0).stop 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 shutdown_connections 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55962cfcf4b0 0x55962cfe9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55962cfc9240 0x55962cfdb4b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55962d04d570 0x55962d04f960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 --2- 172.21.15.7:0/829885325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55962cfceb90 0x55962cfdb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 >> 172.21.15.7:0/829885325 conn(0x55962cef5170 msgr2=0x55962cfcffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faa7fe00700 0 client.26861 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:06:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 shutdown_connections 2024-06-13T07:06:04.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 1 -- 172.21.15.7:0/829885325 wait complete. 2024-06-13T07:06:04.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.901+0000 7faaaf6bbb80 20 client.26861 trim_cache size 0 max 16384 2024-06-13T07:06:04.905 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:06:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.903+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 msgr2=0x7faa840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.903+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 0x7faa840a51b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faa9000b2d0 tx=0x7faa90005c00 comp rx=0 tx=0).stop 2024-06-13T07:06:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.904+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 msgr2=0x7faaa010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.904+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa9000d980 tx=0x7faa9000dd40 comp rx=0 tx=0).stop 2024-06-13T07:06:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 shutdown_connections 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa01155e0 0x7faaa010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaa0109490 0x7faaa010af00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7faa840a2cf0 0x7faa840a51b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 --2- 172.21.15.7:0/4262641704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaa0109db0 0x7faaa010b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 >> 172.21.15.7:0/4262641704 conn(0x7faaa006d600 msgr2=0x7faaa0071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 shutdown_connections 2024-06-13T07:06:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:04.909+0000 7faaa4e00700 1 -- 172.21.15.7:0/4262641704 wait complete. 2024-06-13T07:06:05.834 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:05 smithi007 ceph-mon[32077]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-06-13T07:06:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:05 smithi189 ceph-mon[36616]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-06-13T07:06:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:05 smithi175 ceph-mon[36886]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.3 MiB/s wr, 497 op/s 2024-06-13T07:06:07.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:06:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:06:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:07 smithi189 ceph-mon[36616]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-06-13T07:06:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:07 smithi175 ceph-mon[36886]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-06-13T07:06:07.883 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 605.0 s, 470.2 tps, lat 215.380 ms stddev 252.362 2024-06-13T07:06:07.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:07 smithi007 ceph-mon[32077]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 471 op/s 2024-06-13T07:06:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:09 smithi189 ceph-mon[36616]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 428 op/s 2024-06-13T07:06:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:09 smithi175 ceph-mon[36886]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 428 op/s 2024-06-13T07:06:09.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:09 smithi007 ceph-mon[32077]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 MiB/s wr, 428 op/s 2024-06-13T07:06:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 -- 172.21.15.7:0/1544587705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c109300 msgr2=0x7fe64c10fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 --2- 172.21.15.7:0/1544587705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c109300 0x7fe64c10fce0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe63800fb60 tx=0x7fe63802e4e0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 -- 172.21.15.7:0/1544587705 shutdown_connections 2024-06-13T07:06:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 --2- 172.21.15.7:0/1544587705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c110220 0x7fe64c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 --2- 172.21.15.7:0/1544587705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c109300 0x7fe64c10fce0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 --2- 172.21.15.7:0/1544587705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c1089a0 0x7fe64c108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 -- 172.21.15.7:0/1544587705 >> 172.21.15.7:0/1544587705 conn(0x7fe64c06ebd0 msgr2=0x7fe64c071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.036+0000 7fe651800640 1 -- 172.21.15.7:0/1544587705 shutdown_connections 2024-06-13T07:06:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.037+0000 7fe651800640 1 -- 172.21.15.7:0/1544587705 wait complete. 2024-06-13T07:06:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.038+0000 7fe651800640 1 Processor -- start 2024-06-13T07:06:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 -- start start 2024-06-13T07:06:11.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 0x7fe64c1a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c109300 0x7fe64c1a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c113cd0 con 0x7fe64c1089a0 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c113b50 con 0x7fe64c109300 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.039+0000 7fe651800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c113e50 con 0x7fe64c110220 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45530/0 (socket says 172.21.15.7:45530) 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 -- 172.21.15.7:0/2112697193 learned_addr learned my addr 172.21.15.7:0/2112697193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64aa00640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 0x7fe64c1a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64a000640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c109300 0x7fe64c1a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 -- 172.21.15.7:0/2112697193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c109300 msgr2=0x7fe64c1a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c109300 0x7fe64c1a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.040+0000 7fe64b400640 1 -- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 msgr2=0x7fe64c1a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.041+0000 7fe64b400640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 0x7fe64c1a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.041+0000 7fe64b400640 1 -- 172.21.15.7:0/2112697193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c1ae650 con 0x7fe64c110220 2024-06-13T07:06:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.041+0000 7fe64aa00640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 0x7fe64c1a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.041+0000 7fe64b400640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe64400eda0 tx=0x7fe64400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.041+0000 7fe643400640 1 -- 172.21.15.7:0/2112697193 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003b70 con 0x7fe64c110220 2024-06-13T07:06:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.042+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1ae940 con 0x7fe64c110220 2024-06-13T07:06:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.042+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1aee60 con 0x7fe64c110220 2024-06-13T07:06:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.042+0000 7fe643400640 1 -- 172.21.15.7:0/2112697193 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644003d10 con 0x7fe64c110220 2024-06-13T07:06:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.042+0000 7fe643400640 1 -- 172.21.15.7:0/2112697193 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003b70 con 0x7fe64c110220 2024-06-13T07:06:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.047+0000 7fe643400640 1 -- 172.21.15.7:0/2112697193 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe644003fc0 con 0x7fe64c110220 2024-06-13T07:06:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.050+0000 7fe654569740 1 Processor -- start 2024-06-13T07:06:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.051+0000 7fe643400640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 0x7fe6300a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.052+0000 7fe643400640 1 -- 172.21.15.7:0/2112697193 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe6440cf160 con 0x7fe64c110220 2024-06-13T07:06:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.052+0000 7fe64a000640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 0x7fe6300a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.052+0000 7fe654569740 1 -- start start 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 0x55715e7b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55715e712420 con 0x55715e7aec00 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55715e6b9c50 con 0x55715e7af5f0 2024-06-13T07:06:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe654569740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55715e70e3e0 con 0x55715e7b4760 2024-06-13T07:06:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe64b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe64b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45538/0 (socket says 172.21.15.7:45538) 2024-06-13T07:06:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.053+0000 7fe64b400640 1 -- 172.21.15.7:0/3636439023 learned_addr learned my addr 172.21.15.7:0/3636439023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.054+0000 7fe64a000640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 0x55715e7b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.054+0000 7fe64b400640 1 -- 172.21.15.7:0/3636439023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 msgr2=0x55715e7b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.055+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64b400640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 0x55715e7b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64b400640 1 -- 172.21.15.7:0/3636439023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 msgr2=0x55715e7af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64b400640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64b400640 1 -- 172.21.15.7:0/3636439023 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715e426210 con 0x55715e7b4760 2024-06-13T07:06:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.056+0000 7fe64a000640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 0x55715e7b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.057+0000 7fe64a000640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 0x7fe6300a5450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe63800fb30 tx=0x7fe638034c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.058+0000 7fe64b400640 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a3d60 tx=0x7fe644097220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95767164d045db0d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.059+0000 7fe637e00640 1 -- 172.21.15.7:0/3636439023 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64409b620 con 0x55715e7b4760 2024-06-13T07:06:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.059+0000 7fe637e00640 1 -- 172.21.15.7:0/3636439023 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64409f9d0 con 0x55715e7b4760 2024-06-13T07:06:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.059+0000 7fe637e00640 1 -- 172.21.15.7:0/3636439023 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440a4060 con 0x55715e7b4760 2024-06-13T07:06:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.060+0000 7fe654569740 1 -- 172.21.15.7:0/3636439023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 msgr2=0x55715e7b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.060+0000 7fe654569740 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a3d60 tx=0x7fe644097220 comp rx=0 tx=0).stop 2024-06-13T07:06:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 -- 172.21.15.7:0/3636439023 shutdown_connections 2024-06-13T07:06:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7b4760 0x55715e7b6bf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7af5f0 0x55715e7b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 --2- 172.21.15.7:0/3636439023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 -- 172.21.15.7:0/3636439023 >> 172.21.15.7:0/3636439023 conn(0x55715e776fb0 msgr2=0x55715e7aa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.063+0000 7fe654569740 1 -- 172.21.15.7:0/3636439023 shutdown_connections 2024-06-13T07:06:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.064+0000 7fe654569740 1 -- 172.21.15.7:0/3636439023 wait complete. 2024-06-13T07:06:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.064+0000 7fe654569740 1 -- start start 2024-06-13T07:06:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.065+0000 7fe654569740 1 Processor -- start 2024-06-13T07:06:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.065+0000 7fe654569740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.065+0000 7fe654569740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 0x55715e7b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.065+0000 7fe654569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.066+0000 7fe64a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 0x55715e7b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.066+0000 7fe64a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 0x55715e7b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45544/0 (socket says 172.21.15.7:45544) 2024-06-13T07:06:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.066+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe64a000640 1 -- 172.21.15.7:0/947207973 learned_addr learned my addr 172.21.15.7:0/947207973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe64b400640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55715e7b76b0 con 0x55715e7aec00 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55715e6cd0b0 con 0x55715e7b4760 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55715e7b7830 con 0x55715e7af5f0 2024-06-13T07:06:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.067+0000 7fe654569740 10 client.0 _ll_register_callbacks cb 0x55715e6b8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.068+0000 7fe64b400640 1 -- 172.21.15.7:0/947207973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 msgr2=0x55715e7b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.068+0000 7fe64b400640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 0x55715e7b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.068+0000 7fe64b400640 1 -- 172.21.15.7:0/947207973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 msgr2=0x55715e7b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.069+0000 7fe64b400640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.069+0000 7fe64b400640 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715e4f2620 con 0x55715e7b4760 2024-06-13T07:06:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.069+0000 7fe64aa00640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.070+0000 7fe64b400640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7c03c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe64400cbe0 tx=0x7fe6440a4290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148b60a4c2beb56c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.070+0000 7fe634c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.071+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b3070 con 0x55715e7b4760 2024-06-13T07:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.071+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe644084dc0 con 0x55715e7b4760 2024-06-13T07:06:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.071+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64409f9d0 con 0x55715e7b4760 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.071+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe64409b620 con 0x55715e7b4760 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.072+0000 7fe654569740 10 client.26870 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.072+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55715e7c36a0 con 0x55715e7b4760 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64409f9d0 con 0x55715e7b4760 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fe64400b030 con 0x55715e7b4760 2024-06-13T07:06:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 10 client.26870 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 20 client.26870 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e7c49f0 0x55715e7c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 4 client.26870 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.073+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55715e71fb50 con 0x55715e7c49f0 2024-06-13T07:06:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.074+0000 7fe64aa00640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e7c49f0 0x55715e7c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.075+0000 7fe64aa00640 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e7c49f0 0x55715e7c6de0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cbac1de3f6898a server_cookie=41921c00389161db in_seq=0 out_seq=0 2024-06-13T07:06:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.075+0000 7fe634c00640 10 client.26870 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.076+0000 7fe634c00640 1 -- 172.21.15.7:0/947207973 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55715e71fb50 con 0x55715e7c49f0 2024-06-13T07:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.076+0000 7fe634c00640 10 client.26870 handle_command_reply: tid=0 2024-06-13T07:06:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.076+0000 7fe654569740 1 client.26870 shutdown 2024-06-13T07:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 msgr2=0x55715e7c03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7c03c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe64400cbe0 tx=0x7fe6440a4290 comp rx=0 tx=0).stop 2024-06-13T07:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 shutdown_connections 2024-06-13T07:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e7c49f0 0x55715e7c6de0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7c03c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7af5f0 0x55715e7b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 --2- 172.21.15.7:0/947207973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7aec00 0x55715e7b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 >> 172.21.15.7:0/947207973 conn(0x55715e776fb0 msgr2=0x55715e7a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.077+0000 7fe634c00640 0 client.26870 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.078+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 shutdown_connections 2024-06-13T07:06:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.078+0000 7fe654569740 1 -- 172.21.15.7:0/947207973 wait complete. 2024-06-13T07:06:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.079+0000 7fe654569740 20 client.26870 trim_cache size 0 max 16384 2024-06-13T07:06:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.108+0000 7fe654569740 1 Processor -- start 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.109+0000 7fe654569740 1 -- start start 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.109+0000 7fe654569740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.109+0000 7fe654569740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.109+0000 7fe654569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.109+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe654569740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0122c0 con 0x55715e7af5f0 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45550/0 (socket says 172.21.15.7:45550) 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64aa00640 1 -- 172.21.15.7:0/623391962 learned_addr learned my addr 172.21.15.7:0/623391962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00fb90 con 0x55715e7b4760 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00d9e0 con 0x55715e7aec00 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64a000640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 -- 172.21.15.7:0/623391962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 msgr2=0x55715e7c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 -- 172.21.15.7:0/623391962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 msgr2=0x55715e7c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64b400640 1 -- 172.21.15.7:0/623391962 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715e7af9d0 con 0x55715e7b4760 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64aa00640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.110+0000 7fe64a000640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.111+0000 7fe64b400640 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7aa140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a3d60 tx=0x7fe644086130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f910713568c6ec3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.111+0000 7fe637400640 1 -- 172.21.15.7:0/623391962 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b3030 con 0x55715e7b4760 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.111+0000 7fe637400640 1 -- 172.21.15.7:0/623391962 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644097d00 con 0x55715e7b4760 2024-06-13T07:06:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.111+0000 7fe637400640 1 -- 172.21.15.7:0/623391962 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64409aa70 con 0x55715e7b4760 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 msgr2=0x55715e7aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7aa140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe6440a3d60 tx=0x7fe644086130 comp rx=0 tx=0).stop 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 shutdown_connections 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e7aa140 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 --2- 172.21.15.7:0/623391962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 >> 172.21.15.7:0/623391962 conn(0x55715e776fb0 msgr2=0x55715e7b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 shutdown_connections 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- 172.21.15.7:0/623391962 wait complete. 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.112+0000 7fe654569740 1 -- start start 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.113+0000 7fe654569740 1 Processor -- start 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.114+0000 7fe654569740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.114+0000 7fe654569740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.114+0000 7fe654569740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.115+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.115+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45558/0 (socket says 172.21.15.7:45558) 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe64b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe64b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54224/0 (socket says 172.21.15.7:54224) 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe654569740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00e050 con 0x55715e7af5f0 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe654569740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00f4c0 con 0x55715e7b4760 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe654569740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00e720 con 0x55715e7aec00 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.116+0000 7fe654569740 10 client.0 _ll_register_callbacks cb 0x7fe64409b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.117+0000 7fe64aa00640 1 -- 172.21.15.7:0/1647489282 learned_addr learned my addr 172.21.15.7:0/1647489282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.117+0000 7fe64a000640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.118+0000 7fe64aa00640 1 -- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 msgr2=0x55715e82d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64aa00640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64aa00640 1 -- 172.21.15.7:0/1647489282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 msgr2=0x55715e7bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64aa00640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7bd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64aa00640 1 -- 172.21.15.7:0/1647489282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715e7c0570 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64b400640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.119+0000 7fe64a000640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7bd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.120+0000 7fe64aa00640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55715e7afe50 tx=0x7fe63c012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c663484026c1f88 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.120+0000 7fe634c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.120+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004220 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.120+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe63c0043c0 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.121+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe644097d00 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.121+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe63c01b510 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.121+0000 7fe654569740 10 client.26809 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.121+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55715e82eb00 con 0x55715e7aec00 2024-06-13T07:06:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644097d00 con 0x55715e7aec00 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fe63c003ae0 con 0x55715e7aec00 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 10 client.26809 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 10 client.26809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 20 client.26809 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e82f530 0x55715e831950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 4 client.26809 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.122+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55715e719ea0 con 0x55715e82f530 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.123+0000 7fe64a000640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e82f530 0x55715e831950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.123+0000 7fe64a000640 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e82f530 0x55715e831950 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64de76e37f2677f9 server_cookie=f6a4c21c1af83f39 in_seq=0 out_seq=0 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.123+0000 7fe634c00640 10 client.26809 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe634c00640 1 -- 172.21.15.7:0/1647489282 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55715e719ea0 con 0x55715e82f530 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe634c00640 10 client.26809 handle_command_reply: tid=0 2024-06-13T07:06:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 client.26809 shutdown 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 msgr2=0x55715e7bcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55715e7afe50 tx=0x7fe63c012820 comp rx=0 tx=0).stop 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 shutdown_connections 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55715e82f530 0x55715e831950 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55715e7b4760 0x55715e82d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55715e7af5f0 0x55715e7bd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 --2- 172.21.15.7:0/1647489282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55715e7aec00 0x55715e7bcfd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 >> 172.21.15.7:0/1647489282 conn(0x55715e776fb0 msgr2=0x55715e7aa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.135+0000 7fe634c00640 0 client.26809 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.136+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 shutdown_connections 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.136+0000 7fe654569740 1 -- 172.21.15.7:0/1647489282 wait complete. 2024-06-13T07:06:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.136+0000 7fe654569740 20 client.26809 trim_cache size 0 max 16384 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 msgr2=0x7fe6300a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 0x7fe6300a5450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe63800fb30 tx=0x7fe638034c00 comp rx=0 tx=0).stop 2024-06-13T07:06:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 msgr2=0x7fe64c1adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe64400eda0 tx=0x7fe64400c6e0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 shutdown_connections 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe6300a2f90 0x7fe6300a5450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c110220 0x7fe64c1adfe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe64c109300 0x7fe64c1a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 --2- 172.21.15.7:0/2112697193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c1089a0 0x7fe64c1a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.137+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 >> 172.21.15.7:0/2112697193 conn(0x7fe64c06ebd0 msgr2=0x7fe64c10bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.139+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 shutdown_connections 2024-06-13T07:06:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:11.139+0000 7fe651800640 1 -- 172.21.15.7:0/2112697193 wait complete. 2024-06-13T07:06:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:11 smithi175 ceph-mon[36886]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 488 op/s 2024-06-13T07:06:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:11 smithi007 ceph-mon[32077]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 488 op/s 2024-06-13T07:06:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:11 smithi189 ceph-mon[36616]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 488 op/s 2024-06-13T07:06:12.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.186+0000 7f67f2600700 1 -- 172.21.15.7:0/1799193456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 msgr2=0x7f67ec10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.186+0000 7f67f2600700 1 --2- 172.21.15.7:0/1799193456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec10f3c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67e400b0a0 tx=0x7f67e400b3b0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.188+0000 7f67f2600700 1 -- 172.21.15.7:0/1799193456 shutdown_connections 2024-06-13T07:06:12.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.188+0000 7f67f2600700 1 --2- 172.21.15.7:0/1799193456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec10f3c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.188+0000 7f67f2600700 1 --2- 172.21.15.7:0/1799193456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.188+0000 7f67f2600700 1 --2- 172.21.15.7:0/1799193456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.188+0000 7f67f2600700 1 -- 172.21.15.7:0/1799193456 >> 172.21.15.7:0/1799193456 conn(0x7f67ec0795f0 msgr2=0x7f67ec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.189+0000 7f67f2600700 1 -- 172.21.15.7:0/1799193456 shutdown_connections 2024-06-13T07:06:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.189+0000 7f67f2600700 1 -- 172.21.15.7:0/1799193456 wait complete. 2024-06-13T07:06:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.195+0000 7f67f2600700 1 Processor -- start 2024-06-13T07:06:12.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.195+0000 7f67f2600700 1 -- start start 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1127d0 con 0x7f67ec1030f0 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec112650 con 0x7f67ec102740 2024-06-13T07:06:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.196+0000 7f67f2600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec112950 con 0x7f67ec107840 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67ebe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67ebe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45582/0 (socket says 172.21.15.7:45582) 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54244/0 (socket says 172.21.15.7:54244) 2024-06-13T07:06:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.197+0000 7f67eb400700 1 -- 172.21.15.7:0/3174933042 learned_addr learned my addr 172.21.15.7:0/3174933042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.198+0000 7f67eaa00700 1 -- 172.21.15.7:0/3174933042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 msgr2=0x7f67ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.198+0000 7f67eaa00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.198+0000 7f67eaa00700 1 -- 172.21.15.7:0/3174933042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 msgr2=0x7f67ec19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.198+0000 7f67eaa00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.198+0000 7f67eaa00700 1 -- 172.21.15.7:0/3174933042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec198f00 con 0x7f67ec1030f0 2024-06-13T07:06:12.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.200+0000 7f67eaa00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec19ed10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d750 tx=0x7f67dc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.200+0000 7f67e3e00700 1 -- 172.21.15.7:0/3174933042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc017070 con 0x7f67ec1030f0 2024-06-13T07:06:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.200+0000 7f67e3e00700 1 -- 172.21.15.7:0/3174933042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc005020 con 0x7f67ec1030f0 2024-06-13T07:06:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.200+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec1991f0 con 0x7f67ec1030f0 2024-06-13T07:06:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.201+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec199730 con 0x7f67ec1030f0 2024-06-13T07:06:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.201+0000 7f67e3e00700 1 -- 172.21.15.7:0/3174933042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00fb30 con 0x7f67ec1030f0 2024-06-13T07:06:12.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.213+0000 7f67fd051b80 1 Processor -- start 2024-06-13T07:06:12.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.218+0000 7f67fd051b80 1 -- start start 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 0x56170648f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x56170648fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 0x56170649c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56170639ea10 con 0x56170640be40 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617063e1b00 con 0x56170648f5b0 2024-06-13T07:06:12.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67fd051b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56170639c8f0 con 0x56170649a300 2024-06-13T07:06:12.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.222+0000 7f67e3e00700 1 -- 172.21.15.7:0/3174933042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f67dc00fcd0 con 0x7f67ec1030f0 2024-06-13T07:06:12.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.223+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x56170648fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.223+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 0x56170649c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.223+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x56170648fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54246/0 (socket says 172.21.15.7:54246) 2024-06-13T07:06:12.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.223+0000 7f67eb400700 1 -- 172.21.15.7:0/1887900775 learned_addr learned my addr 172.21.15.7:0/1887900775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:12.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.223+0000 7f67ebe00700 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 0x56170648f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.224+0000 7f67eaa00700 1 -- 172.21.15.7:0/1887900775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 msgr2=0x56170648fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.224+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x56170648fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.224+0000 7f67eaa00700 1 -- 172.21.15.7:0/1887900775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 msgr2=0x56170648f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.224+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 0x56170648f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.224+0000 7f67eaa00700 1 -- 172.21.15.7:0/1887900775 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561706389f90 con 0x56170649a300 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.225+0000 7f67ebe00700 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 0x56170648f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.226+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 0x56170649c6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cc120 tx=0x7f67dc0d4ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8545c369395e93b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.226+0000 7f67e3e00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 0x7f67d00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.227+0000 7f67e0c00700 1 -- 172.21.15.7:0/1887900775 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0dd070 con 0x56170649a300 2024-06-13T07:06:12.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.227+0000 7f67e3e00700 1 -- 172.21.15.7:0/3174933042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f67dc0cea20 con 0x7f67ec1030f0 2024-06-13T07:06:12.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.228+0000 7f67e0c00700 1 -- 172.21.15.7:0/1887900775 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0cee50 con 0x56170649a300 2024-06-13T07:06:12.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.228+0000 7f67e0c00700 1 -- 172.21.15.7:0/1887900775 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc087150 con 0x56170649a300 2024-06-13T07:06:12.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.229+0000 7f67ebe00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 0x7f67d00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.230+0000 7f67fd051b80 1 -- 172.21.15.7:0/1887900775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 msgr2=0x56170649c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.230+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 0x56170649c6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cc120 tx=0x7f67dc0d4ed0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.231+0000 7f67fd051b80 1 -- 172.21.15.7:0/1887900775 shutdown_connections 2024-06-13T07:06:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.231+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170649a300 0x56170649c6f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.231+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170640be40 0x56170648f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.231+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1887900775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x56170648fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.231+0000 7f67fd051b80 1 -- 172.21.15.7:0/1887900775 >> 172.21.15.7:0/1887900775 conn(0x5617063bc170 msgr2=0x56170648b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.232+0000 7f67fd051b80 1 -- 172.21.15.7:0/1887900775 shutdown_connections 2024-06-13T07:06:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.233+0000 7f67fd051b80 1 -- 172.21.15.7:0/1887900775 wait complete. 2024-06-13T07:06:12.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.234+0000 7f67ebe00700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 0x7f67d00a51e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009850 tx=0x7f67e4009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.234+0000 7f67fd051b80 1 -- start start 2024-06-13T07:06:12.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.234+0000 7f67fd051b80 1 Processor -- start 2024-06-13T07:06:12.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.236+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x561706496c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x5617064971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x561706496c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45602/0 (socket says 172.21.15.7:45602) 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.237+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x561706496c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54262/0 (socket says 172.21.15.7:54262) 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67ebe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x5617064971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67fd051b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56170637df60 con 0x56170649a300 2024-06-13T07:06:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67eb400700 1 -- 172.21.15.7:0/1214788623 learned_addr learned my addr 172.21.15.7:0/1214788623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:12.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617063ddac0 con 0x56170648f5b0 2024-06-13T07:06:12.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56170649cfc0 con 0x56170640be40 2024-06-13T07:06:12.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.238+0000 7f67fd051b80 10 client.0 _ll_register_callbacks cb 0x56170637b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.239+0000 7f67eb400700 1 -- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 msgr2=0x561706496c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.239+0000 7f67eb400700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x561706496c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.239+0000 7f67eb400700 1 -- 172.21.15.7:0/1214788623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 msgr2=0x5617064971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.240+0000 7f67eb400700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x5617064971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.240+0000 7f67eb400700 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617063308d0 con 0x56170640be40 2024-06-13T07:06:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.240+0000 7f67ebe00700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x5617064971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.240+0000 7f67eb400700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f67d4003e40 tx=0x7f67d40116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc72b133a9d7a386 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.240+0000 7f67d9600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:06:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.241+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d401b670 con 0x56170640be40 2024-06-13T07:06:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.241+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f67d401b810 con 0x56170640be40 2024-06-13T07:06:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.241+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67dc0cee50 con 0x56170640be40 2024-06-13T07:06:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.241+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f67d401ba50 con 0x56170640be40 2024-06-13T07:06:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.242+0000 7f67fd051b80 10 client.26821 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.242+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5617064a7660 con 0x56170640be40 2024-06-13T07:06:12.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0cee50 con 0x56170640be40 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f67d4025420 con 0x56170640be40 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 10 client.26821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 20 client.26821 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5617064a9bf0 0x5617064ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 4 client.26821 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.243+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617063ef000 con 0x5617064a9bf0 2024-06-13T07:06:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.248+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5617064a9bf0 0x5617064ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.249+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5617064a9bf0 0x5617064ac050 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936164b970b46cd7 server_cookie=14670e9ca244a551 in_seq=0 out_seq=0 2024-06-13T07:06:12.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.249+0000 7f67d9600700 10 client.26821 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:12.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.250+0000 7f67d9600700 1 -- 172.21.15.7:0/1214788623 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617063ef000 con 0x5617064a9bf0 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.250+0000 7f67d9600700 10 client.26821 handle_command_reply: tid=0 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 client.26821 shutdown 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 msgr2=0x561706499980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f67d4003e40 tx=0x7f67d40116c0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 shutdown_connections 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5617064a9bf0 0x5617064ac050 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x5617064971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x561706499980 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1214788623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x561706496c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 >> 172.21.15.7:0/1214788623 conn(0x5617063bc170 msgr2=0x56170648a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.251+0000 7f67d9600700 0 client.26821 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.252+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 shutdown_connections 2024-06-13T07:06:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.253+0000 7f67fd051b80 1 -- 172.21.15.7:0/1214788623 wait complete. 2024-06-13T07:06:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.253+0000 7f67fd051b80 20 client.26821 trim_cache size 0 max 16384 2024-06-13T07:06:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.288+0000 7f67fd051b80 1 Processor -- start 2024-06-13T07:06:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.288+0000 7f67fd051b80 1 -- start start 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.288+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170648f5b0 0x56170649c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x56170649c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67fd051b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8011250 con 0x56170649a300 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67fd051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67c800eae0 con 0x56170640be40 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67fd051b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67c800c930 con 0x56170648f5b0 2024-06-13T07:06:12.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67eb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54270/0 (socket says 172.21.15.7:54270) 2024-06-13T07:06:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67eb400700 1 -- 172.21.15.7:0/1270319257 learned_addr learned my addr 172.21.15.7:0/1270319257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.289+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170648f5b0 0x56170649c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 -- 172.21.15.7:0/1270319257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170648f5b0 msgr2=0x56170649c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170648f5b0 0x56170649c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 -- 172.21.15.7:0/1270319257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 msgr2=0x56170649c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x56170649c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 -- 172.21.15.7:0/1270319257 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617064ac6d0 con 0x56170640be40 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67eb400700 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f67d40257c0 tx=0x7f67d4003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951751fa07b304bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.290+0000 7f67dbe00700 1 -- 172.21.15.7:0/1270319257 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4025420 con 0x56170640be40 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.291+0000 7f67dbe00700 1 -- 172.21.15.7:0/1270319257 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4016070 con 0x56170640be40 2024-06-13T07:06:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.291+0000 7f67dbe00700 1 -- 172.21.15.7:0/1270319257 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40052f0 con 0x56170640be40 2024-06-13T07:06:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.292+0000 7f67fd051b80 1 -- 172.21.15.7:0/1270319257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 msgr2=0x56170649b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.292+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f67d40257c0 tx=0x7f67d4003680 comp rx=0 tx=0).stop 2024-06-13T07:06:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.297+0000 7f67fd051b80 1 -- 172.21.15.7:0/1270319257 shutdown_connections 2024-06-13T07:06:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.297+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x56170649c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.297+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170640be40 0x56170649b6f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.297+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1270319257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170648f5b0 0x56170649c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.297+0000 7f67fd051b80 1 -- 172.21.15.7:0/1270319257 >> 172.21.15.7:0/1270319257 conn(0x5617063bc170 msgr2=0x561706495570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.298+0000 7f67fd051b80 1 -- 172.21.15.7:0/1270319257 shutdown_connections 2024-06-13T07:06:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.298+0000 7f67fd051b80 1 -- 172.21.15.7:0/1270319257 wait complete. 2024-06-13T07:06:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.299+0000 7f67fd051b80 1 -- start start 2024-06-13T07:06:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.300+0000 7f67fd051b80 1 Processor -- start 2024-06-13T07:06:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.301+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.301+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x5617064a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.302+0000 7f67fd051b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x561706493550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.302+0000 7f67fd051b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67c800cfa0 con 0x56170649a300 2024-06-13T07:06:12.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.302+0000 7f67fd051b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67c800e410 con 0x56170648f5b0 2024-06-13T07:06:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.302+0000 7f67fd051b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67c800d670 con 0x56170640be40 2024-06-13T07:06:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.302+0000 7f67fd051b80 10 client.0 _ll_register_callbacks cb 0x56170637b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45616/0 (socket says 172.21.15.7:45616) 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 -- 172.21.15.7:0/1975657041 learned_addr learned my addr 172.21.15.7:0/1975657041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x5617064a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67ebe00700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x561706493550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 -- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 msgr2=0x5617064a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x5617064a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 -- 172.21.15.7:0/1975657041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 msgr2=0x561706493550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x561706493550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67eb400700 1 -- 172.21.15.7:0/1975657041 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561706493de0 con 0x56170640be40 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.303+0000 7f67ebe00700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x561706493550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.304+0000 7f67eb400700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67d401b730 tx=0x7f67d4004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ff35754a2202a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.304+0000 7f67d9600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.305+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4025bc0 con 0x56170640be40 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.305+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f67d4025d60 con 0x56170640be40 2024-06-13T07:06:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.305+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4016070 con 0x56170640be40 2024-06-13T07:06:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.306+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f67d403f050 con 0x56170640be40 2024-06-13T07:06:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.306+0000 7f67fd051b80 10 client.26824 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.306+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5617064a1e90 con 0x56170640be40 2024-06-13T07:06:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.306+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4016070 con 0x56170640be40 2024-06-13T07:06:12.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f67d4042020 con 0x56170640be40 2024-06-13T07:06:12.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 10 client.26824 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:12.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 20 client.26824 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561706514bc0 0x561706516fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 4 client.26824 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:06:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.309+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617063e9580 con 0x561706514bc0 2024-06-13T07:06:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.311+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561706514bc0 0x561706516fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.312+0000 7f67eaa00700 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561706514bc0 0x561706516fe0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb4e7fedd0654ba server_cookie=6ad270eb909edc64 in_seq=0 out_seq=0 2024-06-13T07:06:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.312+0000 7f67d9600700 10 client.26824 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67d9600700 1 -- 172.21.15.7:0/1975657041 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617063e9580 con 0x561706514bc0 2024-06-13T07:06:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67d9600700 10 client.26824 handle_command_reply: tid=0 2024-06-13T07:06:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 client.26824 shutdown 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 msgr2=0x5617064a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f67d401b730 tx=0x7f67d4004490 comp rx=0 tx=0).stop 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 shutdown_connections 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x561706514bc0 0x561706516fe0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67d9600700 0 client.26824 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56170649a300 0x561706493550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56170640be40 0x5617064a25b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 --2- 172.21.15.7:0/1975657041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56170648f5b0 0x5617064a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.313+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 >> 172.21.15.7:0/1975657041 conn(0x5617063bc170 msgr2=0x561706490fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.314+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 shutdown_connections 2024-06-13T07:06:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.314+0000 7f67fd051b80 1 -- 172.21.15.7:0/1975657041 wait complete. 2024-06-13T07:06:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.314+0000 7f67fd051b80 20 client.26824 trim_cache size 0 max 16384 2024-06-13T07:06:12.316 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:06:12.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.315+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 msgr2=0x7f67d00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.315+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 0x7f67d00a51e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009850 tx=0x7f67e4009780 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.315+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 msgr2=0x7f67ec19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.315+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec19ed10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d750 tx=0x7f67dc00db10 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 shutdown_connections 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ec107840 0x7f67ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67d00a2d20 0x7f67d00a51e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67ec102740 0x7f67ec19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 --2- 172.21.15.7:0/3174933042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67ec1030f0 0x7f67ec19ed10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 >> 172.21.15.7:0/3174933042 conn(0x7f67ec0795f0 msgr2=0x7f67ec1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 shutdown_connections 2024-06-13T07:06:12.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:12.316+0000 7f67f2600700 1 -- 172.21.15.7:0/3174933042 wait complete. 2024-06-13T07:06:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 610.0 s, 555.2 tps, lat 179.975 ms stddev 255.012 2024-06-13T07:06:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:13 smithi175 ceph-mon[36886]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 467 op/s 2024-06-13T07:06:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:13 smithi007 ceph-mon[32077]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 467 op/s 2024-06-13T07:06:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:13 smithi189 ceph-mon[36616]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.4 MiB/s wr, 467 op/s 2024-06-13T07:06:15.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:06:15.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:06:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:15 smithi175 ceph-mon[36886]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-06-13T07:06:15.887 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:15 smithi189 ceph-mon[36616]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-06-13T07:06:15.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:15 smithi007 ceph-mon[32077]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-06-13T07:06:16.059 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:06:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:17 smithi175 ceph-mon[36886]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 445 op/s 2024-06-13T07:06:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 615.0 s, 488.4 tps, lat 208.426 ms stddev 236.561 2024-06-13T07:06:17.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:17 smithi007 ceph-mon[32077]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 445 op/s 2024-06-13T07:06:18.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:17 smithi189 ceph-mon[36616]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 445 op/s 2024-06-13T07:06:19.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:19 smithi007 ceph-mon[32077]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-06-13T07:06:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:19 smithi189 ceph-mon[36616]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-06-13T07:06:20.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:19 smithi175 ceph-mon[36886]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-06-13T07:06:20.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:06:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:06:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7f39e750-7e75-41ac-9e71-ea792baf0372", 2024-06-13T07:06:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:06:20.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.568+0000 7fe5d5e00700 1 -- 172.21.15.7:0/188737692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0109490 msgr2=0x7fe5d0109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.568+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/188737692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0109490 0x7fe5d0109870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b0a0 tx=0x7fe5c800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.571+0000 7fe5d5e00700 1 -- 172.21.15.7:0/188737692 shutdown_connections 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.571+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/188737692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d01155e0 0x7fe5d01179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.571+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/188737692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d0109490 0x7fe5d0109870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.571+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/188737692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109db0 0x7fe5d01150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.571+0000 7fe5d5e00700 1 -- 172.21.15.7:0/188737692 >> 172.21.15.7:0/188737692 conn(0x7fe5d006d600 msgr2=0x7fe5d006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.572+0000 7fe5d5e00700 1 -- 172.21.15.7:0/188737692 shutdown_connections 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.572+0000 7fe5d5e00700 1 -- 172.21.15.7:0/188737692 wait complete. 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.573+0000 7fe5d5e00700 1 Processor -- start 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.574+0000 7fe5d5e00700 1 -- start start 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109490 0x7fe5d010af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 0x7fe5d010b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d011b050 con 0x7fe5d0109490 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d011aed0 con 0x7fe5d01155e0 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.575+0000 7fe5d5e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d011b1d0 con 0x7fe5d0109db0 2024-06-13T07:06:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.576+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 0x7fe5d010b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 0x7fe5d010b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45646/0 (socket says 172.21.15.7:45646) 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54290/0 (socket says 172.21.15.7:54290) 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cf400700 1 -- 172.21.15.7:0/2186038984 learned_addr learned my addr 172.21.15.7:0/2186038984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cea00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109490 0x7fe5d010af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cf400700 1 -- 172.21.15.7:0/2186038984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 msgr2=0x7fe5d010b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.577+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 0x7fe5d010b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.578+0000 7fe5cf400700 1 -- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109490 msgr2=0x7fe5d010af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.578+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109490 0x7fe5d010af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.578+0000 7fe5cf400700 1 -- 172.21.15.7:0/2186038984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d01b3c30 con 0x7fe5d01155e0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.578+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d980 tx=0x7fe5b800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.584+0000 7fe5d4a00700 1 -- 172.21.15.7:0/2186038984 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8017070 con 0x7fe5d01155e0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.584+0000 7fe5d4a00700 1 -- 172.21.15.7:0/2186038984 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8009e20 con 0x7fe5d01155e0 2024-06-13T07:06:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.584+0000 7fe5d4a00700 1 -- 172.21.15.7:0/2186038984 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8005440 con 0x7fe5d01155e0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.585+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d01b3ec0 con 0x7fe5d01155e0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.585+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d01b42c0 con 0x7fe5d01155e0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.601+0000 7fe5d4a00700 1 -- 172.21.15.7:0/2186038984 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe5b8004750 con 0x7fe5d01155e0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.604+0000 7fe5e077ab80 1 Processor -- start 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.605+0000 7fe5e077ab80 1 -- start start 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.607+0000 7fe5d4a00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 0x7fe5b40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.608+0000 7fe5d4a00700 1 -- 172.21.15.7:0/2186038984 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe5b80cf0f0 con 0x7fe5d01155e0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.608+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a0d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.608+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a0dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.608+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.609+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.609+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54304/0 (socket says 172.21.15.7:54304) 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.609+0000 7fe5cf400700 1 -- 172.21.15.7:0/2195540738 learned_addr learned my addr 172.21.15.7:0/2195540738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.610+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c81391ca90 con 0x55c813989ec0 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.610+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c81395fb80 con 0x55c813a18380 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.610+0000 7fe5ce000700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 0x7fe5b40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.610+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c81391a970 con 0x55c813a0d630 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.610+0000 7fe5ce000700 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a0dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.611+0000 7fe5cf400700 1 -- 172.21.15.7:0/2195540738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 msgr2=0x55c813a0dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.611+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a0dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.611+0000 7fe5cf400700 1 -- 172.21.15.7:0/2195540738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 msgr2=0x55c813a0d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.611+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a0d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.611+0000 7fe5cf400700 1 -- 172.21.15.7:0/2195540738 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c813908010 con 0x55c813a18380 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.612+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8097a70 tx=0x7fe5b809bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408abcd026ba4ced server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.612+0000 7fe5ce000700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 0x7fe5b40a5230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55c813a0a9a0 tx=0x7fe5c000a570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.613+0000 7fe5c4c00700 1 -- 172.21.15.7:0/2195540738 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80a4280 con 0x55c813a18380 2024-06-13T07:06:20.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.613+0000 7fe5c4c00700 1 -- 172.21.15.7:0/2195540738 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8097040 con 0x55c813a18380 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 msgr2=0x55c813a1a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8097a70 tx=0x7fe5b809bd10 comp rx=0 tx=0).stop 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 shutdown_connections 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a1a770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a0d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2195540738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a0dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 >> 172.21.15.7:0/2195540738 conn(0x55c81393a1f0 msgr2=0x55c813a09530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 shutdown_connections 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.614+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2195540738 wait complete. 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.615+0000 7fe5e077ab80 1 -- start start 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.615+0000 7fe5e077ab80 1 Processor -- start 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.616+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.616+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a14cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.616+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.617+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a14cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c8138fbfe0 con 0x55c813a0d630 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c81395bb40 con 0x55c813a18380 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c813a1b080 con 0x55c813989ec0 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a14cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34480/0 (socket says 172.21.15.7:34480) 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54310/0 (socket says 172.21.15.7:54310) 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 -- 172.21.15.7:0/2518478228 learned_addr learned my addr 172.21.15.7:0/2518478228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45662/0 (socket says 172.21.15.7:45662) 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5e077ab80 10 client.0 _ll_register_callbacks cb 0x55c8138f97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:20.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 -- 172.21.15.7:0/2518478228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 msgr2=0x55c813a179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 -- 172.21.15.7:0/2518478228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 msgr2=0x55c813a14cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a14cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.618+0000 7fe5cf400700 1 -- 172.21.15.7:0/2518478228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8138ae950 con 0x55c813a18380 2024-06-13T07:06:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.619+0000 7fe5cf400700 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009520 tx=0x7fe5b809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c262f80ed9e3a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.619+0000 7fe5bd600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.621+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80b8020 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.622+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe5b800edf0 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.622+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b8097040 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.622+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe5b80b3030 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.622+0000 7fe5e077ab80 10 client.26906 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.622+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c813a36720 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.623+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8097040 con 0x55c813a18380 2024-06-13T07:06:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b80a3df0 con 0x55c813a18380 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 10 client.26906 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 10 client.26906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 20 client.26906 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813a38bf0 0x55c813a3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 4 client.26906 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c81396d080 con 0x55c813a38bf0 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.626+0000 7fe5cea00700 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813a38bf0 0x55c813a3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.630+0000 7fe5cea00700 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813a38bf0 0x55c813a3afe0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4a870d4ea8acce server_cookie=7b1abf8ac5c145b3 in_seq=0 out_seq=0 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.630+0000 7fe5bd600700 10 client.26906 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.630+0000 7fe5bd600700 1 -- 172.21.15.7:0/2518478228 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c81396d080 con 0x55c813a38bf0 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.630+0000 7fe5bd600700 10 client.26906 handle_command_reply: tid=0 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.631+0000 7fe5e077ab80 1 client.26906 shutdown 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.631+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 msgr2=0x55c813a15230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.631+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009520 tx=0x7fe5b809be80 comp rx=0 tx=0).stop 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 shutdown_connections 2024-06-13T07:06:20.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813a38bf0 0x55c813a3afe0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a15230 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/2518478228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a14cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5bd600700 0 client.26906 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 >> 172.21.15.7:0/2518478228 conn(0x55c81393a1f0 msgr2=0x55c813a087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.632+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 shutdown_connections 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.633+0000 7fe5e077ab80 1 -- 172.21.15.7:0/2518478228 wait complete. 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.633+0000 7fe5e077ab80 20 client.26906 trim_cache size 0 max 16384 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.681+0000 7fe5e077ab80 1 Processor -- start 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.681+0000 7fe5e077ab80 1 -- start start 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a17530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01b6db0 con 0x55c813989ec0 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01b4640 con 0x55c813a18380 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01abbf0 con 0x55c813a0d630 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.682+0000 7fe5ce000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45672/0 (socket says 172.21.15.7:45672) 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 -- 172.21.15.7:0/758197305 learned_addr learned my addr 172.21.15.7:0/758197305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5cf400700 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a17530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5cea00700 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 -- 172.21.15.7:0/758197305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 msgr2=0x55c813a17530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a17530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 -- 172.21.15.7:0/758197305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 msgr2=0x55c813a096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.683+0000 7fe5ce000700 1 -- 172.21.15.7:0/758197305 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c813a0ec50 con 0x55c813a0d630 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.684+0000 7fe5ce000700 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000a480 tx=0x7fe5c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ebc528940b6db6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.684+0000 7fe5bfe00700 1 -- 172.21.15.7:0/758197305 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000cdd0 con 0x55c813a0d630 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.684+0000 7fe5bfe00700 1 -- 172.21.15.7:0/758197305 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c0029960 con 0x55c813a0d630 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.684+0000 7fe5bfe00700 1 -- 172.21.15.7:0/758197305 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000cdd0 con 0x55c813a0d630 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.686+0000 7fe5e077ab80 1 -- 172.21.15.7:0/758197305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 msgr2=0x55c813a13cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.686+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000a480 tx=0x7fe5c00047c0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 -- 172.21.15.7:0/758197305 shutdown_connections 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813989ec0 0x55c813a096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/758197305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813a0d630 0x55c813a13cf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 -- 172.21.15.7:0/758197305 >> 172.21.15.7:0/758197305 conn(0x55c81393a1f0 msgr2=0x55c813a135e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.689+0000 7fe5e077ab80 1 -- 172.21.15.7:0/758197305 shutdown_connections 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.690+0000 7fe5e077ab80 1 -- 172.21.15.7:0/758197305 wait complete. 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.691+0000 7fe5e077ab80 1 -- start start 2024-06-13T07:06:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.691+0000 7fe5e077ab80 1 Processor -- start 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a3e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01ac1b0 con 0x55c813a0d630 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01ad620 con 0x55c813a18380 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d01ac880 con 0x55c813989ec0 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.692+0000 7fe5e077ab80 10 client.0 _ll_register_callbacks cb 0x55c8138f97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.693+0000 7fe5cea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.693+0000 7fe5cf400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.693+0000 7fe5cea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45678/0 (socket says 172.21.15.7:45678) 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.693+0000 7fe5cea00700 1 -- 172.21.15.7:0/855340480 learned_addr learned my addr 172.21.15.7:0/855340480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5ce000700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a3e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5cea00700 1 -- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 msgr2=0x55c813a3a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5cea00700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a3a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5cea00700 1 -- 172.21.15.7:0/855340480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 msgr2=0x55c813a3e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5cea00700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a3e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.694+0000 7fe5cea00700 1 -- 172.21.15.7:0/855340480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c813a3adb0 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.695+0000 7fe5cea00700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c80094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abfb67bd5cc4b2e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.695+0000 7fe5bd600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.696+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802b060 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.696+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c0029960 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.696+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fe5c80074e0 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.698+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe5c8032050 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.698+0000 7fe5e077ab80 10 client.26842 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.698+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c813aa40b0 con 0x55c813989ec0 2024-06-13T07:06:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c0029960 con 0x55c813989ec0 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fe5c8024070 con 0x55c813989ec0 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 10 client.26842 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 10 client.26842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 20 client.26842 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813aa4740 0x55c813aa6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 4 client.26842 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.699+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c813967600 con 0x55c813aa4740 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.700+0000 7fe5ce000700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813aa4740 0x55c813aa6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.700+0000 7fe5bd600700 10 client.26842 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.701+0000 7fe5ce000700 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813aa4740 0x55c813aa6b80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb4f887457e9670 server_cookie=e42ef2d1fbd762e6 in_seq=0 out_seq=0 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5bd600700 1 -- 172.21.15.7:0/855340480 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c813967600 con 0x55c813aa4740 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5bd600700 10 client.26842 handle_command_reply: tid=0 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5e077ab80 1 client.26842 shutdown 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 msgr2=0x55c813a3e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c80094d0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 shutdown_connections 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.702+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c813aa4740 0x55c813aa6b80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.703+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c813a18380 0x55c813a3a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.703+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c813989ec0 0x55c813a3e150 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.703+0000 7fe5e077ab80 1 --2- 172.21.15.7:0/855340480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c813a0d630 0x55c813a3e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.703+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 >> 172.21.15.7:0/855340480 conn(0x55c81393a1f0 msgr2=0x55c813a13e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.703+0000 7fe5bd600700 0 client.26842 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.706+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 shutdown_connections 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.706+0000 7fe5e077ab80 1 -- 172.21.15.7:0/855340480 wait complete. 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.706+0000 7fe5e077ab80 20 client.26842 trim_cache size 0 max 16384 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.708+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 msgr2=0x7fe5b40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.708+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 0x7fe5b40a5230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55c813a0a9a0 tx=0x7fe5c000a570 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.708+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 msgr2=0x7fe5d010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.708+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d980 tx=0x7fe5b800dd40 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 shutdown_connections 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fe5b40a2d70 0x7fe5b40a5230 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5d01155e0 0x7fe5d010b980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d0109490 0x7fe5d010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 --2- 172.21.15.7:0/2186038984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5d0109db0 0x7fe5d010b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 >> 172.21.15.7:0/2186038984 conn(0x7fe5d006d600 msgr2=0x7fe5d010f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 shutdown_connections 2024-06-13T07:06:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:20.709+0000 7fe5d5e00700 1 -- 172.21.15.7:0/2186038984 wait complete. 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:06:21.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:21 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-06-13T07:06:22.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:06:22.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:06:22.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:06:22.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:21 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:06:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:21 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 620.0 s, 525.8 tps, lat 189.148 ms stddev 209.100 2024-06-13T07:06:22.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:22 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:06:22.893 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:22 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.894 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:22 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:06:22.894 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:22 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.894 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:22 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:22.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:06:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:22 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:06:23.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:22 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:22 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:06:23.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:22 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:22 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:22 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:06:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:22 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:22 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:06:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:22 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:22 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:06:23.680 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:06:23.894 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:23 smithi007 ceph-mon[32077]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2024-06-13T07:06:24.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:23 smithi189 ceph-mon[36616]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2024-06-13T07:06:24.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:23 smithi175 ceph-mon[36886]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 -- 172.21.15.7:0/3308736524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 msgr2=0x7f594c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 --2- 172.21.15.7:0/3308736524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c104040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f593c009900 tx=0x7f593c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 -- 172.21.15.7:0/3308736524 shutdown_connections 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 --2- 172.21.15.7:0/3308736524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c1057d0 0x7f594c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 --2- 172.21.15.7:0/3308736524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c104e10 0x7f594c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 --2- 172.21.15.7:0/3308736524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c104040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.867+0000 7f5952e00640 1 -- 172.21.15.7:0/3308736524 >> 172.21.15.7:0/3308736524 conn(0x7f594c0ff190 msgr2=0x7f594c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.869+0000 7f5952e00640 1 -- 172.21.15.7:0/3308736524 shutdown_connections 2024-06-13T07:06:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.869+0000 7f5952e00640 1 -- 172.21.15.7:0/3308736524 wait complete. 2024-06-13T07:06:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.871+0000 7f5952e00640 1 Processor -- start 2024-06-13T07:06:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.874+0000 7f5952e00640 1 -- start start 2024-06-13T07:06:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.874+0000 7f5952e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.874+0000 7f5952e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 0x7f594c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.874+0000 7f5952e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f594be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5952e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10f8c0 con 0x7f594c1057d0 2024-06-13T07:06:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5952e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10f740 con 0x7f594c104e10 2024-06-13T07:06:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5952e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f594c10fa40 con 0x7f594c103c20 2024-06-13T07:06:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5950e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5950e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43446/0 (socket says 172.21.15.7:43446) 2024-06-13T07:06:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.875+0000 7f5950e00640 1 -- 172.21.15.7:0/2594655088 learned_addr learned my addr 172.21.15.7:0/2594655088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f5950e00640 1 -- 172.21.15.7:0/2594655088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 msgr2=0x7f594c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f594b400640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 0x7f594c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f5950e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f5950e00640 1 -- 172.21.15.7:0/2594655088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 msgr2=0x7f594c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f5950e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 0x7f594c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f5950e00640 1 -- 172.21.15.7:0/2594655088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594c1a1b10 con 0x7f594c1057d0 2024-06-13T07:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.876+0000 7f594be00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.877+0000 7f594b400640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 0x7f594c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.877+0000 7f5950e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f594400eaa0 tx=0x7f594400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.878+0000 7f5948c00640 1 -- 172.21.15.7:0/2594655088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594400cdb0 con 0x7f594c1057d0 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.878+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594c1a1e00 con 0x7f594c1057d0 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.878+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594c1a22c0 con 0x7f594c1057d0 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.879+0000 7f5948c00640 1 -- 172.21.15.7:0/2594655088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5944005020 con 0x7f594c1057d0 2024-06-13T07:06:24.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.879+0000 7f5948c00640 1 -- 172.21.15.7:0/2594655088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59440116d0 con 0x7f594c1057d0 2024-06-13T07:06:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.884+0000 7f5955b63740 1 Processor -- start 2024-06-13T07:06:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.887+0000 7f5948c00640 1 -- 172.21.15.7:0/2594655088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5944004a40 con 0x7f594c1057d0 2024-06-13T07:06:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 -- start start 2024-06-13T07:06:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf225e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf226b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abf21c1420 con 0x55abf225e5f0 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abf2168c50 con 0x55abf226bf70 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.888+0000 7f5955b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55abf21bd3e0 con 0x55abf225dc00 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.889+0000 7f594b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.889+0000 7f594b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55656/0 (socket says 172.21.15.7:55656) 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.889+0000 7f594b400640 1 -- 172.21.15.7:0/669654154 learned_addr learned my addr 172.21.15.7:0/669654154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.889+0000 7f594b400640 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf225e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 -- 172.21.15.7:0/669654154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 msgr2=0x55abf225e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf225e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 -- 172.21.15.7:0/669654154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 msgr2=0x55abf226b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf226b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 -- 172.21.15.7:0/669654154 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf1ed5210 con 0x55abf226bf70 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.890+0000 7f594b400640 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f59380127f0 tx=0x7f593800d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc426ae7e7eb73f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.891+0000 7f5941600640 1 -- 172.21.15.7:0/669654154 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593800aa60 con 0x55abf226bf70 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.891+0000 7f5941600640 1 -- 172.21.15.7:0/669654154 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5938012e30 con 0x55abf226bf70 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.891+0000 7f5941600640 1 -- 172.21.15.7:0/669654154 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380103d0 con 0x55abf226bf70 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5948c00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 0x7f59300a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 -- 172.21.15.7:0/669654154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 msgr2=0x55abf226e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f59380127f0 tx=0x7f593800d3c0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5948c00640 1 -- 172.21.15.7:0/2594655088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f59440cf590 con 0x7f594c1057d0 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 -- 172.21.15.7:0/669654154 shutdown_connections 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf226e400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf226b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 --2- 172.21.15.7:0/669654154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf225e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.892+0000 7f5955b63740 1 -- 172.21.15.7:0/669654154 >> 172.21.15.7:0/669654154 conn(0x55abf2225fb0 msgr2=0x55abf2259680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.893+0000 7f594be00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 0x7f59300a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.893+0000 7f5955b63740 1 -- 172.21.15.7:0/669654154 shutdown_connections 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.894+0000 7f5955b63740 1 -- 172.21.15.7:0/669654154 wait complete. 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.894+0000 7f5955b63740 1 -- start start 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.895+0000 7f5955b63740 1 Processor -- start 2024-06-13T07:06:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.895+0000 7f594be00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 0x7f59300a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f593c009850 tx=0x7f593c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 0x55abf2265150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 0x55abf2265690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf226bf70 0x55abf226a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55abf225f360 con 0x55abf225dc00 2024-06-13T07:06:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55abf217c0b0 con 0x55abf225e5f0 2024-06-13T07:06:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55abf225f4e0 con 0x55abf226bf70 2024-06-13T07:06:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.896+0000 7f5955b63740 10 client.0 _ll_register_callbacks cb 0x55abf2167f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.897+0000 7f594be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 0x55abf2265150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.897+0000 7f594be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 0x55abf2265150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43472/0 (socket says 172.21.15.7:43472) 2024-06-13T07:06:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.897+0000 7f594be00640 1 -- 172.21.15.7:0/1032411044 learned_addr learned my addr 172.21.15.7:0/1032411044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 0x55abf2265690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f5950e00640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf226bf70 0x55abf226a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 -- 172.21.15.7:0/1032411044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf226bf70 msgr2=0x55abf226a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf226bf70 0x55abf226a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 -- 172.21.15.7:0/1032411044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 msgr2=0x55abf2265150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 0x55abf2265150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.898+0000 7f594b400640 1 -- 172.21.15.7:0/1032411044 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf1fa1620 con 0x55abf225e5f0 2024-06-13T07:06:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.899+0000 7f594b400640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 0x55abf2265690 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5938012480 tx=0x7f593800dd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bed231eebbc431 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.899+0000 7f5936000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.899+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938029020 con 0x55abf225e5f0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.899+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f593801d630 con 0x55abf225e5f0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.900+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5938012e30 con 0x55abf225e5f0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.900+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f593801d830 con 0x55abf225e5f0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.900+0000 7f5955b63740 10 client.26921 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.900+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55abf22728a0 con 0x55abf225e5f0 2024-06-13T07:06:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.900+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5938012e30 con 0x55abf225e5f0 2024-06-13T07:06:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f5938025840 con 0x55abf225e5f0 2024-06-13T07:06:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 10 client.26921 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 20 client.26921 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf2273bf0 0x55abf2275fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 4 client.26921 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.903+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abf21ceb50 con 0x55abf2273bf0 2024-06-13T07:06:24.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.906+0000 7f594be00640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf2273bf0 0x55abf2275fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.907+0000 7f594be00640 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf2273bf0 0x55abf2275fe0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dd88ac6cfc1f4c server_cookie=b6bc80ef487ceb4 in_seq=0 out_seq=0 2024-06-13T07:06:24.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.907+0000 7f5936000640 10 client.26921 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.908+0000 7f5936000640 1 -- 172.21.15.7:0/1032411044 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abf21ceb50 con 0x55abf2273bf0 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.908+0000 7f5936000640 10 client.26921 handle_command_reply: tid=0 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.908+0000 7f5955b63740 1 client.26921 shutdown 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.909+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 msgr2=0x55abf2265690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.909+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 0x55abf2265690 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5938012480 tx=0x7f593800dd70 comp rx=0 tx=0).stop 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 shutdown_connections 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf2273bf0 0x55abf2275fe0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf226bf70 0x55abf226a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf225e5f0 0x55abf2265690 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5936000640 0 client.26921 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 --2- 172.21.15.7:0/1032411044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225dc00 0x55abf2265150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 >> 172.21.15.7:0/1032411044 conn(0x55abf2225fb0 msgr2=0x55abf2258840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 shutdown_connections 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 1 -- 172.21.15.7:0/1032411044 wait complete. 2024-06-13T07:06:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.910+0000 7f5955b63740 20 client.26921 trim_cache size 0 max 16384 2024-06-13T07:06:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 Processor -- start 2024-06-13T07:06:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 -- start start 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf2263cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf225c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f592c023300 con 0x55abf225e5f0 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f592c020bd0 con 0x55abf226bf70 2024-06-13T07:06:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.937+0000 7f5955b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f592c01ea20 con 0x55abf225dc00 2024-06-13T07:06:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.938+0000 7f594be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.938+0000 7f594b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf2263cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.938+0000 7f594be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38740/0 (socket says 172.21.15.7:38740) 2024-06-13T07:06:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.938+0000 7f594be00640 1 -- 172.21.15.7:0/2549904423 learned_addr learned my addr 172.21.15.7:0/2549904423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f5950e00640 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf225c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f594be00640 1 -- 172.21.15.7:0/2549904423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 msgr2=0x55abf225c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f594be00640 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf225c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f594be00640 1 -- 172.21.15.7:0/2549904423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 msgr2=0x55abf2263cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f594be00640 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf2263cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.939+0000 7f594be00640 1 -- 172.21.15.7:0/2549904423 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf2276350 con 0x55abf225dc00 2024-06-13T07:06:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.940+0000 7f594be00640 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f593c0092a0 tx=0x7f593c03b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=659073b546f8c1be server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.940+0000 7f5940c00640 1 -- 172.21.15.7:0/2549904423 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c055070 con 0x55abf225dc00 2024-06-13T07:06:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.940+0000 7f5940c00640 1 -- 172.21.15.7:0/2549904423 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f593c040040 con 0x55abf225dc00 2024-06-13T07:06:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.940+0000 7f5940c00640 1 -- 172.21.15.7:0/2549904423 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c03f070 con 0x55abf225dc00 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.941+0000 7f5955b63740 1 -- 172.21.15.7:0/2549904423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 msgr2=0x55abf22637b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.941+0000 7f5955b63740 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f593c0092a0 tx=0x7f593c03b000 comp rx=0 tx=0).stop 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.942+0000 7f5955b63740 1 -- 172.21.15.7:0/2549904423 shutdown_connections 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.942+0000 7f5955b63740 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf225c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.942+0000 7f5955b63740 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf2263cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.942+0000 7f5955b63740 1 --2- 172.21.15.7:0/2549904423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf22637b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.942+0000 7f5955b63740 1 -- 172.21.15.7:0/2549904423 >> 172.21.15.7:0/2549904423 conn(0x55abf2225fb0 msgr2=0x55abf2260160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.943+0000 7f5955b63740 1 -- 172.21.15.7:0/2549904423 shutdown_connections 2024-06-13T07:06:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.943+0000 7f5955b63740 1 -- 172.21.15.7:0/2549904423 wait complete. 2024-06-13T07:06:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.943+0000 7f5955b63740 1 -- start start 2024-06-13T07:06:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.943+0000 7f5955b63740 1 Processor -- start 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf2284fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf22854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf2286500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f592c01f090 con 0x55abf225e5f0 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f592c020500 con 0x55abf226bf70 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f592c01f760 con 0x55abf225dc00 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5955b63740 10 client.0 _ll_register_callbacks cb 0x7f593800dde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f594be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf2284fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f594be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf2284fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38754/0 (socket says 172.21.15.7:38754) 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f594be00640 1 -- 172.21.15.7:0/86310449 learned_addr learned my addr 172.21.15.7:0/86310449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.944+0000 7f5950e00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf2286500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f594b400640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf22854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 -- 172.21.15.7:0/86310449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 msgr2=0x55abf2284fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf2284fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 -- 172.21.15.7:0/86310449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 msgr2=0x55abf22854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf22854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 -- 172.21.15.7:0/86310449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf2286af0 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5950e00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf2286500 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59440059b0 tx=0x7f594409ffa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dbd4f9798743a82 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.945+0000 7f5936000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594401f040 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f594409b100 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f593c040040 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f59440cf3b0 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5955b63740 10 client.26927 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55abf2287e70 con 0x55abf226bf70 2024-06-13T07:06:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.946+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f593c040040 con 0x55abf226bf70 2024-06-13T07:06:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.947+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f594400f040 con 0x55abf226bf70 2024-06-13T07:06:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.948+0000 7f5955b63740 10 client.26927 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.948+0000 7f5955b63740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.948+0000 7f5955b63740 20 client.26927 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.949+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf22deae0 0x55abf22e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.949+0000 7f5955b63740 4 client.26927 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.949+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abf21c8ea0 con 0x55abf22deae0 2024-06-13T07:06:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.950+0000 7f594be00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf22deae0 0x55abf22e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.950+0000 7f594be00640 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf22deae0 0x55abf22e0f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9eeb402e521645f server_cookie=b47cb8adfe91329f in_seq=0 out_seq=0 2024-06-13T07:06:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:24.951+0000 7f5936000640 10 client.26927 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.159+0000 7f5936000640 1 -- 172.21.15.7:0/86310449 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55abf21c8ea0 con 0x55abf22deae0 2024-06-13T07:06:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.160+0000 7f5936000640 10 client.26927 handle_command_reply: tid=0 2024-06-13T07:06:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.160+0000 7f5955b63740 1 client.26927 shutdown 2024-06-13T07:06:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 msgr2=0x55abf2286500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf2286500 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59440059b0 tx=0x7f594409ffa0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 shutdown_connections 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55abf22deae0 0x55abf22e0f00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55abf226bf70 0x55abf2286500 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55abf225e5f0 0x55abf22854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 --2- 172.21.15.7:0/86310449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abf225dc00 0x55abf2284fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 >> 172.21.15.7:0/86310449 conn(0x55abf2225fb0 msgr2=0x55abf225c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5936000640 0 client.26927 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 shutdown_connections 2024-06-13T07:06:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.162+0000 7f5955b63740 1 -- 172.21.15.7:0/86310449 wait complete. 2024-06-13T07:06:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.163+0000 7f5955b63740 20 client.26927 trim_cache size 0 max 16384 2024-06-13T07:06:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:06:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:06:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 msgr2=0x7f59300a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 0x7f59300a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f593c009850 tx=0x7f593c031400 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 msgr2=0x7f594c1a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f594400eaa0 tx=0x7f594400ef60 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 shutdown_connections 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f59300a2c80 0x7f59300a5140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f594c1057d0 0x7f594c1a14a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f594c104e10 0x7f594c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 --2- 172.21.15.7:0/2594655088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f594c103c20 0x7f594c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 >> 172.21.15.7:0/2594655088 conn(0x7f594c0ff190 msgr2=0x7f594c108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 shutdown_connections 2024-06-13T07:06:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:25.166+0000 7f5952e00640 1 -- 172.21.15.7:0/2594655088 wait complete. 2024-06-13T07:06:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:25 smithi007 ceph-mon[32077]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-06-13T07:06:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:25 smithi189 ceph-mon[36616]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-06-13T07:06:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:25 smithi175 ceph-mon[36886]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.8 MiB/s wr, 566 op/s 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.331+0000 7f7585800700 1 -- 172.21.15.7:0/3645114046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7580074fe0 msgr2=0x7f758010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.331+0000 7f7585800700 1 --2- 172.21.15.7:0/3645114046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7580074fe0 0x7f758010e9b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f75700098f0 tx=0x7f7570009c00 comp rx=0 tx=0).stop 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 -- 172.21.15.7:0/3645114046 shutdown_connections 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 --2- 172.21.15.7:0/3645114046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f75801112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 --2- 172.21.15.7:0/3645114046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75800746c0 0x7f7580074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 --2- 172.21.15.7:0/3645114046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7580074fe0 0x7f758010e9b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 -- 172.21.15.7:0/3645114046 >> 172.21.15.7:0/3645114046 conn(0x7f75800795f0 msgr2=0x7f7580079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 -- 172.21.15.7:0/3645114046 shutdown_connections 2024-06-13T07:06:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.332+0000 7f7585800700 1 -- 172.21.15.7:0/3645114046 wait complete. 2024-06-13T07:06:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.335+0000 7f7585800700 1 Processor -- start 2024-06-13T07:06:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.335+0000 7f7585800700 1 -- start start 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.335+0000 7f7585800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 0x7f758019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f7585800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 0x7f758019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f7585800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f7580196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f7585800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7580104e60 con 0x7f75800746c0 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f7585800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7580104ce0 con 0x7f758010eef0 2024-06-13T07:06:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f7585800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7580104fe0 con 0x7f7580074fe0 2024-06-13T07:06:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f757f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f7580196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f757f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f7580196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55694/0 (socket says 172.21.15.7:55694) 2024-06-13T07:06:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f757f400700 1 -- 172.21.15.7:0/4029181648 learned_addr learned my addr 172.21.15.7:0/4029181648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.336+0000 7f757e000700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 0x7f758019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.337+0000 7f757ea00700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 0x7f758019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.340+0000 7f757e000700 1 -- 172.21.15.7:0/4029181648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 msgr2=0x7f7580196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.340+0000 7f757e000700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f7580196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.340+0000 7f757e000700 1 -- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 msgr2=0x7f758019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.342+0000 7f757e000700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 0x7f758019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.342+0000 7f757e000700 1 -- 172.21.15.7:0/4029181648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580197630 con 0x7f7580074fe0 2024-06-13T07:06:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.342+0000 7f757ea00700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 0x7f758019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.342+0000 7f757e000700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 0x7f758019cc90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7570000c00 tx=0x7f757000fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.345+0000 7f7577400700 1 -- 172.21.15.7:0/4029181648 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570005640 con 0x7f7580074fe0 2024-06-13T07:06:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.345+0000 7f7577400700 1 -- 172.21.15.7:0/4029181648 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757000ba40 con 0x7f7580074fe0 2024-06-13T07:06:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.345+0000 7f7577400700 1 -- 172.21.15.7:0/4029181648 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570017400 con 0x7f7580074fe0 2024-06-13T07:06:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.348+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75801978c0 con 0x7f7580074fe0 2024-06-13T07:06:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.348+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a35b0 con 0x7f7580074fe0 2024-06-13T07:06:27.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.372+0000 7f7577400700 1 -- 172.21.15.7:0/4029181648 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7570003680 con 0x7f7580074fe0 2024-06-13T07:06:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.376+0000 7f7577400700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 0x7f75640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.376+0000 7f7577400700 1 -- 172.21.15.7:0/4029181648 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f75700d06e0 con 0x7f7580074fe0 2024-06-13T07:06:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.379+0000 7f757ea00700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 0x7f75640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.384+0000 7f7590152b80 1 Processor -- start 2024-06-13T07:06:27.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.399+0000 7f757ea00700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 0x7f75640a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75780060b0 tx=0x7f75780076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.399+0000 7f7590152b80 1 -- start start 2024-06-13T07:06:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.399+0000 7f7590152b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f7590152b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdd35b0 0x563e9bdd3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f7590152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdde300 0x563e9bde06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f7590152b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e9bce2a10 con 0x563e9bdd35b0 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f7590152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563e9bd25b00 con 0x563e9bdde300 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f7590152b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e9bce08f0 con 0x563e9bd4fe40 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f757ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f757ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38782/0 (socket says 172.21.15.7:38782) 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f757ea00700 1 -- 172.21.15.7:0/2615565303 learned_addr learned my addr 172.21.15.7:0/2615565303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.402+0000 7f757f400700 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdde300 0x563e9bde06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 -- 172.21.15.7:0/2615565303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdde300 msgr2=0x563e9bde06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdde300 0x563e9bde06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 -- 172.21.15.7:0/2615565303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdd35b0 msgr2=0x563e9bdd3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdd35b0 0x563e9bdd3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 -- 172.21.15.7:0/2615565303 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9bccdf90 con 0x563e9bd4fe40 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.403+0000 7f757ea00700 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7578007a80 tx=0x7f7578010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f0efba0052db98 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.404+0000 7f756fe00700 1 -- 172.21.15.7:0/2615565303 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578036070 con 0x563e9bd4fe40 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.404+0000 7f756fe00700 1 -- 172.21.15.7:0/2615565303 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7578031900 con 0x563e9bd4fe40 2024-06-13T07:06:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.404+0000 7f756fe00700 1 -- 172.21.15.7:0/2615565303 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780158c0 con 0x563e9bd4fe40 2024-06-13T07:06:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 -- 172.21.15.7:0/2615565303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 msgr2=0x563e9bdd3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7578007a80 tx=0x7f7578010000 comp rx=0 tx=0).stop 2024-06-13T07:06:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 -- 172.21.15.7:0/2615565303 shutdown_connections 2024-06-13T07:06:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdde300 0x563e9bde06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bd4fe40 0x563e9bdd3070 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7578007a80 tx=0x7f7578010000 comp rx=0 tx=0).stop 2024-06-13T07:06:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 --2- 172.21.15.7:0/2615565303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdd35b0 0x563e9bdd3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.405+0000 7f7590152b80 1 -- 172.21.15.7:0/2615565303 >> 172.21.15.7:0/2615565303 conn(0x563e9bd00170 msgr2=0x563e9bdcf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.408+0000 7f7590152b80 1 -- 172.21.15.7:0/2615565303 shutdown_connections 2024-06-13T07:06:27.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.408+0000 7f7590152b80 1 -- 172.21.15.7:0/2615565303 wait complete. 2024-06-13T07:06:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.409+0000 7f7590152b80 1 -- start start 2024-06-13T07:06:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.410+0000 7f7590152b80 1 Processor -- start 2024-06-13T07:06:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.411+0000 7f7590152b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bdd35b0 0x563e9bdddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.411+0000 7f7590152b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bddb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.412+0000 7f757e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bddb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.413+0000 7f7590152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 0x563e9bdea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.413+0000 7f757ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bdd35b0 0x563e9bdddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f757e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bddb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43534/0 (socket says 172.21.15.7:43534) 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f757e000700 1 -- 172.21.15.7:0/491019435 learned_addr learned my addr 172.21.15.7:0/491019435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e9bcc1f60 con 0x563e9bdde300 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563e9bd21ac0 con 0x563e9bddb730 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e9bde0fc0 con 0x563e9bdd35b0 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.414+0000 7f757f400700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 0x563e9bdea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f7590152b80 10 client.0 _ll_register_callbacks cb 0x563e9bcbf720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f757f400700 1 -- 172.21.15.7:0/491019435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bdd35b0 msgr2=0x563e9bdddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f757f400700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bdd35b0 0x563e9bdddd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f757f400700 1 -- 172.21.15.7:0/491019435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 msgr2=0x563e9bddb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f757f400700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bddb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.415+0000 7f757f400700 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9bc748d0 con 0x563e9bddb730 2024-06-13T07:06:27.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.416+0000 7f757f400700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 0x563e9bdea2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f758019d4a0 tx=0x7f7568009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2443d1d7affc974a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.416+0000 7f756cc00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7578031900 con 0x563e9bddb730 2024-06-13T07:06:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f756cc00700 1 -- 172.21.15.7:0/491019435 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568023070 con 0x563e9bddb730 2024-06-13T07:06:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f756cc00700 1 -- 172.21.15.7:0/491019435 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7568013480 con 0x563e9bddb730 2024-06-13T07:06:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f756cc00700 1 -- 172.21.15.7:0/491019435 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f756802f050 con 0x563e9bddb730 2024-06-13T07:06:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f7590152b80 10 client.26939 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.418+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=38}) -- 0x563e9bdebdc0 con 0x563e9bddb730 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f756cc00700 1 -- 172.21.15.7:0/491019435 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f75680136c0 con 0x563e9bddb730 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 10 client.26939 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 20 client.26939 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9bdee2f0 0x563e9bdf06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 4 client.26939 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.419+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e9bd33000 con 0x563e9bdee2f0 2024-06-13T07:06:27.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.422+0000 7f757ea00700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9bdee2f0 0x563e9bdf06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.423+0000 7f757ea00700 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9bdee2f0 0x563e9bdf06e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e5dab9a75ed258 server_cookie=196eaf717dc6270c in_seq=0 out_seq=0 2024-06-13T07:06:27.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.423+0000 7f756cc00700 10 client.26939 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:27.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.424+0000 7f756cc00700 1 -- 172.21.15.7:0/491019435 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563e9bd33000 con 0x563e9bdee2f0 2024-06-13T07:06:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.424+0000 7f756cc00700 10 client.26939 handle_command_reply: tid=0 2024-06-13T07:06:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.424+0000 7f7590152b80 1 client.26939 shutdown 2024-06-13T07:06:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 msgr2=0x563e9bdea2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 0x563e9bdea2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f758019d4a0 tx=0x7f7568009dc0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 shutdown_connections 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9bdee2f0 0x563e9bdf06e0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bddb730 0x563e9bdea2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bdd35b0 0x563e9bdddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 --2- 172.21.15.7:0/491019435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bddb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 >> 172.21.15.7:0/491019435 conn(0x563e9bd00170 msgr2=0x563e9bdce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.425+0000 7f756cc00700 0 client.26939 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.426+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 shutdown_connections 2024-06-13T07:06:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.426+0000 7f7590152b80 1 -- 172.21.15.7:0/491019435 wait complete. 2024-06-13T07:06:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.426+0000 7f7590152b80 20 client.26939 trim_cache size 0 max 16384 2024-06-13T07:06:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.472+0000 7f7590152b80 1 Processor -- start 2024-06-13T07:06:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 -- start start 2024-06-13T07:06:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a6030 con 0x563e9bdde300 2024-06-13T07:06:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a38c0 con 0x563e9bdd35b0 2024-06-13T07:06:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.474+0000 7f7590152b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7580072640 con 0x563e9bddb730 2024-06-13T07:06:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.476+0000 7f757e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.476+0000 7f757ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.476+0000 7f757ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55724/0 (socket says 172.21.15.7:55724) 2024-06-13T07:06:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.476+0000 7f757e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38806/0 (socket says 172.21.15.7:38806) 2024-06-13T07:06:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.477+0000 7f757ea00700 1 -- 172.21.15.7:0/1793363001 learned_addr learned my addr 172.21.15.7:0/1793363001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.478+0000 7f757e000700 1 -- 172.21.15.7:0/1793363001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 msgr2=0x563e9bdcf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.478+0000 7f757e000700 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.478+0000 7f757e000700 1 -- 172.21.15.7:0/1793363001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 msgr2=0x563e9bdea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.478+0000 7f757e000700 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.478+0000 7f757e000700 1 -- 172.21.15.7:0/1793363001 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9bdd9600 con 0x563e9bddb730 2024-06-13T07:06:27.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.479+0000 7f757ea00700 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:27.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.480+0000 7f757e000700 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f757009cc20 tx=0x7f757009c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2e0ace2ad99862 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.480+0000 7f756f400700 1 -- 172.21.15.7:0/1793363001 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757009d390 con 0x563e9bddb730 2024-06-13T07:06:27.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.480+0000 7f756f400700 1 -- 172.21.15.7:0/1793363001 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75700b3070 con 0x563e9bddb730 2024-06-13T07:06:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.480+0000 7f756f400700 1 -- 172.21.15.7:0/1793363001 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570099300 con 0x563e9bddb730 2024-06-13T07:06:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.481+0000 7f7590152b80 1 -- 172.21.15.7:0/1793363001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 msgr2=0x563e9bdce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.481+0000 7f7590152b80 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f757009cc20 tx=0x7f757009c7d0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 -- 172.21.15.7:0/1793363001 shutdown_connections 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdcf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 --2- 172.21.15.7:0/1793363001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdce5c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 -- 172.21.15.7:0/1793363001 >> 172.21.15.7:0/1793363001 conn(0x563e9bd00170 msgr2=0x563e9bdcdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 -- 172.21.15.7:0/1793363001 shutdown_connections 2024-06-13T07:06:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.482+0000 7f7590152b80 1 -- 172.21.15.7:0/1793363001 wait complete. 2024-06-13T07:06:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.483+0000 7f7590152b80 1 -- start start 2024-06-13T07:06:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.483+0000 7f7590152b80 1 Processor -- start 2024-06-13T07:06:27.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.487+0000 7f7590152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.487+0000 7f7590152b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdf61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.488+0000 7f757ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.488+0000 7f757ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55730/0 (socket says 172.21.15.7:55730) 2024-06-13T07:06:27.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.488+0000 7f757ea00700 1 -- 172.21.15.7:0/2921409936 learned_addr learned my addr 172.21.15.7:0/2921409936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:27.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.489+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdf66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.491+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7580072cb0 con 0x563e9bdde300 2024-06-13T07:06:27.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.491+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7580074120 con 0x563e9bdd35b0 2024-06-13T07:06:27.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.491+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7580073380 con 0x563e9bddb730 2024-06-13T07:06:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.492+0000 7f757e000700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdf61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.494+0000 7f7590152b80 10 client.0 _ll_register_callbacks cb 0x563e9bcbf720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.494+0000 7f757f400700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdf66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.495+0000 7f757ea00700 1 -- 172.21.15.7:0/2921409936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 msgr2=0x563e9bdf61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.495+0000 7f757ea00700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdf61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.495+0000 7f757ea00700 1 -- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 msgr2=0x563e9bdf66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.495+0000 7f757ea00700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdf66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.495+0000 7f757ea00700 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9bde4a80 con 0x563e9bdd35b0 2024-06-13T07:06:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.496+0000 7f757ea00700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x563e9bdd03b0 tx=0x7f75780060e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4572385b05dc2923 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.496+0000 7f757f400700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdf66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.496+0000 7f756cc00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.497+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578036040 con 0x563e9bdd35b0 2024-06-13T07:06:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.497+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7578031ab0 con 0x563e9bdd35b0 2024-06-13T07:06:27.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.497+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75700b3070 con 0x563e9bdd35b0 2024-06-13T07:06:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.499+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7578017190 con 0x563e9bdd35b0 2024-06-13T07:06:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.499+0000 7f7590152b80 10 client.26945 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.499+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563e9be59330 con 0x563e9bdd35b0 2024-06-13T07:06:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75700b3070 con 0x563e9bdd35b0 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f7578015ce0 con 0x563e9bdd35b0 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 10 client.26945 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 20 client.26945 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9be59560 0x563e9be5b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 4 client.26945 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e9bd2d580 con 0x563e9be59560 2024-06-13T07:06:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.503+0000 7f757e000700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9be59560 0x563e9be5b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.505+0000 7f757e000700 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9be59560 0x563e9be5b950 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611acde6c91e23fa server_cookie=6a68d1cae5a93b13 in_seq=0 out_seq=0 2024-06-13T07:06:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.505+0000 7f756cc00700 10 client.26945 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.506+0000 7f756cc00700 1 -- 172.21.15.7:0/2921409936 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e9bd2d580 con 0x563e9be59560 2024-06-13T07:06:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.506+0000 7f756cc00700 10 client.26945 handle_command_reply: tid=0 2024-06-13T07:06:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.506+0000 7f7590152b80 1 client.26945 shutdown 2024-06-13T07:06:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 msgr2=0x563e9bdf5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x563e9bdd03b0 tx=0x7f75780060e0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 shutdown_connections 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563e9be59560 0x563e9be5b950 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e9bdde300 0x563e9bdf66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563e9bdd35b0 0x563e9bdf5c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 --2- 172.21.15.7:0/2921409936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e9bddb730 0x563e9bdf61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 >> 172.21.15.7:0/2921409936 conn(0x563e9bd00170 msgr2=0x563e9bdce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.507+0000 7f756cc00700 0 client.26945 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.508+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 shutdown_connections 2024-06-13T07:06:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.508+0000 7f7590152b80 1 -- 172.21.15.7:0/2921409936 wait complete. 2024-06-13T07:06:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.508+0000 7f7590152b80 20 client.26945 trim_cache size 0 max 16384 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "7f39e750-7e75-41ac-9e71-ea792baf0372": { 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7f39e750-7e75-41ac-9e71-ea792baf0372", 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:06:27.509 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 msgr2=0x7f75640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 0x7f75640a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75780060b0 tx=0x7f75780076a0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 msgr2=0x7f758019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 0x7f758019cc90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7570000c00 tx=0x7f757000fa80 comp rx=0 tx=0).stop 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 shutdown_connections 2024-06-13T07:06:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75640a2d70 0x7f75640a5230 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f758010eef0 0x7f7580196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75800746c0 0x7f758019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 --2- 172.21.15.7:0/4029181648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580074fe0 0x7f758019cc90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.510+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 >> 172.21.15.7:0/4029181648 conn(0x7f75800795f0 msgr2=0x7f75800ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.514+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 shutdown_connections 2024-06-13T07:06:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:06:27.515+0000 7f7585800700 1 -- 172.21.15.7:0/4029181648 wait complete. 2024-06-13T07:06:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 625.0 s, 402.4 tps, lat 247.105 ms stddev 287.857 2024-06-13T07:06:27.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:27 smithi007 ceph-mon[32077]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 526 op/s 2024-06-13T07:06:27.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:27 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:06:27.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:27 smithi175 ceph-mon[36886]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 526 op/s 2024-06-13T07:06:27.918 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:27 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:06:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:27 smithi189 ceph-mon[36616]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 526 op/s 2024-06-13T07:06:28.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:27 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:06:28.954 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:28 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:06:28.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:28 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:06:29.039 INFO:tasks.cephfs.filesystem:scrub status for tag:7f39e750-7e75-41ac-9e71-ea792baf0372 - {'path': '/', 'tag': '7f39e750-7e75-41ac-9e71-ea792baf0372', 'options': 'recursive,force'} 2024-06-13T07:06:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:28 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:06:29.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:28 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:06:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:28 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:06:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:28 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:06:30.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:29 smithi189 ceph-mon[36616]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.6 MiB/s wr, 519 op/s 2024-06-13T07:06:30.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:29 smithi175 ceph-mon[36886]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.6 MiB/s wr, 519 op/s 2024-06-13T07:06:30.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:29 smithi007 ceph-mon[32077]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.6 MiB/s wr, 519 op/s 2024-06-13T07:06:32.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:31 smithi189 ceph-mon[36616]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 574 op/s 2024-06-13T07:06:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:31 smithi175 ceph-mon[36886]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 574 op/s 2024-06-13T07:06:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:31 smithi007 ceph-mon[32077]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 574 op/s 2024-06-13T07:06:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 630.0 s, 494.2 tps, lat 202.428 ms stddev 253.030 2024-06-13T07:06:34.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:33 smithi189 ceph-mon[36616]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.7 MiB/s wr, 551 op/s 2024-06-13T07:06:34.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:33 smithi175 ceph-mon[36886]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.7 MiB/s wr, 551 op/s 2024-06-13T07:06:34.167 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:33 smithi007 ceph-mon[32077]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.7 MiB/s wr, 551 op/s 2024-06-13T07:06:36.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:35 smithi189 ceph-mon[36616]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-13T07:06:36.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:35 smithi175 ceph-mon[36886]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-13T07:06:36.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:35 smithi007 ceph-mon[32077]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.0 MiB/s wr, 515 op/s 2024-06-13T07:06:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 635.0 s, 464.4 tps, lat 211.364 ms stddev 223.598 2024-06-13T07:06:38.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:37 smithi189 ceph-mon[36616]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.3 MiB/s wr, 429 op/s 2024-06-13T07:06:38.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:37 smithi175 ceph-mon[36886]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.3 MiB/s wr, 429 op/s 2024-06-13T07:06:38.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:37 smithi007 ceph-mon[32077]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.3 MiB/s wr, 429 op/s 2024-06-13T07:06:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 -- 172.21.15.7:0/1150872697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02981076e0 msgr2=0x7f0298109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 --2- 172.21.15.7:0/1150872697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02981076e0 0x7f0298109b30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0290009be0 tx=0x7f029002a6c0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 -- 172.21.15.7:0/1150872697 shutdown_connections 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 --2- 172.21.15.7:0/1150872697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02981076e0 0x7f0298109b30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 --2- 172.21.15.7:0/1150872697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0298104db0 0x7f02981071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 --2- 172.21.15.7:0/1150872697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298102480 0x7f0298104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 -- 172.21.15.7:0/1150872697 >> 172.21.15.7:0/1150872697 conn(0x7f02980fbe70 msgr2=0x7f02980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 -- 172.21.15.7:0/1150872697 shutdown_connections 2024-06-13T07:06:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.373+0000 7f029ee00640 1 -- 172.21.15.7:0/1150872697 wait complete. 2024-06-13T07:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.374+0000 7f029ee00640 1 Processor -- start 2024-06-13T07:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 -- start start 2024-06-13T07:06:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298104db0 0x7f029819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02981076e0 0x7f02981a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f029810d680 con 0x7f0298104db0 2024-06-13T07:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f029810d500 con 0x7f0298102480 2024-06-13T07:06:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.375+0000 7f029ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f029810d800 con 0x7f02981076e0 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.376+0000 7f0297e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.376+0000 7f0297e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44434/0 (socket says 172.21.15.7:44434) 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.376+0000 7f0297e00640 1 -- 172.21.15.7:0/3148422843 learned_addr learned my addr 172.21.15.7:0/3148422843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.376+0000 7f029ce00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02981076e0 0x7f02981a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f028fe00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298104db0 0x7f029819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f0297e00640 1 -- 172.21.15.7:0/3148422843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02981076e0 msgr2=0x7f02981a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f0297e00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02981076e0 0x7f02981a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f0297e00640 1 -- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298104db0 msgr2=0x7f029819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f0297e00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298104db0 0x7f029819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.377+0000 7f0297e00640 1 -- 172.21.15.7:0/3148422843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02981a1970 con 0x7f0298102480 2024-06-13T07:06:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.378+0000 7f0297e00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f028800c920 tx=0x7f028800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.379+0000 7f0295600640 1 -- 172.21.15.7:0/3148422843 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288007c70 con 0x7f0298102480 2024-06-13T07:06:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.379+0000 7f0295600640 1 -- 172.21.15.7:0/3148422843 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0288007e10 con 0x7f0298102480 2024-06-13T07:06:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.379+0000 7f0295600640 1 -- 172.21.15.7:0/3148422843 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288018890 con 0x7f0298102480 2024-06-13T07:06:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.379+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02981a1c60 con 0x7f0298102480 2024-06-13T07:06:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.379+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02981a2160 con 0x7f0298102480 2024-06-13T07:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.382+0000 7f0295600640 1 -- 172.21.15.7:0/3148422843 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0288018a70 con 0x7f0298102480 2024-06-13T07:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 Processor -- start 2024-06-13T07:06:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 -- start start 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c87500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 0x5610c8752bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5610c86ae420 con 0x5610c8750760 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5610c8655c50 con 0x5610c874ac00 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f02a1d95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610c86aa3e0 con 0x5610c874b5f0 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f029ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f029ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44438/0 (socket says 172.21.15.7:44438) 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.383+0000 7f029ce00640 1 -- 172.21.15.7:0/2472620433 learned_addr learned my addr 172.21.15.7:0/2472620433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f0297e00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c87500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f028fe00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 0x5610c8752bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f0295600640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 0x7f027c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f029ce00640 1 -- 172.21.15.7:0/2472620433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 msgr2=0x5610c87500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f029ce00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c87500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f029ce00640 1 -- 172.21.15.7:0/2472620433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 msgr2=0x5610c8752bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f029ce00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 0x5610c8752bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f029ce00640 1 -- 172.21.15.7:0/2472620433 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c83c2210 con 0x5610c874ac00 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f028fe00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 0x5610c8752bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.384+0000 7f0295600640 1 -- 172.21.15.7:0/3148422843 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f02880cf3c0 con 0x7f0298102480 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.385+0000 7f029ce00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 0x7f027c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.385+0000 7f029ce00640 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0290009be0 tx=0x7f0290030be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f37328a84d28153 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.385+0000 7f028d600640 1 -- 172.21.15.7:0/2472620433 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290041040 con 0x5610c874ac00 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.385+0000 7f028d600640 1 -- 172.21.15.7:0/2472620433 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0290003e50 con 0x5610c874ac00 2024-06-13T07:06:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.385+0000 7f028d600640 1 -- 172.21.15.7:0/2472620433 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290036040 con 0x5610c874ac00 2024-06-13T07:06:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 -- 172.21.15.7:0/2472620433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 msgr2=0x5610c874b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0290009be0 tx=0x7f0290030be0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f029ce00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 0x7f027c0a5260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0290049000 tx=0x7f029003d850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 -- 172.21.15.7:0/2472620433 shutdown_connections 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8750760 0x5610c8752bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c87500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 --2- 172.21.15.7:0/2472620433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874ac00 0x5610c874b020 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 -- 172.21.15.7:0/2472620433 >> 172.21.15.7:0/2472620433 conn(0x5610c8712fb0 msgr2=0x5610c8746680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 -- 172.21.15.7:0/2472620433 shutdown_connections 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.386+0000 7f02a1d95740 1 -- 172.21.15.7:0/2472620433 wait complete. 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.387+0000 7f02a1d95740 1 -- start start 2024-06-13T07:06:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.387+0000 7f02a1d95740 1 Processor -- start 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.387+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874ac00 0x5610c875d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.387+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c875dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.387+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f02a1d95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5610c8753710 con 0x5610c874ac00 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f02a1d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5610c86690b0 con 0x5610c8750760 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f02a1d95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610c8753890 con 0x5610c874b5f0 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f02a1d95740 10 client.0 _ll_register_callbacks cb 0x5610c8654f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f029ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f029ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:44442/0 (socket says 172.21.15.7:44442) 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f029ce00640 1 -- 172.21.15.7:0/3503252419 learned_addr learned my addr 172.21.15.7:0/3503252419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.388+0000 7f0297e00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874ac00 0x5610c875d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 -- 172.21.15.7:0/3503252419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 msgr2=0x5610c875dc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:06:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c875dc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 -- 172.21.15.7:0/3503252419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874ac00 msgr2=0x5610c875d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874ac00 0x5610c875d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 -- 172.21.15.7:0/3503252419 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c848e620 con 0x5610c8750760 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f029ce00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0290034040 tx=0x7f0290040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714a2b0eb37a9a24 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.389+0000 7f0286000640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.390+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0290036030 con 0x5610c8750760 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.390+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f029003d460 con 0x5610c8750760 2024-06-13T07:06:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.390+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0290003e50 con 0x5610c8750760 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.391+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f029003d660 con 0x5610c8750760 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.391+0000 7f02a1d95740 10 client.26957 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.391+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5610c8756b10 con 0x5610c8750760 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0290003e50 con 0x5610c8750760 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f02900303a0 con 0x5610c8750760 2024-06-13T07:06:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 10 client.26957 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 20 client.26957 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8757f20 0x5610c875a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 4 client.26957 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.392+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610c86bbb50 con 0x5610c8757f20 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.393+0000 7f0297e00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8757f20 0x5610c875a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.393+0000 7f0286000640 10 client.26957 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.393+0000 7f0297e00640 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8757f20 0x5610c875a350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf35be01c60a78 server_cookie=9bd60b0c93894427 in_seq=0 out_seq=0 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.394+0000 7f0286000640 1 -- 172.21.15.7:0/3503252419 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5610c86bbb50 con 0x5610c8757f20 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.394+0000 7f0286000640 10 client.26957 handle_command_reply: tid=0 2024-06-13T07:06:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.394+0000 7f02a1d95740 1 client.26957 shutdown 2024-06-13T07:06:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.394+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 msgr2=0x5610c8764c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.394+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0290034040 tx=0x7f0290040000 comp rx=0 tx=0).stop 2024-06-13T07:06:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 shutdown_connections 2024-06-13T07:06:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8757f20 0x5610c875a350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8750760 0x5610c8764c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874b5f0 0x5610c875dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3503252419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874ac00 0x5610c875d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 >> 172.21.15.7:0/3503252419 conn(0x5610c8712fb0 msgr2=0x5610c8745840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f0286000640 0 client.26957 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 shutdown_connections 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 1 -- 172.21.15.7:0/3503252419 wait complete. 2024-06-13T07:06:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.395+0000 7f02a1d95740 20 client.26957 trim_cache size 0 max 16384 2024-06-13T07:06:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 Processor -- start 2024-06-13T07:06:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 -- start start 2024-06-13T07:06:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8712fb0 0x5610c8746810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 0x5610c877b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02981aaaf0 con 0x5610c874b5f0 2024-06-13T07:06:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a83c0 con 0x5610c8712fb0 2024-06-13T07:06:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f02a1d95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a6210 con 0x5610c874ac00 2024-06-13T07:06:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f028fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f028fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38986/0 (socket says 172.21.15.7:38986) 2024-06-13T07:06:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.416+0000 7f028fe00640 1 -- 172.21.15.7:0/3649303552 learned_addr learned my addr 172.21.15.7:0/3649303552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f0297e00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8712fb0 0x5610c8746810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f029ce00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 0x5610c877b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 -- 172.21.15.7:0/3649303552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8712fb0 msgr2=0x5610c8746810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8712fb0 0x5610c8746810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 -- 172.21.15.7:0/3649303552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 msgr2=0x5610c877b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 0x5610c877b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 -- 172.21.15.7:0/3649303552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c874b9d0 con 0x5610c874ac00 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f029ce00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 0x5610c877b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028fe00640 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02800044f0 tx=0x7f028000db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71db5ed02aef9528 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028cc00640 1 -- 172.21.15.7:0/3649303552 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280003b50 con 0x5610c874ac00 2024-06-13T07:06:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028cc00640 1 -- 172.21.15.7:0/3649303552 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280003cf0 con 0x5610c874ac00 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.417+0000 7f028cc00640 1 -- 172.21.15.7:0/3649303552 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028000f680 con 0x5610c874ac00 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 -- 172.21.15.7:0/3649303552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 msgr2=0x5610c8764c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02800044f0 tx=0x7f028000db30 comp rx=0 tx=0).stop 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 -- 172.21.15.7:0/3649303552 shutdown_connections 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c874b5f0 0x5610c877b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8764c50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3649303552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c8712fb0 0x5610c8746810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 -- 172.21.15.7:0/3649303552 >> 172.21.15.7:0/3649303552 conn(0x5610c875a680 msgr2=0x5610c8746100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.418+0000 7f02a1d95740 1 -- 172.21.15.7:0/3649303552 shutdown_connections 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.419+0000 7f02a1d95740 1 -- 172.21.15.7:0/3649303552 wait complete. 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.419+0000 7f02a1d95740 1 -- start start 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.419+0000 7f02a1d95740 1 Processor -- start 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8760da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 0x5610c8763d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a6880 con 0x5610c8712fb0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a7cf0 con 0x5610c874b5f0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02981a6f50 con 0x5610c874ac00 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f02a1d95740 10 client.0 _ll_register_callbacks cb 0x7f028000d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.420+0000 7f0297e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58306/0 (socket says 172.21.15.7:58306) 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 -- 172.21.15.7:0/3570851405 learned_addr learned my addr 172.21.15.7:0/3570851405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f029ce00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 0x5610c8763d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f028fe00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8760da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 -- 172.21.15.7:0/3570851405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 msgr2=0x5610c8760da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8760da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 -- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 msgr2=0x5610c8763d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 0x5610c8763d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 -- 172.21.15.7:0/3570851405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c8764360 con 0x5610c8712fb0 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f029ce00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 0x5610c8763d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.421+0000 7f0297e00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f02880ccb80 tx=0x7f02880d2ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a3ed95e5f6d48b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f0286000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028809b8e0 con 0x5610c8712fb0 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f02880afe00 con 0x5610c8712fb0 2024-06-13T07:06:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0280003cf0 con 0x5610c8712fb0 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f028809fc90 con 0x5610c8712fb0 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f02a1d95740 10 client.16290 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.422+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5610c87dc700 con 0x5610c8712fb0 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0280003cf0 con 0x5610c8712fb0 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f0288097f50 con 0x5610c8712fb0 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 10 client.16290 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 20 client.16290 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8750760 0x5610c87df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 4 client.16290 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610c86b5ea0 con 0x5610c8750760 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.423+0000 7f028fe00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8750760 0x5610c87df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.424+0000 7f028fe00640 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8750760 0x5610c87df010 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52a36e8f84fe15f server_cookie=766a1dba07e48779 in_seq=0 out_seq=0 2024-06-13T07:06:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.424+0000 7f0286000640 10 client.16290 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f0286000640 1 -- 172.21.15.7:0/3570851405 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5610c86b5ea0 con 0x5610c8750760 2024-06-13T07:06:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f0286000640 10 client.16290 handle_command_reply: tid=0 2024-06-13T07:06:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 client.16290 shutdown 2024-06-13T07:06:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 msgr2=0x5610c8760860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f02880ccb80 tx=0x7f02880d2ea0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 shutdown_connections 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5610c8750760 0x5610c87df010 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5610c874b5f0 0x5610c8763d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610c874ac00 0x5610c8760da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f0286000640 0 client.16290 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 --2- 172.21.15.7:0/3570851405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610c8712fb0 0x5610c8760860 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.437+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 >> 172.21.15.7:0/3570851405 conn(0x5610c875a680 msgr2=0x5610c87495b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.438+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 shutdown_connections 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.438+0000 7f02a1d95740 1 -- 172.21.15.7:0/3570851405 wait complete. 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.438+0000 7f02a1d95740 20 client.16290 trim_cache size 0 max 16384 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:06:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 msgr2=0x7f027c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 0x7f027c0a5260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0290049000 tx=0x7f029003d850 comp rx=0 tx=0).stop 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 msgr2=0x7f0298199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f028800c920 tx=0x7f028800cde0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 shutdown_connections 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f027c0a2da0 0x7f027c0a5260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02981076e0 0x7f02981a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0298104db0 0x7f029819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 --2- 172.21.15.7:0/3148422843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0298102480 0x7f0298199d50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 >> 172.21.15.7:0/3148422843 conn(0x7f02980fbe70 msgr2=0x7f0298101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 shutdown_connections 2024-06-13T07:06:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:38.439+0000 7f029ee00640 1 -- 172.21.15.7:0/3148422843 wait complete. 2024-06-13T07:06:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:39 smithi189 ceph-mon[36616]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-13T07:06:40.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:39 smithi175 ceph-mon[36886]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-13T07:06:40.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:39 smithi007 ceph-mon[32077]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 447 op/s 2024-06-13T07:06:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:41 smithi189 ceph-mon[36616]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 476 op/s 2024-06-13T07:06:42.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:41 smithi175 ceph-mon[36886]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 476 op/s 2024-06-13T07:06:42.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:41 smithi007 ceph-mon[32077]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 476 op/s 2024-06-13T07:06:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 640.0 s, 517.0 tps, lat 196.987 ms stddev 227.575 2024-06-13T07:06:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:43 smithi189 ceph-mon[36616]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-06-13T07:06:44.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:43 smithi175 ceph-mon[36886]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-06-13T07:06:44.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:43 smithi007 ceph-mon[32077]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.7 MiB/s wr, 476 op/s 2024-06-13T07:06:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:45 smithi189 ceph-mon[36616]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2024-06-13T07:06:46.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:45 smithi175 ceph-mon[36886]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2024-06-13T07:06:46.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:45 smithi007 ceph-mon[32077]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2024-06-13T07:06:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 645.0 s, 581.0 tps, lat 176.525 ms stddev 236.609 2024-06-13T07:06:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:47 smithi189 ceph-mon[36616]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-06-13T07:06:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:47 smithi175 ceph-mon[36886]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-06-13T07:06:48.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:47 smithi007 ceph-mon[32077]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-06-13T07:06:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:49 smithi189 ceph-mon[36616]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-06-13T07:06:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:49 smithi175 ceph-mon[36886]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-06-13T07:06:50.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:49 smithi007 ceph-mon[32077]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-06-13T07:06:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.719+0000 7f7adf200640 1 -- 172.21.15.7:0/370070979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad80990f0 msgr2=0x7f7ad8099570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.719+0000 7f7adf200640 1 --2- 172.21.15.7:0/370070979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad80990f0 0x7f7ad8099570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009900 tx=0x7f7acc02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 -- 172.21.15.7:0/370070979 shutdown_connections 2024-06-13T07:06:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 --2- 172.21.15.7:0/370070979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad8099ab0 0x7f7ad80a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 --2- 172.21.15.7:0/370070979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad80990f0 0x7f7ad8099570 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 --2- 172.21.15.7:0/370070979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad8097f20 0x7f7ad8098360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 -- 172.21.15.7:0/370070979 >> 172.21.15.7:0/370070979 conn(0x7f7ad80934e0 msgr2=0x7f7ad8095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 -- 172.21.15.7:0/370070979 shutdown_connections 2024-06-13T07:06:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.722+0000 7f7adf200640 1 -- 172.21.15.7:0/370070979 wait complete. 2024-06-13T07:06:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.724+0000 7f7adf200640 1 Processor -- start 2024-06-13T07:06:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.728+0000 7f7adf200640 1 -- start start 2024-06-13T07:06:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.728+0000 7f7adf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 0x7f7ad812e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 0x7f7ad812e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 0x7f7ad812e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52970/0 (socket says 172.21.15.7:52970) 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.728+0000 7f7adf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 0x7f7ad812e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adde00640 1 -- 172.21.15.7:0/3268209146 learned_addr learned my addr 172.21.15.7:0/3268209146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad8099ab0 0x7f7ad8135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad80a3c40 con 0x7f7ad8099ab0 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad80a3ac0 con 0x7f7ad80990f0 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.729+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad80a3dc0 con 0x7f7ad8097f20 2024-06-13T07:06:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 0x7f7ad812e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 -- 172.21.15.7:0/3268209146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 msgr2=0x7f7ad812e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 0x7f7ad812e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 -- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad8099ab0 msgr2=0x7f7ad8135810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad8099ab0 0x7f7ad8135810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.730+0000 7f7add400640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad8135e80 con 0x7f7ad80990f0 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.731+0000 7f7add400640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 0x7f7ad812e7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009850 tx=0x7f7acc032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.731+0000 7f7ad6a00640 1 -- 172.21.15.7:0/3268209146 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc03f030 con 0x7f7ad80990f0 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.731+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad8136110 con 0x7f7ad80990f0 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.731+0000 7f7ad6a00640 1 -- 172.21.15.7:0/3268209146 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7acc003ad0 con 0x7f7ad80990f0 2024-06-13T07:06:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.731+0000 7f7ad6a00640 1 -- 172.21.15.7:0/3268209146 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc035070 con 0x7f7ad80990f0 2024-06-13T07:06:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.732+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad8136620 con 0x7f7ad80990f0 2024-06-13T07:06:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.738+0000 7f7ad6a00640 1 -- 172.21.15.7:0/3268209146 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7acc042030 con 0x7f7ad80990f0 2024-06-13T07:06:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.743+0000 7f7ad6a00640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 0x7f7ac40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.744+0000 7f7ad6a00640 1 -- 172.21.15.7:0/3268209146 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7acc0ef8c0 con 0x7f7ad80990f0 2024-06-13T07:06:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.745+0000 7f7adde00640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 0x7f7ac40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.748+0000 7f7ae777f740 1 Processor -- start 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 -- start start 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7adde00640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 0x7f7ac40a51a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0005fd0 tx=0x7f7ad000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0c93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0c9b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eae0bee420 con 0x55eae0c99150 2024-06-13T07:06:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eae0b95c50 con 0x55eae0c93fe0 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.749+0000 7f7ae777f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eae0bea3e0 con 0x55eae0c935f0 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49088/0 (socket says 172.21.15.7:49088) 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 -- 172.21.15.7:0/4236914194 learned_addr learned my addr 172.21.15.7:0/4236914194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7adde00640 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0c93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 -- 172.21.15.7:0/4236914194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 msgr2=0x55eae0c93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0c93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 -- 172.21.15.7:0/4236914194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 msgr2=0x55eae0c9b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0c9b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 -- 172.21.15.7:0/4236914194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae0902210 con 0x55eae0c93fe0 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7adde00640 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0c93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.750+0000 7f7add400640 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0c44c0 tx=0x7f7acc003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b00143a3869d40 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.751+0000 7f7acb000640 1 -- 172.21.15.7:0/4236914194 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc049040 con 0x55eae0c93fe0 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.751+0000 7f7acb000640 1 -- 172.21.15.7:0/4236914194 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7acc0bbd80 con 0x55eae0c93fe0 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.751+0000 7f7acb000640 1 -- 172.21.15.7:0/4236914194 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0b8360 con 0x55eae0c93fe0 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 -- 172.21.15.7:0/4236914194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 msgr2=0x55eae0c98ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0c44c0 tx=0x7f7acc003f60 comp rx=0 tx=0).stop 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 -- 172.21.15.7:0/4236914194 shutdown_connections 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0c9b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c98ae0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 --2- 172.21.15.7:0/4236914194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0c93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 -- 172.21.15.7:0/4236914194 >> 172.21.15.7:0/4236914194 conn(0x55eae0c52fb0 msgr2=0x55eae0c86860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 -- 172.21.15.7:0/4236914194 shutdown_connections 2024-06-13T07:06:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.752+0000 7f7ae777f740 1 -- 172.21.15.7:0/4236914194 wait complete. 2024-06-13T07:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.753+0000 7f7ae777f740 1 -- start start 2024-06-13T07:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.753+0000 7f7ae777f740 1 Processor -- start 2024-06-13T07:06:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0c9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 0x55eae0ca4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eae0c9bec0 con 0x55eae0c935f0 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eae0ba90b0 con 0x55eae0c93fe0 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eae0c9c040 con 0x55eae0c99150 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.754+0000 7f7ae777f740 10 client.0 _ll_register_callbacks cb 0x55eae0b94f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.755+0000 7f7add400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.755+0000 7f7adde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0c9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.755+0000 7f7add400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49102/0 (socket says 172.21.15.7:49102) 2024-06-13T07:06:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.755+0000 7f7add400640 1 -- 172.21.15.7:0/3538319103 learned_addr learned my addr 172.21.15.7:0/3538319103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.756+0000 7f7ade800640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 0x55eae0ca4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.756+0000 7f7add400640 1 -- 172.21.15.7:0/3538319103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 msgr2=0x55eae0ca4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7add400640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 0x55eae0ca4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7add400640 1 -- 172.21.15.7:0/3538319103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 msgr2=0x55eae0c9d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7add400640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0c9d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7add400640 1 -- 172.21.15.7:0/3538319103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae09ce620 con 0x55eae0c93fe0 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7adde00640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0c9d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.757+0000 7f7ade800640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 0x55eae0ca4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.758+0000 7f7add400640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0bbd10 tx=0x7f7acc0c49f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad96e3f4646231d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.758+0000 7f7abf400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:06:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.758+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc09f060 con 0x55eae0c93fe0 2024-06-13T07:06:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.758+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7acc03b460 con 0x55eae0c93fe0 2024-06-13T07:06:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.758+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7acc0bbd80 con 0x55eae0c93fe0 2024-06-13T07:06:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.761+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7acc049070 con 0x55eae0c93fe0 2024-06-13T07:06:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.761+0000 7f7ae777f740 10 client.26975 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.761+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eae0c9bec0 con 0x55eae0c93fe0 2024-06-13T07:06:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7acc0bbd80 con 0x55eae0c93fe0 2024-06-13T07:06:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f7acc0a2080 con 0x55eae0c93fe0 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 10 client.26975 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 10 client.26975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 20 client.26975 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0ca94a0 0x55eae0cab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 4 client.26975 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.763+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae0bfbb50 con 0x55eae0ca94a0 2024-06-13T07:06:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.764+0000 7f7adde00640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0ca94a0 0x55eae0cab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.765+0000 7f7adde00640 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0ca94a0 0x55eae0cab8c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b166bdfcdb13960 server_cookie=7f4fb76517e5bead in_seq=0 out_seq=0 2024-06-13T07:06:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.765+0000 7f7abf400640 10 client.26975 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.766+0000 7f7abf400640 1 -- 172.21.15.7:0/3538319103 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eae0bfbb50 con 0x55eae0ca94a0 2024-06-13T07:06:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.766+0000 7f7abf400640 10 client.26975 handle_command_reply: tid=0 2024-06-13T07:06:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.767+0000 7f7ae777f740 1 client.26975 shutdown 2024-06-13T07:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.770+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 msgr2=0x55eae0c9dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.770+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0bbd10 tx=0x7f7acc0c49f0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 shutdown_connections 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0ca94a0 0x55eae0cab8c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c99150 0x55eae0ca4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0c9dbc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 --2- 172.21.15.7:0/3538319103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0c9d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7abf400640 0 client.26975 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 >> 172.21.15.7:0/3538319103 conn(0x55eae0c52fb0 msgr2=0x55eae0c859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.773+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 shutdown_connections 2024-06-13T07:06:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.774+0000 7f7ae777f740 1 -- 172.21.15.7:0/3538319103 wait complete. 2024-06-13T07:06:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.774+0000 7f7ae777f740 20 client.26975 trim_cache size 0 max 16384 2024-06-13T07:06:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.827+0000 7f7ae777f740 1 Processor -- start 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.827+0000 7f7ae777f740 1 -- start start 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0cb1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 0x55eae0ca4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 0x55eae0cb16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813eef0 con 0x55eae0c935f0 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813c7c0 con 0x55eae0c99150 2024-06-13T07:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ae777f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813a630 con 0x55eae0c93fe0 2024-06-13T07:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7adde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0cb1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7adde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0cb1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-06-13T07:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7adde00640 1 -- 172.21.15.7:0/1254860581 learned_addr learned my addr 172.21.15.7:0/1254860581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7ade800640 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 0x55eae0cb16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.829+0000 7f7ade800640 1 -- 172.21.15.7:0/1254860581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 msgr2=0x55eae0ca4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.828+0000 7f7add400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 0x55eae0ca4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7ade800640 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 0x55eae0ca4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7ade800640 1 -- 172.21.15.7:0/1254860581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 msgr2=0x55eae0cb1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7ade800640 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0cb1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7ade800640 1 -- 172.21.15.7:0/1254860581 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae0c943c0 con 0x55eae0c99150 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7add400640 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 0x55eae0ca4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.830+0000 7f7ade800640 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 0x55eae0cb16b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7ae004fa30 tx=0x7f7ae006c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b7867430dbd9b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.831+0000 7f7aca600640 1 -- 172.21.15.7:0/1254860581 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00693e0 con 0x55eae0c99150 2024-06-13T07:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.831+0000 7f7aca600640 1 -- 172.21.15.7:0/1254860581 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0069a60 con 0x55eae0c99150 2024-06-13T07:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.831+0000 7f7aca600640 1 -- 172.21.15.7:0/1254860581 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae007a870 con 0x55eae0c99150 2024-06-13T07:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.832+0000 7f7ae777f740 1 -- 172.21.15.7:0/1254860581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 msgr2=0x55eae0cb16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.832+0000 7f7ae777f740 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 0x55eae0cb16b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7ae004fa30 tx=0x7f7ae006c900 comp rx=0 tx=0).stop 2024-06-13T07:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.834+0000 7f7ae777f740 1 -- 172.21.15.7:0/1254860581 shutdown_connections 2024-06-13T07:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.834+0000 7f7ae777f740 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c99150 0x55eae0cb16b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.834+0000 7f7ae777f740 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c93fe0 0x55eae0ca4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.834+0000 7f7ae777f740 1 --2- 172.21.15.7:0/1254860581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c935f0 0x55eae0cb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.834+0000 7f7ae777f740 1 -- 172.21.15.7:0/1254860581 >> 172.21.15.7:0/1254860581 conn(0x55eae0c52fb0 msgr2=0x55eae0c89db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.835+0000 7f7ae777f740 1 -- 172.21.15.7:0/1254860581 shutdown_connections 2024-06-13T07:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.835+0000 7f7ae777f740 1 -- 172.21.15.7:0/1254860581 wait complete. 2024-06-13T07:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.835+0000 7f7ae777f740 1 -- start start 2024-06-13T07:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.835+0000 7f7ae777f740 1 Processor -- start 2024-06-13T07:06:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.837+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0ca07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0ca37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813ac80 con 0x55eae0c99150 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813c0f0 con 0x55eae0c93fe0 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad813b350 con 0x55eae0c935f0 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7ae777f740 10 client.0 _ll_register_callbacks cb 0x7f7acc0bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7adde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7adde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53008/0 (socket says 172.21.15.7:53008) 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.838+0000 7f7adde00640 1 -- 172.21.15.7:0/225788472 learned_addr learned my addr 172.21.15.7:0/225788472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7adde00640 1 -- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 msgr2=0x55eae0ca07d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7adde00640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0ca07d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7adde00640 1 -- 172.21.15.7:0/225788472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 msgr2=0x55eae0ca37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7ade800640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0ca37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7adde00640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0ca37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.839+0000 7f7adde00640 1 -- 172.21.15.7:0/225788472 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae0ca3d90 con 0x55eae0c935f0 2024-06-13T07:06:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.842+0000 7f7adde00640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00117f0 tx=0x7f7ad0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccae3a70d2481a1b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:06:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.842+0000 7f7abf400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:06:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.842+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad002c070 con 0x55eae0c935f0 2024-06-13T07:06:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.842+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7ad001e830 con 0x55eae0c935f0 2024-06-13T07:06:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.842+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae0069a60 con 0x55eae0c935f0 2024-06-13T07:06:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.843+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7ad0034050 con 0x55eae0c935f0 2024-06-13T07:06:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.843+0000 7f7ae777f740 10 client.26893 fetch_fsmap learned FSMap version 38 2024-06-13T07:06:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.843+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eae0d251b0 con 0x55eae0c935f0 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.844+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0069a60 con 0x55eae0c935f0 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f7ad002f030 con 0x55eae0c935f0 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 10 client.26893 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 20 client.26893 populate_metadata read hostname 'smithi007' 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0d25b10 0x55eae0d27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 4 client.26893 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:06:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eae0bf5ea0 con 0x55eae0d25b10 2024-06-13T07:06:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.845+0000 7f7add400640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0d25b10 0x55eae0d27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:06:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.846+0000 7f7add400640 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0d25b10 0x55eae0d27f30 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266429635a224a4b server_cookie=3556d09857d70f98 in_seq=0 out_seq=0 2024-06-13T07:06:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:51.846+0000 7f7abf400640 10 client.26893 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:51 smithi189 ceph-mon[36616]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 501 op/s 2024-06-13T07:06:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:51 smithi175 ceph-mon[36886]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 501 op/s 2024-06-13T07:06:52.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:51 smithi007 ceph-mon[32077]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 MiB/s wr, 501 op/s 2024-06-13T07:06:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.393+0000 7f7abf400640 1 -- 172.21.15.7:0/225788472 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eae0bf5ea0 con 0x55eae0d25b10 2024-06-13T07:06:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.393+0000 7f7abf400640 10 client.26893 handle_command_reply: tid=0 2024-06-13T07:06:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.393+0000 7f7ae777f740 1 client.26893 shutdown 2024-06-13T07:06:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 msgr2=0x55eae0ca0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00117f0 tx=0x7f7ad0004730 comp rx=0 tx=0).stop 2024-06-13T07:06:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 shutdown_connections 2024-06-13T07:06:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55eae0d25b10 0x55eae0d27f30 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7abf400640 0 client.26893 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae0c99150 0x55eae0ca37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae0c93fe0 0x55eae0ca07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 --2- 172.21.15.7:0/225788472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eae0c935f0 0x55eae0ca0290 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 >> 172.21.15.7:0/225788472 conn(0x55eae0c52fb0 msgr2=0x55eae0c89db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.395+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 shutdown_connections 2024-06-13T07:06:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.396+0000 7f7ae777f740 1 -- 172.21.15.7:0/225788472 wait complete. 2024-06-13T07:06:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.396+0000 7f7ae777f740 20 client.26893 trim_cache size 0 max 16384 2024-06-13T07:06:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:06:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:06:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:06:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 msgr2=0x7f7ac40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 0x7f7ac40a51a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0005fd0 tx=0x7f7ad000b4a0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 msgr2=0x7f7ad812e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 0x7f7ad812e7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009850 tx=0x7f7acc032c80 comp rx=0 tx=0).stop 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 shutdown_connections 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7ac40a2ce0 0x7f7ac40a51a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad8099ab0 0x7f7ad8135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.735+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad80990f0 0x7f7ad812e7a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.736+0000 7f7adf200640 1 --2- 172.21.15.7:0/3268209146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ad8097f20 0x7f7ad812e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.736+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 >> 172.21.15.7:0/3268209146 conn(0x7f7ad80934e0 msgr2=0x7f7ad809c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.736+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 shutdown_connections 2024-06-13T07:06:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:06:52.736+0000 7f7adf200640 1 -- 172.21.15.7:0/3268209146 wait complete. 2024-06-13T07:06:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 650.0 s, 654.4 tps, lat 141.532 ms stddev 168.854 2024-06-13T07:06:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:53 smithi189 ceph-mon[36616]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 480 op/s 2024-06-13T07:06:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:53 smithi175 ceph-mon[36886]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 480 op/s 2024-06-13T07:06:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:53 smithi007 ceph-mon[32077]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 480 op/s 2024-06-13T07:06:56.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:55 smithi189 ceph-mon[36616]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-06-13T07:06:56.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:55 smithi175 ceph-mon[36886]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-06-13T07:06:56.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:55 smithi007 ceph-mon[32077]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2024-06-13T07:06:57.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:56 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:57.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:56 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:57.167 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:56 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:06:57.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 655.0 s, 485.0 tps, lat 209.075 ms stddev 263.929 2024-06-13T07:06:58.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:57 smithi189 ceph-mon[36616]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-06-13T07:06:58.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:57 smithi175 ceph-mon[36886]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-06-13T07:06:58.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:57 smithi007 ceph-mon[32077]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 6.7 MiB/s wr, 475 op/s 2024-06-13T07:06:59.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:06:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:59.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:58 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:06:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:59.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:59.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:58 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:06:59.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:06:59.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:06:59.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:58 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:06:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:00.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:06:59 smithi189 ceph-mon[36616]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-06-13T07:07:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:06:59 smithi175 ceph-mon[36886]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-06-13T07:07:00.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:06:59 smithi007 ceph-mon[32077]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.7 MiB/s wr, 496 op/s 2024-06-13T07:07:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:01 smithi189 ceph-mon[36616]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-06-13T07:07:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:01 smithi175 ceph-mon[36886]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-06-13T07:07:02.159 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:01 smithi007 ceph-mon[32077]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-06-13T07:07:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 660.0 s, 391.6 tps, lat 264.070 ms stddev 333.877 2024-06-13T07:07:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.447+0000 7f6829c00700 1 -- 172.21.15.7:0/3565152622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68241133d0 msgr2=0x7f68241157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.447+0000 7f6829c00700 1 --2- 172.21.15.7:0/3565152622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68241133d0 0x7f68241157c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f681c009f90 tx=0x7f681c00b330 comp rx=0 tx=0).stop 2024-06-13T07:07:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 -- 172.21.15.7:0/3565152622 shutdown_connections 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 --2- 172.21.15.7:0/3565152622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68241133d0 0x7f68241157c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 --2- 172.21.15.7:0/3565152622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824074830 0x7f6824074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 --2- 172.21.15.7:0/3565152622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6824075150 0x7f6824112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 -- 172.21.15.7:0/3565152622 >> 172.21.15.7:0/3565152622 conn(0x7f68240fdf10 msgr2=0x7f6824100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 -- 172.21.15.7:0/3565152622 shutdown_connections 2024-06-13T07:07:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.448+0000 7f6829c00700 1 -- 172.21.15.7:0/3565152622 wait complete. 2024-06-13T07:07:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.450+0000 7f6829c00700 1 Processor -- start 2024-06-13T07:07:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.452+0000 7f6829c00700 1 -- start start 2024-06-13T07:07:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.452+0000 7f6829c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6824074830 0x7f68241128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.452+0000 7f6829c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 0x7f6824112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.452+0000 7f6829c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 0x7f682410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6823400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 0x7f6824112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6823e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6824074830 0x7f68241128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6823400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 0x7f6824112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55628/0 (socket says 172.21.15.7:55628) 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6823400700 1 -- 172.21.15.7:0/2907610277 learned_addr learned my addr 172.21.15.7:0/2907610277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6824118d10 con 0x7f6824074830 2024-06-13T07:07:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6824118b90 con 0x7f6824075150 2024-06-13T07:07:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6824118e90 con 0x7f68241133d0 2024-06-13T07:07:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.453+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 0x7f682410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.454+0000 7f6829200700 1 -- 172.21.15.7:0/2907610277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 msgr2=0x7f6824112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.454+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 0x7f6824112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.454+0000 7f6829200700 1 -- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6824074830 msgr2=0x7f68241128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.454+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6824074830 0x7f68241128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.454+0000 7f6829200700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682410c3f0 con 0x7f68241133d0 2024-06-13T07:07:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.455+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 0x7f682410baf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f681c009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.457+0000 7f6820c00700 1 -- 172.21.15.7:0/2907610277 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c021070 con 0x7f68241133d0 2024-06-13T07:07:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.457+0000 7f6820c00700 1 -- 172.21.15.7:0/2907610277 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681c007b40 con 0x7f68241133d0 2024-06-13T07:07:03.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.457+0000 7f6820c00700 1 -- 172.21.15.7:0/2907610277 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c01b710 con 0x7f68241133d0 2024-06-13T07:07:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.457+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682410c680 con 0x7f68241133d0 2024-06-13T07:07:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.458+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6824110600 con 0x7f68241133d0 2024-06-13T07:07:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.468+0000 7f6820c00700 1 -- 172.21.15.7:0/2907610277 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f681c024070 con 0x7f68241133d0 2024-06-13T07:07:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.469+0000 7f6835a7fb80 1 Processor -- start 2024-06-13T07:07:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.469+0000 7f6835a7fb80 1 -- start start 2024-06-13T07:07:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 0x563639c0eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c11920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563639b13a10 con 0x563639c03d60 2024-06-13T07:07:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563639b56b00 con 0x563639c04680 2024-06-13T07:07:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.470+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563639b118f0 con 0x563639c0f530 2024-06-13T07:07:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.472+0000 7f6820c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 0x7f68040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.471+0000 7f6823400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.472+0000 7f6823400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c04140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43790/0 (socket says 172.21.15.7:43790) 2024-06-13T07:07:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.473+0000 7f6823400700 1 -- 172.21.15.7:0/373152516 learned_addr learned my addr 172.21.15.7:0/373152516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.473+0000 7f6823400700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c11920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.474+0000 7f6820c00700 1 -- 172.21.15.7:0/2907610277 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f681c00f030 con 0x7f68241133d0 2024-06-13T07:07:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.475+0000 7f6823e00700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 0x563639c0eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.475+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 0x7f68040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.477+0000 7f6823e00700 1 -- 172.21.15.7:0/373152516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 msgr2=0x563639c11920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.477+0000 7f6823e00700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c11920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.477+0000 7f6823e00700 1 -- 172.21.15.7:0/373152516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 msgr2=0x563639c04140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.478+0000 7f6823e00700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c04140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.478+0000 7f6823e00700 1 -- 172.21.15.7:0/373152516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563639afef90 con 0x563639c04680 2024-06-13T07:07:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.478+0000 7f6823400700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.479+0000 7f6829200700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 0x7f68040a5160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f681c00b770 tx=0x7f681c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.479+0000 7f6823e00700 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 0x563639c0eff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f680c00bc80 tx=0x7f680c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989e2d6f0efb572d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.479+0000 7f6819600700 1 -- 172.21.15.7:0/373152516 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c010ca0 con 0x563639c04680 2024-06-13T07:07:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.481+0000 7f6819600700 1 -- 172.21.15.7:0/373152516 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c004510 con 0x563639c04680 2024-06-13T07:07:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 -- 172.21.15.7:0/373152516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 msgr2=0x563639c0eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6819600700 1 -- 172.21.15.7:0/373152516 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c005280 con 0x563639c04680 2024-06-13T07:07:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 0x563639c0eff0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f680c00bc80 tx=0x7f680c00bf90 comp rx=0 tx=0).stop 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 -- 172.21.15.7:0/373152516 shutdown_connections 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/373152516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c04680 0x563639c0eff0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.482+0000 7f6835a7fb80 1 -- 172.21.15.7:0/373152516 >> 172.21.15.7:0/373152516 conn(0x563639b31170 msgr2=0x563639c00280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.483+0000 7f6835a7fb80 1 -- 172.21.15.7:0/373152516 shutdown_connections 2024-06-13T07:07:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.483+0000 7f6835a7fb80 1 -- 172.21.15.7:0/373152516 wait complete. 2024-06-13T07:07:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.484+0000 7f6835a7fb80 1 -- start start 2024-06-13T07:07:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.485+0000 7f6835a7fb80 1 Processor -- start 2024-06-13T07:07:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.485+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.485+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.485+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.486+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563639af2f60 con 0x563639c04680 2024-06-13T07:07:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.486+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563639b52ac0 con 0x563639c03d60 2024-06-13T07:07:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.486+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563639c12020 con 0x563639c0f530 2024-06-13T07:07:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.486+0000 7f6835a7fb80 10 client.0 _ll_register_callbacks cb 0x563639aec380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.487+0000 7f6823e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.488+0000 7f6823e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43806/0 (socket says 172.21.15.7:43806) 2024-06-13T07:07:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.488+0000 7f6823e00700 1 -- 172.21.15.7:0/3369094005 learned_addr learned my addr 172.21.15.7:0/3369094005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823400700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823400700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823e00700 1 -- 172.21.15.7:0/3369094005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 msgr2=0x563639c0c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823e00700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823e00700 1 -- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 msgr2=0x563639c0eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823e00700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823e00700 1 -- 172.21.15.7:0/3369094005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563639aa58d0 con 0x563639c04680 2024-06-13T07:07:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.489+0000 7f6823400700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.490+0000 7f6823e00700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f680c00b2b0 tx=0x7f680c010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8291fb87f1eb5d24 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.490+0000 7f6812000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:07:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.491+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c02a060 con 0x563639c04680 2024-06-13T07:07:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.491+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f680c004d10 con 0x563639c04680 2024-06-13T07:07:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.492+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c004510 con 0x563639c04680 2024-06-13T07:07:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.493+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f680c004f10 con 0x563639c04680 2024-06-13T07:07:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.493+0000 7f6835a7fb80 10 client.16311 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.493+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563639c2d4f0 con 0x563639c04680 2024-06-13T07:07:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.495+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c004510 con 0x563639c04680 2024-06-13T07:07:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f680c011960 con 0x563639c04680 2024-06-13T07:07:03.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 10 client.16311 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 20 client.16311 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c2f9c0 0x563639c31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 4 client.16311 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563639b64000 con 0x563639c2f9c0 2024-06-13T07:07:03.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.497+0000 7f6823400700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c2f9c0 0x563639c31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.498+0000 7f6823400700 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c2f9c0 0x563639c31db0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30670272b408018b server_cookie=a0c61ab158eefc44 in_seq=0 out_seq=0 2024-06-13T07:07:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.498+0000 7f6812000700 10 client.16311 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.499+0000 7f6812000700 1 -- 172.21.15.7:0/3369094005 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563639b64000 con 0x563639c2f9c0 2024-06-13T07:07:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.499+0000 7f6812000700 10 client.16311 handle_command_reply: tid=0 2024-06-13T07:07:03.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.499+0000 7f6835a7fb80 1 client.16311 shutdown 2024-06-13T07:07:03.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 msgr2=0x563639c0bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f680c00b2b0 tx=0x7f680c010aa0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 shutdown_connections 2024-06-13T07:07:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c2f9c0 0x563639c31db0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3369094005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c04680 0x563639c0bea0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 >> 172.21.15.7:0/3369094005 conn(0x563639b31170 msgr2=0x563639bffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6812000700 0 client.16311 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 shutdown_connections 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.501+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3369094005 wait complete. 2024-06-13T07:07:03.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.502+0000 7f6835a7fb80 20 client.16311 trim_cache size 0 max 16384 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.552+0000 7f6835a7fb80 1 Processor -- start 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.552+0000 7f6835a7fb80 1 -- start start 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 0x563639c1a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6800011290 con 0x563639c0ec20 2024-06-13T07:07:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f680000eb20 con 0x563639c03d60 2024-06-13T07:07:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f680000c930 con 0x563639c0f530 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6823e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6823e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52936/0 (socket says 172.21.15.7:52936) 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.553+0000 7f6823e00700 1 -- 172.21.15.7:0/3790977421 learned_addr learned my addr 172.21.15.7:0/3790977421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823e00700 1 -- 172.21.15.7:0/3790977421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 msgr2=0x563639c0c410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823e00700 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823e00700 1 -- 172.21.15.7:0/3790977421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 msgr2=0x563639c1a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823400700 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 0x563639c1a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823e00700 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 0x563639c1a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823e00700 1 -- 172.21.15.7:0/3790977421 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563639c0a410 con 0x563639c0f530 2024-06-13T07:07:03.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.554+0000 7f6823400700 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 0x563639c1a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.555+0000 7f6823e00700 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f680c024ca0 tx=0x7f680c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819cfa28cb6766f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.556+0000 7f6818c00700 1 -- 172.21.15.7:0/3790977421 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c034080 con 0x563639c0f530 2024-06-13T07:07:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.556+0000 7f6818c00700 1 -- 172.21.15.7:0/3790977421 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c023e60 con 0x563639c0f530 2024-06-13T07:07:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.556+0000 7f6818c00700 1 -- 172.21.15.7:0/3790977421 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c004f10 con 0x563639c0f530 2024-06-13T07:07:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.557+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3790977421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 msgr2=0x563639c0e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.557+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f680c024ca0 tx=0x7f680c000bc0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.558+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3790977421 shutdown_connections 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.558+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c0ec20 0x563639c1a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.558+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c03d60 0x563639c0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.558+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/3790977421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c0e6e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.558+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3790977421 >> 172.21.15.7:0/3790977421 conn(0x563639b31170 msgr2=0x563639c1a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.559+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3790977421 shutdown_connections 2024-06-13T07:07:03.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.559+0000 7f6835a7fb80 1 -- 172.21.15.7:0/3790977421 wait complete. 2024-06-13T07:07:03.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.560+0000 7f6835a7fb80 1 -- start start 2024-06-13T07:07:03.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.560+0000 7f6835a7fb80 1 Processor -- start 2024-06-13T07:07:03.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 0x563639c17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f680000cfe0 con 0x563639c03d60 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f680000e450 con 0x563639c0ec20 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f680000d6b0 con 0x563639c0f530 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.561+0000 7f6835a7fb80 10 client.0 _ll_register_callbacks cb 0x563639aec380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.562+0000 7f6823e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 0x563639c17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.562+0000 7f6823400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.562+0000 7f6823400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c12320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52946/0 (socket says 172.21.15.7:52946) 2024-06-13T07:07:03.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.562+0000 7f6823400700 1 -- 172.21.15.7:0/729207858 learned_addr learned my addr 172.21.15.7:0/729207858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.563+0000 7f6823400700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.563+0000 7f6823e00700 1 -- 172.21.15.7:0/729207858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 msgr2=0x563639c12320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.563+0000 7f6823e00700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c12320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.563+0000 7f6823e00700 1 -- 172.21.15.7:0/729207858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 msgr2=0x563639c17550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.566+0000 7f6823e00700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c17550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.566+0000 7f6823e00700 1 -- 172.21.15.7:0/729207858 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563639c12a20 con 0x563639c0ec20 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.566+0000 7f6823400700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c17550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.567+0000 7f6823e00700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 0x563639c17a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f680c024ca0 tx=0x7f680c005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363795099d55ac28 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.567+0000 7f6812000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.567+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c03c070 con 0x563639c0ec20 2024-06-13T07:07:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.567+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f680c023310 con 0x563639c0ec20 2024-06-13T07:07:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.569+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c023e60 con 0x563639c0ec20 2024-06-13T07:07:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.569+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f680c034070 con 0x563639c0ec20 2024-06-13T07:07:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.569+0000 7f6835a7fb80 10 client.26993 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.569+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563639c9ad00 con 0x563639c0ec20 2024-06-13T07:07:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.570+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c023e60 con 0x563639c0ec20 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.570+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f680c03f030 con 0x563639c0ec20 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.570+0000 7f6835a7fb80 10 client.26993 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.570+0000 7f6835a7fb80 10 client.26993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.571+0000 7f6835a7fb80 20 client.26993 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.571+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c9b150 0x563639c9d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.571+0000 7f6835a7fb80 4 client.26993 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:07:03.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.571+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563639b5e580 con 0x563639c9b150 2024-06-13T07:07:03.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.572+0000 7f6823400700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c9b150 0x563639c9d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:03.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.574+0000 7f6823400700 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c9b150 0x563639c9d540 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac1549daa30b35d server_cookie=47fda36d19f354f3 in_seq=0 out_seq=0 2024-06-13T07:07:03.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.574+0000 7f6812000700 10 client.26993 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:03.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.575+0000 7f6812000700 1 -- 172.21.15.7:0/729207858 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563639b5e580 con 0x563639c9b150 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.575+0000 7f6812000700 10 client.26993 handle_command_reply: tid=0 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.575+0000 7f6835a7fb80 1 client.26993 shutdown 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.575+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 msgr2=0x563639c17a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.575+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 0x563639c17a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f680c024ca0 tx=0x7f680c005ce0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 shutdown_connections 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563639c9b150 0x563639c9d540 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563639c0f530 0x563639c12320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563639c03d60 0x563639c17550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 --2- 172.21.15.7:0/729207858 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563639c0ec20 0x563639c17a90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6812000700 0 client.26993 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:03.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 >> 172.21.15.7:0/729207858 conn(0x563639b31170 msgr2=0x563639c1aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 shutdown_connections 2024-06-13T07:07:03.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.576+0000 7f6835a7fb80 1 -- 172.21.15.7:0/729207858 wait complete. 2024-06-13T07:07:03.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.579+0000 7f6835a7fb80 20 client.26993 trim_cache size 0 max 16384 2024-06-13T07:07:03.582 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:07:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:07:03.582 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:07:03.582 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:07:03.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.583+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 msgr2=0x7f68040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.583+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 0x7f68040a5160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f681c00b770 tx=0x7f681c009040 comp rx=0 tx=0).stop 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.583+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 msgr2=0x7f682410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.583+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 0x7f682410baf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f681c009460 comp rx=0 tx=0).stop 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 shutdown_connections 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68241133d0 0x7f682410baf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6824074830 0x7f68241128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68040a2ca0 0x7f68040a5160 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 --2- 172.21.15.7:0/2907610277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6824075150 0x7f6824112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 >> 172.21.15.7:0/2907610277 conn(0x7f68240fdf10 msgr2=0x7f68240feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 shutdown_connections 2024-06-13T07:07:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:03.584+0000 7f6829c00700 1 -- 172.21.15.7:0/2907610277 wait complete. 2024-06-13T07:07:04.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:03 smithi189 ceph-mon[36616]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.1 MiB/s wr, 514 op/s 2024-06-13T07:07:04.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:03 smithi175 ceph-mon[36886]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.1 MiB/s wr, 514 op/s 2024-06-13T07:07:04.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:03 smithi007 ceph-mon[32077]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.1 MiB/s wr, 514 op/s 2024-06-13T07:07:05.450 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:07:05.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.992+0000 7f268e000640 1 -- 172.21.15.7:0/854814208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 msgr2=0x7f2688110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.992+0000 7f268e000640 1 --2- 172.21.15.7:0/854814208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688110570 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2680005fd0 tx=0x7f268002b3e0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.992+0000 7f268e000640 1 -- 172.21.15.7:0/854814208 shutdown_connections 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 --2- 172.21.15.7:0/854814208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688110570 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 --2- 172.21.15.7:0/854814208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f268810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 --2- 172.21.15.7:0/854814208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f268810b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 -- 172.21.15.7:0/854814208 >> 172.21.15.7:0/854814208 conn(0x7f268806ea10 msgr2=0x7f2688070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 -- 172.21.15.7:0/854814208 shutdown_connections 2024-06-13T07:07:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.993+0000 7f268e000640 1 -- 172.21.15.7:0/854814208 wait complete. 2024-06-13T07:07:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 Processor -- start 2024-06-13T07:07:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 -- start start 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f26881135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688112170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2688115f40 con 0x7f268810b5d0 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.995+0000 7f268e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2688115dc0 con 0x7f2688108c10 2024-06-13T07:07:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f268e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26881160c0 con 0x7f268810e0b0 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f2686a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f2686a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43842/0 (socket says 172.21.15.7:43842) 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f2686a00640 1 -- 172.21.15.7:0/2641110600 learned_addr learned my addr 172.21.15.7:0/2641110600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f2687400640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f26881135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.996+0000 7f2687e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 -- 172.21.15.7:0/2641110600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 msgr2=0x7f2688112170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688112170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 -- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 msgr2=0x7f2688111c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 -- 172.21.15.7:0/2641110600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2688112a30 con 0x7f2688108c10 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2686a00640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f2687400640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f26881135a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9c0 tx=0x7f267800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f267fe00640 1 -- 172.21.15.7:0/2641110600 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267800ccc0 con 0x7f2688108c10 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.997+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26881aa610 con 0x7f2688108c10 2024-06-13T07:07:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.998+0000 7f267fe00640 1 -- 172.21.15.7:0/2641110600 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2678005020 con 0x7f2688108c10 2024-06-13T07:07:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.998+0000 7f267fe00640 1 -- 172.21.15.7:0/2641110600 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678004a00 con 0x7f2688108c10 2024-06-13T07:07:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:05.998+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881aabb0 con 0x7f2688108c10 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.003+0000 7f2690e1e740 1 Processor -- start 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.003+0000 7f267fe00640 1 -- 172.21.15.7:0/2641110600 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2678005560 con 0x7f2688108c10 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f267fe00640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 0x7f266c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f267fe00640 1 -- 172.21.15.7:0/2641110600 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f26780cf380 con 0x7f2688108c10 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 -- start start 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2687e00640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 0x7f266c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8ac00 0x55c838a8b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c8389ee420 con 0x55c838a8ac00 2024-06-13T07:07:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c838995c50 con 0x55c838a8b5f0 2024-06-13T07:07:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.005+0000 7f2690e1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8389ea3e0 con 0x55c838a90760 2024-06-13T07:07:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.006+0000 7f2687400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8ac00 0x55c838a8b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.006+0000 7f2687e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.006+0000 7f2687e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52974/0 (socket says 172.21.15.7:52974) 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.006+0000 7f2687e00640 1 -- 172.21.15.7:0/592092919 learned_addr learned my addr 172.21.15.7:0/592092919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 0x7f266c0a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f268002b900 tx=0x7f2680032400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 -- 172.21.15.7:0/592092919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 msgr2=0x55c838a900f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a900f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 -- 172.21.15.7:0/592092919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8ac00 msgr2=0x55c838a8b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8ac00 0x55c838a8b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 -- 172.21.15.7:0/592092919 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c838702210 con 0x55c838a90760 2024-06-13T07:07:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.007+0000 7f2687e00640 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f268000afd0 tx=0x7f2680041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4e334cdedf0e25 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.008+0000 7f2673e00640 1 -- 172.21.15.7:0/592092919 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680043040 con 0x55c838a90760 2024-06-13T07:07:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.009+0000 7f2673e00640 1 -- 172.21.15.7:0/592092919 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2680032c10 con 0x55c838a90760 2024-06-13T07:07:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.009+0000 7f2673e00640 1 -- 172.21.15.7:0/592092919 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680037040 con 0x55c838a90760 2024-06-13T07:07:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 -- 172.21.15.7:0/592092919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 msgr2=0x55c838a92bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f268000afd0 tx=0x7f2680041000 comp rx=0 tx=0).stop 2024-06-13T07:07:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 -- 172.21.15.7:0/592092919 shutdown_connections 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a92bf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a900f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 --2- 172.21.15.7:0/592092919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8ac00 0x55c838a8b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.010+0000 7f2690e1e740 1 -- 172.21.15.7:0/592092919 >> 172.21.15.7:0/592092919 conn(0x55c838a52fb0 msgr2=0x55c838a86680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.011+0000 7f2690e1e740 1 -- 172.21.15.7:0/592092919 shutdown_connections 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.011+0000 7f2690e1e740 1 -- 172.21.15.7:0/592092919 wait complete. 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.011+0000 7f2690e1e740 1 -- start start 2024-06-13T07:07:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.012+0000 7f2690e1e740 1 Processor -- start 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.013+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 0x55c838a94000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a90760 0x55c838a9b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c838a936b0 con 0x55c838a90760 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c8389a90b0 con 0x55c838a8b5f0 2024-06-13T07:07:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c838a93830 con 0x55c838a8ac00 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2690e1e740 10 client.0 _ll_register_callbacks cb 0x55c838994f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2686a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2687400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 0x55c838a94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2686a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55694/0 (socket says 172.21.15.7:55694) 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2686a00640 1 -- 172.21.15.7:0/2832921918 learned_addr learned my addr 172.21.15.7:0/2832921918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2687400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 0x55c838a94000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52982/0 (socket says 172.21.15.7:52982) 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2687e00640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a90760 0x55c838a9b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2686a00640 1 -- 172.21.15.7:0/2832921918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 msgr2=0x55c838a94000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.014+0000 7f2686a00640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 0x55c838a94000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2686a00640 1 -- 172.21.15.7:0/2832921918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a90760 msgr2=0x55c838a9b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2686a00640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a90760 0x55c838a9b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2686a00640 1 -- 172.21.15.7:0/2832921918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8387ce620 con 0x55c838a8b5f0 2024-06-13T07:07:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2686a00640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f267400e7f0 tx=0x7f267400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9b17c4a0012d9e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2670c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.015+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x55c838a8b5f0 2024-06-13T07:07:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.016+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f267400c6b0 con 0x55c838a8b5f0 2024-06-13T07:07:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.017+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2680032c10 con 0x55c838a8b5f0 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.017+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2674021050 con 0x55c838a8b5f0 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.017+0000 7f2690e1e740 10 client.27002 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.017+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c838a9ece0 con 0x55c838a8b5f0 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.018+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2680032c10 con 0x55c838a8b5f0 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.018+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f2674013be0 con 0x55c838a8b5f0 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 10 client.27002 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 20 client.27002 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838aa04c0 0x55c838aa2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 4 client.27002 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.019+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8389fbb50 con 0x55c838aa04c0 2024-06-13T07:07:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.021+0000 7f2687400640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838aa04c0 0x55c838aa2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.021+0000 7f2687400640 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838aa04c0 0x55c838aa2900 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65becc950bd912ae server_cookie=5a8ca57e2558f58c in_seq=0 out_seq=0 2024-06-13T07:07:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.021+0000 7f2670c00640 10 client.27002 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.022+0000 7f2670c00640 1 -- 172.21.15.7:0/2832921918 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c8389fbb50 con 0x55c838aa04c0 2024-06-13T07:07:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.022+0000 7f2670c00640 10 client.27002 handle_command_reply: tid=0 2024-06-13T07:07:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.022+0000 7f2690e1e740 1 client.27002 shutdown 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 msgr2=0x55c838a94540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f267400e7f0 tx=0x7f267400ecb0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 shutdown_connections 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838aa04c0 0x55c838aa2900 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a90760 0x55c838a9b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8b5f0 0x55c838a94540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 --2- 172.21.15.7:0/2832921918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a8ac00 0x55c838a94000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 >> 172.21.15.7:0/2832921918 conn(0x55c838a52fb0 msgr2=0x55c838a85840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2670c00640 0 client.27002 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.023+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 shutdown_connections 2024-06-13T07:07:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.031+0000 7f2690e1e740 1 -- 172.21.15.7:0/2832921918 wait complete. 2024-06-13T07:07:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.031+0000 7f2690e1e740 20 client.27002 trim_cache size 0 max 16384 2024-06-13T07:07:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.064+0000 7f2690e1e740 1 Processor -- start 2024-06-13T07:07:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 -- start start 2024-06-13T07:07:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a9b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b3330 con 0x55c838a8b5f0 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b0c00 con 0x55c838a8ac00 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2690e1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26881aea50 con 0x55c838a90760 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.067+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55708/0 (socket says 172.21.15.7:55708) 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 -- 172.21.15.7:0/723585026 learned_addr learned my addr 172.21.15.7:0/723585026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2686a00640 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a9b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 -- 172.21.15.7:0/723585026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 msgr2=0x55c838a9c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 -- 172.21.15.7:0/723585026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 msgr2=0x55c838a9b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a9b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.068+0000 7f2687400640 1 -- 172.21.15.7:0/723585026 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c838a87330 con 0x55c838a8ac00 2024-06-13T07:07:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.071+0000 7f2687400640 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f267800c6a0 tx=0x7f26780ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1d08efd1fae708 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.071+0000 7f2673400640 1 -- 172.21.15.7:0/723585026 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267801c040 con 0x55c838a8ac00 2024-06-13T07:07:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.071+0000 7f2673400640 1 -- 172.21.15.7:0/723585026 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26780cf120 con 0x55c838a8ac00 2024-06-13T07:07:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.071+0000 7f2673400640 1 -- 172.21.15.7:0/723585026 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678097eb0 con 0x55c838a8ac00 2024-06-13T07:07:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 -- 172.21.15.7:0/723585026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 msgr2=0x55c838a9bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f267800c6a0 tx=0x7f26780ce040 comp rx=0 tx=0).stop 2024-06-13T07:07:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 -- 172.21.15.7:0/723585026 shutdown_connections 2024-06-13T07:07:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a9b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 --2- 172.21.15.7:0/723585026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a9bfd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.072+0000 7f2690e1e740 1 -- 172.21.15.7:0/723585026 >> 172.21.15.7:0/723585026 conn(0x55c838a52fb0 msgr2=0x55c838a8c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.074+0000 7f2690e1e740 1 -- 172.21.15.7:0/723585026 shutdown_connections 2024-06-13T07:07:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.074+0000 7f2690e1e740 1 -- 172.21.15.7:0/723585026 wait complete. 2024-06-13T07:07:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.075+0000 7f2690e1e740 1 -- start start 2024-06-13T07:07:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.075+0000 7f2690e1e740 1 Processor -- start 2024-06-13T07:07:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.076+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a99010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a99550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a99010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2687400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a99010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55718/0 (socket says 172.21.15.7:55718) 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2687400640 1 -- 172.21.15.7:0/3091121323 learned_addr learned my addr 172.21.15.7:0/3091121323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26881af0c0 con 0x55c838a8b5f0 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b0530 con 0x55c838a8ac00 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26881af790 con 0x55c838a90760 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2687e00640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.077+0000 7f2690e1e740 10 client.0 _ll_register_callbacks cb 0x7f26740097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2686a00640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a99550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2687e00640 1 -- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 msgr2=0x55c838a99010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2687e00640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a99010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2687e00640 1 -- 172.21.15.7:0/3091121323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 msgr2=0x55c838a99550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2687e00640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a99550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.078+0000 7f2687e00640 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c838a9ab00 con 0x55c838a90760 2024-06-13T07:07:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.079+0000 7f2687e00640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9a510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2680032780 tx=0x7f2680004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1589dc2c4584c21c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.079+0000 7f2670c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.081+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680054070 con 0x55c838a90760 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.081+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f26800324c0 con 0x55c838a90760 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.081+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26780cf120 con 0x55c838a90760 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.084+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f268005f050 con 0x55c838a90760 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.084+0000 7f2690e1e740 10 client.26926 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.084+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c838b2d3f0 con 0x55c838a90760 2024-06-13T07:07:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:05 smithi189 ceph-mon[36616]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 591 op/s 2024-06-13T07:07:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26780cf120 con 0x55c838a90760 2024-06-13T07:07:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f2680046050 con 0x55c838a90760 2024-06-13T07:07:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 10 client.26926 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 10 client.26926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 20 client.26926 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838b2d940 0x55c838b2fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 4 client.26926 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.088+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8389f5ea0 con 0x55c838b2d940 2024-06-13T07:07:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.089+0000 7f2687400640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838b2d940 0x55c838b2fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.089+0000 7f2687400640 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838b2d940 0x55c838b2fd30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8892906c7dc2ff server_cookie=3b18b686ba8c980 in_seq=0 out_seq=0 2024-06-13T07:07:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.089+0000 7f2670c00640 10 client.26926 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:06.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:05 smithi175 ceph-mon[36886]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 591 op/s 2024-06-13T07:07:06.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:05 smithi007 ceph-mon[32077]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 591 op/s 2024-06-13T07:07:06.337 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.374+0000 7f2670c00640 1 -- 172.21.15.7:0/3091121323 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c8389f5ea0 con 0x55c838b2d940 2024-06-13T07:07:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.374+0000 7f2670c00640 10 client.26926 handle_command_reply: tid=0 2024-06-13T07:07:06.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.374+0000 7f2690e1e740 1 client.26926 shutdown 2024-06-13T07:07:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 msgr2=0x55c838a9a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9a510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2680032780 tx=0x7f2680004480 comp rx=0 tx=0).stop 2024-06-13T07:07:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 shutdown_connections 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c838b2d940 0x55c838b2fd30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c838a90760 0x55c838a9a510 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c838a8b5f0 0x55c838a99550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 --2- 172.21.15.7:0/3091121323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c838a8ac00 0x55c838a99010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2670c00640 0 client.26926 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.378+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 >> 172.21.15.7:0/3091121323 conn(0x55c838a52fb0 msgr2=0x55c838a9bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.379+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 shutdown_connections 2024-06-13T07:07:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.379+0000 7f2690e1e740 1 -- 172.21.15.7:0/3091121323 wait complete. 2024-06-13T07:07:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.379+0000 7f2690e1e740 20 client.26926 trim_cache size 0 max 16384 2024-06-13T07:07:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:07:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:07:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:07:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 msgr2=0x7f266c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 0x7f266c0a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f268002b900 tx=0x7f2680032400 comp rx=0 tx=0).stop 2024-06-13T07:07:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 msgr2=0x7f26881135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f26881135a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f267800e9c0 tx=0x7f267800ee80 comp rx=0 tx=0).stop 2024-06-13T07:07:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 shutdown_connections 2024-06-13T07:07:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f266c0a2cb0 0x7f266c0a5170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268810e0b0 0x7f2688112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268810b5d0 0x7f2688111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.381+0000 7f268e000640 1 --2- 172.21.15.7:0/2641110600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2688108c10 0x7f26881135a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.382+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 >> 172.21.15.7:0/2641110600 conn(0x7f268806ea10 msgr2=0x7f268810ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.382+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 shutdown_connections 2024-06-13T07:07:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:06.382+0000 7f268e000640 1 -- 172.21.15.7:0/2641110600 wait complete. 2024-06-13T07:07:07.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 665.0 s, 565.6 tps, lat 177.052 ms stddev 214.682 2024-06-13T07:07:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:07 smithi175 ceph-mon[36886]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:07:08.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:07 smithi007 ceph-mon[32077]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:07:08.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:07 smithi189 ceph-mon[36616]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2024-06-13T07:07:10.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:09 smithi175 ceph-mon[36886]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-06-13T07:07:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 -- 172.21.15.7:0/585900929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 msgr2=0x7f54100a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 --2- 172.21.15.7:0/585900929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f54100a2be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5418068f20 tx=0x7f54180512d0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 -- 172.21.15.7:0/585900929 shutdown_connections 2024-06-13T07:07:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 --2- 172.21.15.7:0/585900929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f54100a2be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 --2- 172.21.15.7:0/585900929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 0x7f541009c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 --2- 172.21.15.7:0/585900929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f5410005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.122+0000 7f5417e00700 1 -- 172.21.15.7:0/585900929 >> 172.21.15.7:0/585900929 conn(0x7f541000a7b0 msgr2=0x7f541000abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.123+0000 7f5417e00700 1 -- 172.21.15.7:0/585900929 shutdown_connections 2024-06-13T07:07:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.123+0000 7f5417e00700 1 -- 172.21.15.7:0/585900929 wait complete. 2024-06-13T07:07:10.124 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:09 smithi007 ceph-mon[32077]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-06-13T07:07:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.124+0000 7f5417e00700 1 Processor -- start 2024-06-13T07:07:10.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.125+0000 7f5417e00700 1 -- start start 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.126+0000 7f5417e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.126+0000 7f5417e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f541012e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5417e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 0x7f5410132cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5417e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54100a6010 con 0x7f541009be50 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5417e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54100a5e90 con 0x7f5410004d10 2024-06-13T07:07:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5417e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54100a6190 con 0x7f54100056d0 2024-06-13T07:07:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5416a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5416a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:55750/0 (socket says 172.21.15.7:55750) 2024-06-13T07:07:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5416a00700 1 -- 172.21.15.7:0/1453239377 learned_addr learned my addr 172.21.15.7:0/1453239377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5416000700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f541012e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.127+0000 7f5417400700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 0x7f5410132cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 -- 172.21.15.7:0/1453239377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 msgr2=0x7f541012e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f541012e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 -- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 msgr2=0x7f5410132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 0x7f5410132cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 -- 172.21.15.7:0/1453239377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5410133410 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f5416a00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f540000ec10 tx=0x7f540000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540000cd40 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400004500 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.128+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400010690 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.129+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5410133700 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.129+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541013af80 con 0x7f5410004d10 2024-06-13T07:07:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.130+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54100048c0 con 0x7f5410004d10 2024-06-13T07:07:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.134+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5400003f10 con 0x7f5410004d10 2024-06-13T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.140+0000 7f541ca00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 0x7f53fc0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.140+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f54000cf720 con 0x7f5410004d10 2024-06-13T07:07:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.141+0000 7f5416000700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 0x7f53fc0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.141+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f540009c2e0 con 0x7f5410004d10 2024-06-13T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.143+0000 7f5416000700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 0x7f53fc0a51e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54100932c0 tx=0x7f5408009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:10.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:09 smithi189 ceph-mon[36616]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-06-13T07:07:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.626+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f541013b210 con 0x7f5410004d10 2024-06-13T07:07:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.628+0000 7f541ca00700 1 -- 172.21.15.7:0/1453239377 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11868 (secure 0 0 0) 0x7f540009bbe0 con 0x7f5410004d10 2024-06-13T07:07:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:07:10.631 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:04:39.673793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T07:07:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 msgr2=0x7f53fc0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 0x7f53fc0a51e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54100932c0 tx=0x7f5408009190 comp rx=0 tx=0).stop 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 msgr2=0x7f541012e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f540000ec10 tx=0x7f540000ef20 comp rx=0 tx=0).stop 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 shutdown_connections 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541009be50 0x7f5410132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f53fc0a2d20 0x7f53fc0a51e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5410004d10 0x7f541012e400 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.632+0000 7f5417e00700 1 --2- 172.21.15.7:0/1453239377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54100056d0 0x7f541012e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.633+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 >> 172.21.15.7:0/1453239377 conn(0x7f541000a7b0 msgr2=0x7f54100a1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.633+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 shutdown_connections 2024-06-13T07:07:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:10.633+0000 7f5417e00700 1 -- 172.21.15.7:0/1453239377 wait complete. 2024-06-13T07:07:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-06-13T07:07:10.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:10 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1453239377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:07:11.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:10 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1453239377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:07:11.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:10 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1453239377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:07:11.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:07:12.166 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:11 smithi007 ceph-mon[32077]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-06-13T07:07:12.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:11 smithi189 ceph-mon[36616]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-06-13T07:07:12.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:11 smithi175 ceph-mon[36886]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 MiB/s wr, 597 op/s 2024-06-13T07:07:12.637 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 670.0 s, 447.0 tps, lat 220.534 ms stddev 239.726 2024-06-13T07:07:13.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:12 smithi007 ceph-mon[32077]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-06-13T07:07:13.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:12 smithi189 ceph-mon[36616]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-06-13T07:07:13.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:12 smithi175 ceph-mon[36886]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.997+0000 7f9fef800700 1 -- 172.21.15.7:0/2670928507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe810eac0 msgr2=0x7f9fe810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.997+0000 7f9fef800700 1 --2- 172.21.15.7:0/2670928507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe810eac0 0x7f9fe810eea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b0a0 tx=0x7f9fe000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.997+0000 7f9fef800700 1 -- 172.21.15.7:0/2670928507 shutdown_connections 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 --2- 172.21.15.7:0/2670928507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8109480 0x7f9fe8115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 --2- 172.21.15.7:0/2670928507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe810eac0 0x7f9fe810eea0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 --2- 172.21.15.7:0/2670928507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe8108ac0 0x7f9fe8108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 -- 172.21.15.7:0/2670928507 >> 172.21.15.7:0/2670928507 conn(0x7f9fe8075c60 msgr2=0x7f9fe8078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 -- 172.21.15.7:0/2670928507 shutdown_connections 2024-06-13T07:07:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:14.998+0000 7f9fef800700 1 -- 172.21.15.7:0/2670928507 wait complete. 2024-06-13T07:07:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.000+0000 7f9fef800700 1 Processor -- start 2024-06-13T07:07:15.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.001+0000 7f9fef800700 1 -- start start 2024-06-13T07:07:15.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe8109480 0x7f9fe819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 0x7f9fe81a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8118e60 con 0x7f9fe8108ac0 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8118ce0 con 0x7f9fe8109480 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.002+0000 7f9fef800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8118fe0 con 0x7f9fe810eac0 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.003+0000 7f9fece00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.003+0000 7f9fece00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34660/0 (socket says 172.21.15.7:34660) 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.003+0000 7f9fece00700 1 -- 172.21.15.7:0/1674984680 learned_addr learned my addr 172.21.15.7:0/1674984680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.003+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe8109480 0x7f9fe819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.003+0000 7f9fed800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 0x7f9fe81a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fed800700 1 -- 172.21.15.7:0/1674984680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe8109480 msgr2=0x7f9fe819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fed800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe8109480 0x7f9fe819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fed800700 1 -- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 msgr2=0x7f9fe819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fed800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fed800700 1 -- 172.21.15.7:0/1674984680 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81a4350 con 0x7f9fe810eac0 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.004+0000 7f9fece00700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.005+0000 7f9fed800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 0x7f9fe81a3c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e900 tx=0x7f9fd800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.006+0000 7f9fe5600700 1 -- 172.21.15.7:0/1674984680 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8004220 con 0x7f9fe810eac0 2024-06-13T07:07:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.006+0000 7f9fe5600700 1 -- 172.21.15.7:0/1674984680 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd80043c0 con 0x7f9fe810eac0 2024-06-13T07:07:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.006+0000 7f9fe5600700 1 -- 172.21.15.7:0/1674984680 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8004220 con 0x7f9fe810eac0 2024-06-13T07:07:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.007+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81a4640 con 0x7f9fe810eac0 2024-06-13T07:07:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.007+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe807ace0 con 0x7f9fe810eac0 2024-06-13T07:07:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.014+0000 7f9fe5600700 1 -- 172.21.15.7:0/1674984680 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9fd8003c70 con 0x7f9fe810eac0 2024-06-13T07:07:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.016+0000 7f9fe5600700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 0x7f9fcc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.017+0000 7f9fe5600700 1 -- 172.21.15.7:0/1674984680 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fd80cf010 con 0x7f9fe810eac0 2024-06-13T07:07:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.017+0000 7f9fece00700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 0x7f9fcc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.018+0000 7f9fece00700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 0x7f9fcc0a5360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b780 tx=0x7f9fe0009c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.023+0000 7f9ffa1fbb80 1 Processor -- start 2024-06-13T07:07:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.023+0000 7f9ffa1fbb80 1 -- start start 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd658c3a10 con 0x55fd659bb4b0 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd65906b00 con 0x55fd659b5240 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd658c18f0 con 0x55fd659bab90 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9fece00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9fece00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54326/0 (socket says 172.21.15.7:54326) 2024-06-13T07:07:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.024+0000 7f9fece00700 1 -- 172.21.15.7:0/2801537063 learned_addr learned my addr 172.21.15.7:0/2801537063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.025+0000 7f9fed800700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.025+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.026+0000 7f9fece00700 1 -- 172.21.15.7:0/2801537063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 msgr2=0x55fd659b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.026+0000 7f9fece00700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.026+0000 7f9fece00700 1 -- 172.21.15.7:0/2801537063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 msgr2=0x55fd659b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.026+0000 7f9fece00700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.026+0000 7f9fece00700 1 -- 172.21.15.7:0/2801537063 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd658aef90 con 0x55fd659bab90 2024-06-13T07:07:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.027+0000 7f9fece00700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004d10 tx=0x7f9fe002e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=712515732eb3c88b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.027+0000 7f9fed800700 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.027+0000 7f9fde000700 1 -- 172.21.15.7:0/2801537063 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0032070 con 0x55fd659bab90 2024-06-13T07:07:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.027+0000 7f9fde000700 1 -- 172.21.15.7:0/2801537063 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe0009440 con 0x55fd659bab90 2024-06-13T07:07:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.027+0000 7f9fde000700 1 -- 172.21.15.7:0/2801537063 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0005040 con 0x55fd659bab90 2024-06-13T07:07:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.029+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/2801537063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 msgr2=0x55fd659baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.029+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0004d10 tx=0x7f9fe002e000 comp rx=0 tx=0).stop 2024-06-13T07:07:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.030+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/2801537063 shutdown_connections 2024-06-13T07:07:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.030+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.030+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659baf70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.030+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/2801537063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.030+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/2801537063 >> 172.21.15.7:0/2801537063 conn(0x55fd658e1170 msgr2=0x55fd659afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.031+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/2801537063 shutdown_connections 2024-06-13T07:07:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.031+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/2801537063 wait complete. 2024-06-13T07:07:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.032+0000 7f9ffa1fbb80 1 -- start start 2024-06-13T07:07:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.032+0000 7f9ffa1fbb80 1 Processor -- start 2024-06-13T07:07:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.033+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.034+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.034+0000 7f9fece00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.034+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659bea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9fece00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54334/0 (socket says 172.21.15.7:54334) 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9fe7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9fe7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33716/0 (socket says 172.21.15.7:33716) 2024-06-13T07:07:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9fece00700 1 -- 172.21.15.7:0/1756176824 learned_addr learned my addr 172.21.15.7:0/1756176824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd658a2f60 con 0x55fd659bb4b0 2024-06-13T07:07:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.036+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd65902ac0 con 0x55fd659bab90 2024-06-13T07:07:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.037+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd659c1e30 con 0x55fd659b5240 2024-06-13T07:07:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9ffa1fbb80 10 client.0 _ll_register_callbacks cb 0x55fd6589c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9fece00700 1 -- 172.21.15.7:0/1756176824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 msgr2=0x55fd659be500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9fece00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9fece00700 1 -- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 msgr2=0x55fd659bea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9fece00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.038+0000 7f9fece00700 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd658558d0 con 0x55fd659b5240 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.039+0000 7f9fece00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009610 tx=0x7f9fe00417c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc5fe9dc286e340 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.039+0000 7f9fd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.039+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.040+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0041b20 con 0x55fd659b5240 2024-06-13T07:07:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.040+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fe0041cc0 con 0x55fd659b5240 2024-06-13T07:07:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.041+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe0009440 con 0x55fd659b5240 2024-06-13T07:07:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.042+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9fe0003a00 con 0x55fd659b5240 2024-06-13T07:07:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.043+0000 7f9ffa1fbb80 10 client.26944 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.043+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd659cc590 con 0x55fd659b5240 2024-06-13T07:07:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.044+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe0009440 con 0x55fd659b5240 2024-06-13T07:07:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.044+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9fe0011070 con 0x55fd659b5240 2024-06-13T07:07:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.046+0000 7f9ffa1fbb80 10 client.26944 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.047+0000 7f9ffa1fbb80 10 client.26944 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:07:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.047+0000 7f9ffa1fbb80 20 client.26944 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.047+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd659d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.047+0000 7f9ffa1fbb80 4 client.26944 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.047+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd65914000 con 0x55fd659cea60 2024-06-13T07:07:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.051+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd659d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.052+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd659d0e50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=baca2e58742b7bdf server_cookie=3fff640f30079fed in_seq=0 out_seq=0 2024-06-13T07:07:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.052+0000 7f9fd2a00700 10 client.26944 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:07:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.054+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1756176824 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fd65914000 con 0x55fd659cea60 2024-06-13T07:07:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.054+0000 7f9fd2a00700 10 client.26944 handle_command_reply: tid=0 2024-06-13T07:07:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.054+0000 7f9ffa1fbb80 1 client.26944 shutdown 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 msgr2=0x55fd659c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009610 tx=0x7f9fe00417c0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 shutdown_connections 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659bea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659b5240 0x55fd659c1190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd659d0e50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9fd2a00700 0 client.26944 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:07:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1756176824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bab90 0x55fd659be500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 >> 172.21.15.7:0/1756176824 conn(0x55fd658e1170 msgr2=0x55fd659a9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.055+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 shutdown_connections 2024-06-13T07:07:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.058+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1756176824 wait complete. 2024-06-13T07:07:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.058+0000 7f9ffa1fbb80 20 client.26944 trim_cache size 0 max 16384 2024-06-13T07:07:15.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.092+0000 7f9ffa1fbb80 1 Processor -- start 2024-06-13T07:07:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.092+0000 7f9ffa1fbb80 1 -- start start 2024-06-13T07:07:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.093+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659b5240 0x55fd659aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.093+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 0x55fd659b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81ae840 con 0x55fd659b5240 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81ac0d0 con 0x55fd659bb4b0 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8081540 con 0x55fd659bab90 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fece00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659b5240 0x55fd659aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54350/0 (socket says 172.21.15.7:54350) 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 -- 172.21.15.7:0/1292967587 learned_addr learned my addr 172.21.15.7:0/1292967587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 -- 172.21.15.7:0/1292967587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 msgr2=0x55fd659b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fed800700 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 0x55fd659b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 0x55fd659b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 -- 172.21.15.7:0/1292967587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659b5240 msgr2=0x55fd659aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659b5240 0x55fd659aa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.094+0000 7f9fe7e00700 1 -- 172.21.15.7:0/1292967587 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd659c1990 con 0x55fd659bab90 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.095+0000 7f9fed800700 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 0x55fd659b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.095+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005c30 tx=0x7f9fd4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac7d3b43f1b5c75 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.095+0000 7f9fdd600700 1 -- 172.21.15.7:0/1292967587 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4004050 con 0x55fd659bab90 2024-06-13T07:07:15.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.095+0000 7f9fdd600700 1 -- 172.21.15.7:0/1292967587 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd4004830 con 0x55fd659bab90 2024-06-13T07:07:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.095+0000 7f9fdd600700 1 -- 172.21.15.7:0/1292967587 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40054b0 con 0x55fd659bab90 2024-06-13T07:07:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.098+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1292967587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 msgr2=0x55fd659c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.098+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005c30 tx=0x7f9fd4009260 comp rx=0 tx=0).stop 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.098+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1292967587 shutdown_connections 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.098+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659bb4b0 0x55fd659b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.098+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659b5240 0x55fd659aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.099+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1292967587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659c0d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.099+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1292967587 >> 172.21.15.7:0/1292967587 conn(0x55fd658e1170 msgr2=0x55fd659bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.099+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1292967587 shutdown_connections 2024-06-13T07:07:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.099+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1292967587 wait complete. 2024-06-13T07:07:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.100+0000 7f9ffa1fbb80 1 -- start start 2024-06-13T07:07:15.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.102+0000 7f9ffa1fbb80 1 Processor -- start 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8081bb0 con 0x55fd659bb4b0 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8083020 con 0x55fd659b5240 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8082280 con 0x55fd659bab90 2024-06-13T07:07:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9fece00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9fece00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33728/0 (socket says 172.21.15.7:33728) 2024-06-13T07:07:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9fece00700 1 -- 172.21.15.7:0/1303128382 learned_addr learned my addr 172.21.15.7:0/1303128382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.104+0000 7f9ffa1fbb80 10 client.0 _ll_register_callbacks cb 0x55fd6589c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.105+0000 7f9fece00700 1 -- 172.21.15.7:0/1303128382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 msgr2=0x55fd659d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.105+0000 7f9fed800700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.105+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fece00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fece00700 1 -- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 msgr2=0x55fd659d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fece00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fece00700 1 -- 172.21.15.7:0/1303128382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd65a374e0 con 0x55fd659b5240 2024-06-13T07:07:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fed800700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fece00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b3f0 tx=0x7f9fe00094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393d88db08d46b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.108+0000 7f9fd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.109+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0059060 con 0x55fd659b5240 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.109+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd4004830 con 0x55fd659b5240 2024-06-13T07:07:15.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.109+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fe0041910 con 0x55fd659b5240 2024-06-13T07:07:15.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.111+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9fe0032040 con 0x55fd659b5240 2024-06-13T07:07:15.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.111+0000 7f9ffa1fbb80 10 client.27026 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:15.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.111+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd65a38750 con 0x55fd659b5240 2024-06-13T07:07:15.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.111+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd4004830 con 0x55fd659b5240 2024-06-13T07:07:15.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9fe0011070 con 0x55fd659b5240 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 10 client.27026 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 10 client.27026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 20 client.27026 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd65a3ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 4 client.27026 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.113+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd6590e580 con 0x55fd659cea60 2024-06-13T07:07:15.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.114+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd65a3ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.116+0000 7f9fe7e00700 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd65a3ae20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5da59bef50ac0863 server_cookie=7eddc89229c0bc3a in_seq=0 out_seq=0 2024-06-13T07:07:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.117+0000 7f9fd2a00700 10 client.27026 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:07:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.117+0000 7f9fd2a00700 1 -- 172.21.15.7:0/1303128382 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd6590e580 con 0x55fd659cea60 2024-06-13T07:07:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.117+0000 7f9fd2a00700 10 client.27026 handle_command_reply: tid=0 2024-06-13T07:07:15.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.117+0000 7f9ffa1fbb80 1 client.27026 shutdown 2024-06-13T07:07:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 msgr2=0x55fd659d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b3f0 tx=0x7f9fe00094e0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 shutdown_connections 2024-06-13T07:07:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd659bb4b0 0x55fd659d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55fd659cea60 0x55fd65a3ae20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd659b5240 0x55fd659d2e00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 --2- 172.21.15.7:0/1303128382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd659bab90 0x55fd659d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.118+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 >> 172.21.15.7:0/1303128382 conn(0x55fd658e1170 msgr2=0x55fd659b9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.119+0000 7f9fd2a00700 0 client.27026 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.119+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 shutdown_connections 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.119+0000 7f9ffa1fbb80 1 -- 172.21.15.7:0/1303128382 wait complete. 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.119+0000 7f9ffa1fbb80 20 client.27026 trim_cache size 0 max 16384 2024-06-13T07:07:15.121 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:07:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 msgr2=0x7f9fcc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 0x7f9fcc0a5360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000b780 tx=0x7f9fe0009c40 comp rx=0 tx=0).stop 2024-06-13T07:07:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 msgr2=0x7f9fe81a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 0x7f9fe81a3c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e900 tx=0x7f9fd800ec10 comp rx=0 tx=0).stop 2024-06-13T07:07:15.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 shutdown_connections 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe810eac0 0x7f9fe81a3c10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe8108ac0 0x7f9fe819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fcc0a2ea0 0x7f9fcc0a5360 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 --2- 172.21.15.7:0/1674984680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe8109480 0x7f9fe819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 >> 172.21.15.7:0/1674984680 conn(0x7f9fe8075c60 msgr2=0x7f9fe8076260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.121+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 shutdown_connections 2024-06-13T07:07:15.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:15.122+0000 7f9fef800700 1 -- 172.21.15.7:0/1674984680 wait complete. 2024-06-13T07:07:15.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:15 smithi007 ceph-mon[32077]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 573 op/s 2024-06-13T07:07:15.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:15 smithi189 ceph-mon[36616]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 573 op/s 2024-06-13T07:07:15.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:15 smithi175 ceph-mon[36886]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.0 MiB/s wr, 573 op/s 2024-06-13T07:07:17.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:07:17.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:17 smithi189 ceph-mon[36616]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-06-13T07:07:17.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:17 smithi175 ceph-mon[36886]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-06-13T07:07:17.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:17 smithi007 ceph-mon[32077]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-06-13T07:07:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 675.0 s, 443.4 tps, lat 220.687 ms stddev 259.998 2024-06-13T07:07:18.039 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 -- 172.21.15.7:0/2676125454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8108030 msgr2=0x7f9da8108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 --2- 172.21.15.7:0/2676125454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8108030 0x7f9da8108450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ca0 tx=0x7f9d9c02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 -- 172.21.15.7:0/2676125454 shutdown_connections 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 --2- 172.21.15.7:0/2676125454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8109be0 0x7f9da8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 --2- 172.21.15.7:0/2676125454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109220 0x7f9da81096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 --2- 172.21.15.7:0/2676125454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8108030 0x7f9da8108450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 -- 172.21.15.7:0/2676125454 >> 172.21.15.7:0/2676125454 conn(0x7f9da8076ff0 msgr2=0x7f9da8079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 -- 172.21.15.7:0/2676125454 shutdown_connections 2024-06-13T07:07:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.585+0000 7f9dafa00640 1 -- 172.21.15.7:0/2676125454 wait complete. 2024-06-13T07:07:19.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:19 smithi189 ceph-mon[36616]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 Processor -- start 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 -- start start 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 0x7f9da819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8109220 0x7f9da819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8113d10 con 0x7f9da8108030 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8113b90 con 0x7f9da8109220 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.586+0000 7f9dafa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8113e90 con 0x7f9da8109be0 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54374/0 (socket says 172.21.15.7:54374) 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 -- 172.21.15.7:0/3419526321 learned_addr learned my addr 172.21.15.7:0/3419526321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9da7e00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8109220 0x7f9da819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dad000640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 0x7f9da819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 -- 172.21.15.7:0/3419526321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8109220 msgr2=0x7f9da819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8109220 0x7f9da819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 -- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 msgr2=0x7f9da819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 0x7f9da819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dada00640 1 -- 172.21.15.7:0/3419526321 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da81a5f40 con 0x7f9da8109be0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.587+0000 7f9dad000640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 0x7f9da819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9dada00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e970 tx=0x7f9da000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9da5600640 1 -- 172.21.15.7:0/3419526321 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000ccc0 con 0x7f9da8109be0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9da5600640 1 -- 172.21.15.7:0/3419526321 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da000ce60 con 0x7f9da8109be0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9da5600640 1 -- 172.21.15.7:0/3419526321 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0010490 con 0x7f9da8109be0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da81a6230 con 0x7f9da8109be0 2024-06-13T07:07:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.588+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da81a6730 con 0x7f9da8109be0 2024-06-13T07:07:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.594+0000 7f9da5600640 1 -- 172.21.15.7:0/3419526321 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9da0005020 con 0x7f9da8109be0 2024-06-13T07:07:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.595+0000 7f9db27d0740 1 Processor -- start 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.595+0000 7f9db27d0740 1 -- start start 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 0x55f62612b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f62612dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f626089420 con 0x55f6261265f0 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f626030c50 con 0x55f626125c00 2024-06-13T07:07:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9db27d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6260853e0 con 0x55f62612b760 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9da7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9dad000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 0x55f62612b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9dad000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 0x55f62612b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34722/0 (socket says 172.21.15.7:34722) 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9da7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33754/0 (socket says 172.21.15.7:33754) 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9dad000640 1 -- 172.21.15.7:0/255186034 learned_addr learned my addr 172.21.15.7:0/255186034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.596+0000 7f9da7e00640 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f62612dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 -- 172.21.15.7:0/255186034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 msgr2=0x55f62612dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f62612dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 -- 172.21.15.7:0/255186034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 msgr2=0x55f62612b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 0x55f62612b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 -- 172.21.15.7:0/255186034 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f625d9d210 con 0x55f626125c00 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da7e00640 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9da819f830 tx=0x7f9d94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8942938d3ec22618 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da5600640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 0x7f9d900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9d9a000640 1 -- 172.21.15.7:0/255186034 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94004050 con 0x55f626125c00 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9da5600640 1 -- 172.21.15.7:0/3419526321 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9da00cf790 con 0x7f9da8109be0 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9d9a000640 1 -- 172.21.15.7:0/255186034 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9400fca0 con 0x55f626125c00 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.597+0000 7f9d9a000640 1 -- 172.21.15.7:0/255186034 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94003a50 con 0x55f626125c00 2024-06-13T07:07:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9dad000640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 0x7f9d900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 -- 172.21.15.7:0/255186034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 msgr2=0x55f626126020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9da819f830 tx=0x7f9d94004730 comp rx=0 tx=0).stop 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 -- 172.21.15.7:0/255186034 shutdown_connections 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f62612dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f6261265f0 0x55f62612b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 --2- 172.21.15.7:0/255186034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f626125c00 0x55f626126020 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.598+0000 7f9db27d0740 1 -- 172.21.15.7:0/255186034 >> 172.21.15.7:0/255186034 conn(0x55f6260edfb0 msgr2=0x55f626121680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.599+0000 7f9db27d0740 1 -- 172.21.15.7:0/255186034 shutdown_connections 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.599+0000 7f9db27d0740 1 -- 172.21.15.7:0/255186034 wait complete. 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.599+0000 7f9dad000640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 0x7f9d900a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55f626127060 tx=0x7f9d9c005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.599+0000 7f9db27d0740 1 -- start start 2024-06-13T07:07:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.599+0000 7f9db27d0740 1 Processor -- start 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f626125c00 0x55f62612ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f62612e710 con 0x55f626125c00 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f6260440b0 con 0x55f6261265f0 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f62612e890 con 0x55f62612b760 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.600+0000 7f9db27d0740 10 client.0 _ll_register_callbacks cb 0x55f62602ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.601+0000 7f9dada00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.601+0000 7f9dada00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54398/0 (socket says 172.21.15.7:54398) 2024-06-13T07:07:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.601+0000 7f9dada00640 1 -- 172.21.15.7:0/1898782460 learned_addr learned my addr 172.21.15.7:0/1898782460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.601+0000 7f9da7e00640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.601+0000 7f9dada00640 1 -- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 msgr2=0x55f6261304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9dada00640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261304b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9dada00640 1 -- 172.21.15.7:0/1898782460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f626125c00 msgr2=0x55f62612ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9dad000640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f626125c00 0x55f62612ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9dada00640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f626125c00 0x55f62612ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9dada00640 1 -- 172.21.15.7:0/1898782460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f625e69620 con 0x55f62612b760 2024-06-13T07:07:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.602+0000 7f9da7e00640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.604+0000 7f9dada00640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9da009bc40 tx=0x7f9da009c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17112e6875f2383 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.604+0000 7f9d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.604+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0099fb0 con 0x55f62612b760 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.604+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9da0005310 con 0x55f62612b760 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.604+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9400fca0 con 0x55f62612b760 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.605+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9da00a4490 con 0x55f62612b760 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.605+0000 7f9db27d0740 10 client.26959 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.605+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f62614b940 con 0x55f62612b760 2024-06-13T07:07:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.605+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9400fca0 con 0x55f62612b760 2024-06-13T07:07:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9da009ab70 con 0x55f62612b760 2024-06-13T07:07:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 10 client.26959 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 10 client.26959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 20 client.26959 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f62614d0d0 0x55f62614f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 4 client.26959 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.606+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f626096b50 con 0x55f62614d0d0 2024-06-13T07:07:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.609+0000 7f9dad000640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f62614d0d0 0x55f62614f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.614+0000 7f9d8ea00640 10 client.26959 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.614+0000 7f9dad000640 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f62614d0d0 0x55f62614f500 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c326691710e47e50 server_cookie=5a756f4c3de87a0 in_seq=0 out_seq=0 2024-06-13T07:07:19.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:19 smithi175 ceph-mon[36886]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-06-13T07:07:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.621+0000 7f9d8ea00640 1 -- 172.21.15.7:0/1898782460 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f626096b50 con 0x55f62614d0d0 2024-06-13T07:07:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.621+0000 7f9d8ea00640 10 client.26959 handle_command_reply: tid=0 2024-06-13T07:07:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.621+0000 7f9db27d0740 1 client.26959 shutdown 2024-06-13T07:07:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 msgr2=0x55f6261374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9da009bc40 tx=0x7f9da009c4f0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 shutdown_connections 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f62614d0d0 0x55f62614f500 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f62612b760 0x55f6261374f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 --2- 172.21.15.7:0/1898782460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f626125c00 0x55f62612ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 >> 172.21.15.7:0/1898782460 conn(0x55f6260edfb0 msgr2=0x55f626120840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.625+0000 7f9d8ea00640 0 client.26959 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.626+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 shutdown_connections 2024-06-13T07:07:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.626+0000 7f9db27d0740 1 -- 172.21.15.7:0/1898782460 wait complete. 2024-06-13T07:07:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.626+0000 7f9db27d0740 20 client.26959 trim_cache size 0 max 16384 2024-06-13T07:07:19.633 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:19 smithi007 ceph-mon[32077]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-06-13T07:07:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 Processor -- start 2024-06-13T07:07:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 -- start start 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f62612afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81aac90 con 0x55f62612b760 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81a8560 con 0x55f6261265f0 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.671+0000 7f9db27d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80734e0 con 0x55f626125c00 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.672+0000 7f9dad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.672+0000 7f9dad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54404/0 (socket says 172.21.15.7:54404) 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.672+0000 7f9dad000640 1 -- 172.21.15.7:0/380835901 learned_addr learned my addr 172.21.15.7:0/380835901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.672+0000 7f9da7e00640 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dada00640 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f62612afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dad000640 1 -- 172.21.15.7:0/380835901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 msgr2=0x55f6261374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dad000640 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dad000640 1 -- 172.21.15.7:0/380835901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 msgr2=0x55f62612afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dad000640 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f62612afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.673+0000 7f9dad000640 1 -- 172.21.15.7:0/380835901 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f626125fe0 con 0x55f626125c00 2024-06-13T07:07:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.675+0000 7f9dad000640 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9da8109080 tx=0x7f9d9c041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce6d038f745bac6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.675+0000 7f9d99600640 1 -- 172.21.15.7:0/380835901 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c02f610 con 0x55f626125c00 2024-06-13T07:07:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.675+0000 7f9d99600640 1 -- 172.21.15.7:0/380835901 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c036040 con 0x55f626125c00 2024-06-13T07:07:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.676+0000 7f9d99600640 1 -- 172.21.15.7:0/380835901 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c030c10 con 0x55f626125c00 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.677+0000 7f9db27d0740 1 -- 172.21.15.7:0/380835901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 msgr2=0x55f626121810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.677+0000 7f9db27d0740 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9da8109080 tx=0x7f9d9c041000 comp rx=0 tx=0).stop 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 -- 172.21.15.7:0/380835901 shutdown_connections 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f62612afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f6261374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 --2- 172.21.15.7:0/380835901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626121810 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 -- 172.21.15.7:0/380835901 >> 172.21.15.7:0/380835901 conn(0x55f6260edfb0 msgr2=0x55f626121100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 -- 172.21.15.7:0/380835901 shutdown_connections 2024-06-13T07:07:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.678+0000 7f9db27d0740 1 -- 172.21.15.7:0/380835901 wait complete. 2024-06-13T07:07:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.679+0000 7f9db27d0740 1 -- start start 2024-06-13T07:07:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.679+0000 7f9db27d0740 1 Processor -- start 2024-06-13T07:07:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f626132b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f626135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9dad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9dad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54414/0 (socket says 172.21.15.7:54414) 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9dad000640 1 -- 172.21.15.7:0/4129472451 learned_addr learned my addr 172.21.15.7:0/4129472451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81a6a50 con 0x55f62612b760 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81a7e90 con 0x55f6261265f0 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81a70f0 con 0x55f626125c00 2024-06-13T07:07:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.681+0000 7f9db27d0740 10 client.0 _ll_register_callbacks cb 0x7f9da0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.682+0000 7f9dada00640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f626135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.683+0000 7f9da7e00640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f626132b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.683+0000 7f9dad000640 1 -- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 msgr2=0x55f626132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.683+0000 7f9dad000640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f626132b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.683+0000 7f9dad000640 1 -- 172.21.15.7:0/4129472451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 msgr2=0x55f626135ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.683+0000 7f9dad000640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f626135ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.686+0000 7f9dad000640 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f626136190 con 0x55f626125c00 2024-06-13T07:07:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.686+0000 7f9dad000640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c030860 tx=0x7f9d9c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9350adabde339a5f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.686+0000 7f9d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.687+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c061070 con 0x55f626125c00 2024-06-13T07:07:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.687+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9c036040 con 0x55f626125c00 2024-06-13T07:07:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.687+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9d9c030990 con 0x55f626125c00 2024-06-13T07:07:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.688+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c06d050 con 0x55f626125c00 2024-06-13T07:07:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.688+0000 7f9db27d0740 10 client.26965 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:19.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.689+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6261374f0 con 0x55f626125c00 2024-06-13T07:07:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.689+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c036040 con 0x55f626125c00 2024-06-13T07:07:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9d9c030b90 con 0x55f626125c00 2024-06-13T07:07:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 10 client.26965 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 10 client.26965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 20 client.26965 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6261c9230 0x55f6261cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 4 client.26965 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f626090ea0 con 0x55f6261c9230 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.691+0000 7f9da7e00640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6261c9230 0x55f6261cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.692+0000 7f9da7e00640 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6261c9230 0x55f6261cb650 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2faf54d68118fdb7 server_cookie=3a0b63c49b5964c9 in_seq=0 out_seq=0 2024-06-13T07:07:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.692+0000 7f9d8ea00640 10 client.26965 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.727+0000 7f9d8ea00640 1 -- 172.21.15.7:0/4129472451 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f626090ea0 con 0x55f6261c9230 2024-06-13T07:07:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.727+0000 7f9d8ea00640 10 client.26965 handle_command_reply: tid=0 2024-06-13T07:07:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.729+0000 7f9db27d0740 1 client.26965 shutdown 2024-06-13T07:07:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 msgr2=0x55f626132640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c030860 tx=0x7f9d9c02a8c0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 shutdown_connections 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55f6261c9230 0x55f6261cb650 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f62612b760 0x55f626135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f6261265f0 0x55f626132b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 --2- 172.21.15.7:0/4129472451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f626125c00 0x55f626132640 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 >> 172.21.15.7:0/4129472451 conn(0x55f6260edfb0 msgr2=0x55f626121100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9d8ea00640 0 client.26965 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 shutdown_connections 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.730+0000 7f9db27d0740 1 -- 172.21.15.7:0/4129472451 wait complete. 2024-06-13T07:07:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.731+0000 7f9db27d0740 20 client.26965 trim_cache size 0 max 16384 2024-06-13T07:07:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:07:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:07:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 msgr2=0x7f9d900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 0x7f9d900a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55f626127060 tx=0x7f9d9c005d80 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 msgr2=0x7f9da81a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e970 tx=0x7f9da000ed70 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 shutdown_connections 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9d900a2cb0 0x7f9d900a5170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da8109be0 0x7f9da81a58d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9da8109220 0x7f9da819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 --2- 172.21.15.7:0/3419526321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8108030 0x7f9da819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.732+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 >> 172.21.15.7:0/3419526321 conn(0x7f9da8076ff0 msgr2=0x7f9da810c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.733+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 shutdown_connections 2024-06-13T07:07:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:19.733+0000 7f9dafa00640 1 -- 172.21.15.7:0/3419526321 wait complete. 2024-06-13T07:07:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.199+0000 7f1797600700 1 -- 172.21.15.7:0/3162962492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17901073c0 msgr2=0x7f179006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.199+0000 7f1797600700 1 --2- 172.21.15.7:0/3162962492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17901073c0 0x7f179006a1a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1784009990 tx=0x7f1784009ca0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.201+0000 7f1797600700 1 -- 172.21.15.7:0/3162962492 shutdown_connections 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.201+0000 7f1797600700 1 --2- 172.21.15.7:0/3162962492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 0x7f1790105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.201+0000 7f1797600700 1 --2- 172.21.15.7:0/3162962492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17901073c0 0x7f179006a1a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.201+0000 7f1797600700 1 --2- 172.21.15.7:0/3162962492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f179006a6e0 0x7f179006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.201+0000 7f1797600700 1 -- 172.21.15.7:0/3162962492 >> 172.21.15.7:0/3162962492 conn(0x7f1790078340 msgr2=0x7f1790078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.202+0000 7f1797600700 1 -- 172.21.15.7:0/3162962492 shutdown_connections 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.202+0000 7f1797600700 1 -- 172.21.15.7:0/3162962492 wait complete. 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.204+0000 7f1797600700 1 Processor -- start 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.204+0000 7f1797600700 1 -- start start 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.204+0000 7f1797600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1797600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 0x7f179019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1797600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17901073c0 0x7f17901a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1797600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1790118df0 con 0x7f17901073c0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1797600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1790118c70 con 0x7f1790104ed0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1797600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1790118f70 con 0x7f179006a6e0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45482/0 (socket says 172.21.15.7:45482) 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1794c00700 1 -- 172.21.15.7:0/3321742014 learned_addr learned my addr 172.21.15.7:0/3321742014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f178fe00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 0x7f179019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1795600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17901073c0 0x7f17901a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1794c00700 1 -- 172.21.15.7:0/3321742014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 msgr2=0x7f179019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.205+0000 7f1794c00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 0x7f179019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.206+0000 7f1794c00700 1 -- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17901073c0 msgr2=0x7f17901a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.206+0000 7f1794c00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17901073c0 0x7f17901a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.206+0000 7f1794c00700 1 -- 172.21.15.7:0/3321742014 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a42d0 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.206+0000 7f1794c00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f178400bab0 tx=0x7f178400bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.209+0000 7f178d600700 1 -- 172.21.15.7:0/3321742014 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784027070 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.209+0000 7f178d600700 1 -- 172.21.15.7:0/3321742014 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1784003d70 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.209+0000 7f178d600700 1 -- 172.21.15.7:0/3321742014 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784027070 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.210+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a4560 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.210+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a4900 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.218+0000 7f178d600700 1 -- 172.21.15.7:0/3321742014 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1784009d10 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.220+0000 7f178d600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 0x7f17780a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.220+0000 7f178fe00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 0x7f17780a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.223+0000 7f17a1f18b80 1 Processor -- start 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.223+0000 7f178d600700 1 -- 172.21.15.7:0/3321742014 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f178409a7f0 con 0x7f179006a6e0 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.223+0000 7f17a1f18b80 1 -- start start 2024-06-13T07:07:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.224+0000 7f178fe00700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 0x7f17780a5360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f177c006fd0 tx=0x7f177c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.224+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f1794c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f8e9aea10 con 0x558f8eaa5b90 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f8e9f1b00 con 0x558f8eaa0240 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.225+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f8e9ac8f0 con 0x558f8eaa64b0 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f1795600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45492/0 (socket says 172.21.15.7:45492) 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f1794c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33340/0 (socket says 172.21.15.7:33340) 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f1795600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52270/0 (socket says 172.21.15.7:52270) 2024-06-13T07:07:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 -- 172.21.15.7:0/4238266281 learned_addr learned my addr 172.21.15.7:0/4238266281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 -- 172.21.15.7:0/4238266281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 msgr2=0x558f8eaa0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 -- 172.21.15.7:0/4238266281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 msgr2=0x558f8eaa5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f178fe00700 1 -- 172.21.15.7:0/4238266281 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8e999f90 con 0x558f8eaa64b0 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.226+0000 7f1795600700 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.227+0000 7f1794c00700 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.227+0000 7f178fe00700 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d720 tx=0x7f177c010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00362a7dc79e26c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.227+0000 7f1782000700 1 -- 172.21.15.7:0/4238266281 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c024070 con 0x558f8eaa64b0 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.228+0000 7f1782000700 1 -- 172.21.15.7:0/4238266281 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177c00de40 con 0x558f8eaa64b0 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.228+0000 7f1782000700 1 -- 172.21.15.7:0/4238266281 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c01fb30 con 0x558f8eaa64b0 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.229+0000 7f17a1f18b80 1 -- 172.21.15.7:0/4238266281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 msgr2=0x558f8ea9fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.229+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d720 tx=0x7f177c010000 comp rx=0 tx=0).stop 2024-06-13T07:07:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.232+0000 7f17a1f18b80 1 -- 172.21.15.7:0/4238266281 shutdown_connections 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.232+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa0240 0x558f8eaa0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.232+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eaa5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.232+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/4238266281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa64b0 0x558f8ea9fd00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.232+0000 7f17a1f18b80 1 -- 172.21.15.7:0/4238266281 >> 172.21.15.7:0/4238266281 conn(0x558f8e9cc170 msgr2=0x558f8ea9aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.233+0000 7f17a1f18b80 1 -- 172.21.15.7:0/4238266281 shutdown_connections 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.233+0000 7f17a1f18b80 1 -- 172.21.15.7:0/4238266281 wait complete. 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.234+0000 7f17a1f18b80 1 -- start start 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.234+0000 7f17a1f18b80 1 Processor -- start 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 0x558f8eaac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa5b90 0x558f8eaa95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f8e98df60 con 0x558f8eaa0240 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558f8e9edac0 con 0x558f8eaa64b0 2024-06-13T07:07:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f8eaace90 con 0x558f8eaa5b90 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.243+0000 7f17a1f18b80 10 client.0 _ll_register_callbacks cb 0x558f8e987380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1794c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 0x558f8eaac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52282/0 (socket says 172.21.15.7:52282) 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 -- 172.21.15.7:0/3299682221 learned_addr learned my addr 172.21.15.7:0/3299682221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f178fe00700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa5b90 0x558f8eaa95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 -- 172.21.15.7:0/3299682221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa5b90 msgr2=0x558f8eaa95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa5b90 0x558f8eaa95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 -- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 msgr2=0x558f8eaac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 0x558f8eaac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1795600700 1 -- 172.21.15.7:0/3299682221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8e9408d0 con 0x558f8eaa64b0 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.244+0000 7f1794c00700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 0x558f8eaac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.246+0000 7f1795600700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d10 tx=0x7f1788007f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d2c4ac3d92800c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.246+0000 7f1776a00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.246+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178801a030 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.246+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1788013930 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.246+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f8e9ff000 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.247+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1788013b30 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.247+0000 7f17a1f18b80 10 client.27053 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.247+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f8eab76f0 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f8e9ff000 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f1788016430 con 0x558f8eaa64b0 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 10 client.27053 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 10 client.27053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 20 client.27053 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eabc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 4 client.27053 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.248+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8e9f9580 con 0x558f8eab9d80 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.249+0000 7f1794c00700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eabc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.250+0000 7f1794c00700 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eabc170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7236e1fb1983c94 server_cookie=9093a6a458f6b8e5 in_seq=0 out_seq=0 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.250+0000 7f1776a00700 10 client.27053 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.251+0000 7f1776a00700 1 -- 172.21.15.7:0/3299682221 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f8e9f9580 con 0x558f8eab9d80 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.251+0000 7f1776a00700 10 client.27053 handle_command_reply: tid=0 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.251+0000 7f17a1f18b80 1 client.27053 shutdown 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.252+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 msgr2=0x558f8eaa9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.252+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d10 tx=0x7f1788007f80 comp rx=0 tx=0).stop 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 shutdown_connections 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eaa9ae0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa0240 0x558f8eaac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eabc170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f1776a00700 0 client.27053 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3299682221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa5b90 0x558f8eaa95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.254+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 >> 172.21.15.7:0/3299682221 conn(0x558f8e9cc170 msgr2=0x558f8ea94790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.255+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 shutdown_connections 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.256+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3299682221 wait complete. 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.256+0000 7f17a1f18b80 20 client.27053 trim_cache size 0 max 16384 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.288+0000 7f17a1f18b80 1 Processor -- start 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.288+0000 7f17a1f18b80 1 -- start start 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa5b90 0x558f8ea95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 0x558f8eaa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b6f50 con 0x558f8eaa64b0 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b47e0 con 0x558f8eaa5b90 2024-06-13T07:07:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b2630 con 0x558f8eaa0240 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45512/0 (socket says 172.21.15.7:45512) 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 0x558f8eaa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 -- 172.21.15.7:0/3592018533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 msgr2=0x558f8ea9fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 -- 172.21.15.7:0/3592018533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa5b90 msgr2=0x558f8ea95130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa5b90 0x558f8ea95130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.290+0000 7f1795600700 1 -- 172.21.15.7:0/3592018533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8eaa9370 con 0x558f8eaa64b0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.289+0000 7f1794c00700 1 -- 172.21.15.7:0/3592018533 learned_addr learned my addr 172.21.15.7:0/3592018533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.291+0000 7f1794c00700 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.291+0000 7f1795600700 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 0x558f8eaa7040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1788016a90 tx=0x7f1788009160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd781050f5119e13 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.292+0000 7f1781600700 1 -- 172.21.15.7:0/3592018533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788030020 con 0x558f8eaa64b0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.292+0000 7f1781600700 1 -- 172.21.15.7:0/3592018533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788013d90 con 0x558f8eaa64b0 2024-06-13T07:07:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.292+0000 7f1781600700 1 -- 172.21.15.7:0/3592018533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788030020 con 0x558f8eaa64b0 2024-06-13T07:07:21.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:21 smithi007 ceph-mon[32077]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 483 op/s 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.293+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3592018533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 msgr2=0x558f8eaa7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.293+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 0x558f8eaa7040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1788016a90 tx=0x7f1788009160 comp rx=0 tx=0).stop 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.294+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3592018533 shutdown_connections 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.294+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa64b0 0x558f8eaa7040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.294+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8ea9fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.294+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/3592018533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa5b90 0x558f8ea95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.294+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3592018533 >> 172.21.15.7:0/3592018533 conn(0x558f8e9cc170 msgr2=0x558f8eaa6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.295+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3592018533 shutdown_connections 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.295+0000 7f17a1f18b80 1 -- 172.21.15.7:0/3592018533 wait complete. 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.296+0000 7f17a1f18b80 1 -- start start 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.296+0000 7f17a1f18b80 1 Processor -- start 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eab2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eac0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b2ca0 con 0x558f8eaa5b90 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b4110 con 0x558f8eaa64b0 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b3370 con 0x558f8eaa0240 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.297+0000 7f17a1f18b80 10 client.0 _ll_register_callbacks cb 0x558f8eaaa180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.298+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.298+0000 7f1794c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45516/0 (socket says 172.21.15.7:45516) 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.298+0000 7f1794c00700 1 -- 172.21.15.7:0/1339848902 learned_addr learned my addr 172.21.15.7:0/1339848902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:21.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f178fe00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eab2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f1794c00700 1 -- 172.21.15.7:0/1339848902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 msgr2=0x558f8eac0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f1794c00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eac0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f1794c00700 1 -- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 msgr2=0x558f8eab2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f1794c00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eab2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.299+0000 7f1794c00700 1 -- 172.21.15.7:0/1339848902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8eac1580 con 0x558f8eaa0240 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.300+0000 7f1776a00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.301+0000 7f1794c00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17840cd930 tx=0x7f17840a4e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c56af2d0cb9b3a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.302+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784099180 con 0x558f8eaa0240 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.302+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1784022700 con 0x558f8eaa0240 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.302+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788013d90 con 0x558f8eaa0240 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.302+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1784099320 con 0x558f8eaa0240 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 10 client.26980 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:21.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f8eb24430 con 0x558f8eaa0240 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788013d90 con 0x558f8eaa0240 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f1784085ce0 con 0x558f8eaa0240 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 10 client.26980 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 10 client.26980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 20 client.26980 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eb26960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 4 client.26980 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.303+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f8e873250 con 0x558f8eab9d80 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.304+0000 7f178fe00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eb26960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.305+0000 7f178fe00700 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eb26960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93315ec5cfc719c3 server_cookie=d07757b3d463dc5b in_seq=0 out_seq=0 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.305+0000 7f1776a00700 10 client.26980 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.305+0000 7f1776a00700 1 -- 172.21.15.7:0/1339848902 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f8e873250 con 0x558f8eab9d80 2024-06-13T07:07:21.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.305+0000 7f1776a00700 10 client.26980 handle_command_reply: tid=0 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.305+0000 7f17a1f18b80 1 client.26980 shutdown 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 msgr2=0x558f8eab2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f17840cd930 tx=0x7f17840a4e30 comp rx=0 tx=0).stop 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 shutdown_connections 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558f8eaa64b0 0x558f8eac0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x558f8eab9d80 0x558f8eb26960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f8eaa0240 0x558f8eab2330 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 --2- 172.21.15.7:0/1339848902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f8eaa5b90 0x558f8eab2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f1776a00700 0 client.26980 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 >> 172.21.15.7:0/1339848902 conn(0x558f8e9cc170 msgr2=0x558f8eaa7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.306+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 shutdown_connections 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.307+0000 7f17a1f18b80 1 -- 172.21.15.7:0/1339848902 wait complete. 2024-06-13T07:07:21.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.307+0000 7f17a1f18b80 20 client.26980 trim_cache size 0 max 16384 2024-06-13T07:07:21.605 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:07:21.605 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:21 smithi189 ceph-mon[36616]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 483 op/s 2024-06-13T07:07:21.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.312+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 msgr2=0x7f17780a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.312+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 0x7f17780a5360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f177c006fd0 tx=0x7f177c008040 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.312+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 msgr2=0x7f179019f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.312+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f178400bab0 tx=0x7f178400bb90 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 shutdown_connections 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17901073c0 0x7f17901a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f179006a6e0 0x7f179019f2c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f17780a2ea0 0x7f17780a5360 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 --2- 172.21.15.7:0/3321742014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1790104ed0 0x7f179019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 >> 172.21.15.7:0/3321742014 conn(0x7f1790078340 msgr2=0x7f1790100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.313+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 shutdown_connections 2024-06-13T07:07:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:21.314+0000 7f1797600700 1 -- 172.21.15.7:0/3321742014 wait complete. 2024-06-13T07:07:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:21 smithi175 ceph-mon[36886]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 483 op/s 2024-06-13T07:07:22.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 680.0 s, 464.2 tps, lat 221.088 ms stddev 249.152 2024-06-13T07:07:23.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:23 smithi007 ceph-mon[32077]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 432 op/s 2024-06-13T07:07:23.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:23 smithi189 ceph-mon[36616]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 432 op/s 2024-06-13T07:07:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:23 smithi175 ceph-mon[36886]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 432 op/s 2024-06-13T07:07:24.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:07:24.740 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:25.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:25 smithi007 ceph-mon[32077]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 477 op/s 2024-06-13T07:07:25.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:25 smithi189 ceph-mon[36616]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 477 op/s 2024-06-13T07:07:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:25 smithi175 ceph-mon[36886]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 477 op/s 2024-06-13T07:07:27.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:27 smithi189 ceph-mon[36616]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 384 op/s 2024-06-13T07:07:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:27 smithi175 ceph-mon[36886]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 384 op/s 2024-06-13T07:07:27.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:27 smithi007 ceph-mon[32077]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 384 op/s 2024-06-13T07:07:27.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 685.0 s, 466.0 tps, lat 213.015 ms stddev 274.785 2024-06-13T07:07:28.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.336+0000 7f5a52800700 1 -- 172.21.15.7:0/314179873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c108500 msgr2=0x7f5a4c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.337+0000 7f5a52800700 1 --2- 172.21.15.7:0/314179873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c108500 0x7f5a4c1088e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0098f0 tx=0x7f5a3c009c00 comp rx=0 tx=0).stop 2024-06-13T07:07:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.337+0000 7f5a52800700 1 -- 172.21.15.7:0/314179873 shutdown_connections 2024-06-13T07:07:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.337+0000 7f5a52800700 1 --2- 172.21.15.7:0/314179873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.338+0000 7f5a52800700 1 --2- 172.21.15.7:0/314179873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c108500 0x7f5a4c1088e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.338+0000 7f5a52800700 1 --2- 172.21.15.7:0/314179873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06a170 0x7f5a4c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.338+0000 7f5a52800700 1 -- 172.21.15.7:0/314179873 >> 172.21.15.7:0/314179873 conn(0x7f5a4c078340 msgr2=0x7f5a4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.339+0000 7f5a52800700 1 -- 172.21.15.7:0/314179873 shutdown_connections 2024-06-13T07:07:28.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.339+0000 7f5a52800700 1 -- 172.21.15.7:0/314179873 wait complete. 2024-06-13T07:07:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.342+0000 7f5a52800700 1 Processor -- start 2024-06-13T07:07:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.343+0000 7f5a52800700 1 -- start start 2024-06-13T07:07:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c06a170 0x7f5a4c10c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4c108500 0x7f5a4c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c103080 con 0x7f5a4c108500 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c102f00 con 0x7f5a4c06ab30 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a52800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c103200 con 0x7f5a4c06a170 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52312/0 (socket says 172.21.15.7:52312) 2024-06-13T07:07:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.344+0000 7f5a4aa00700 1 -- 172.21.15.7:0/2979274961 learned_addr learned my addr 172.21.15.7:0/2979274961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.345+0000 7f5a4aa00700 1 -- 172.21.15.7:0/2979274961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c06a170 msgr2=0x7f5a4c10c620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:07:28.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.345+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c06a170 0x7f5a4c10c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.345+0000 7f5a4aa00700 1 -- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4c108500 msgr2=0x7f5a4c10d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.345+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4c108500 0x7f5a4c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.345+0000 7f5a4aa00700 1 -- 172.21.15.7:0/2979274961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1ab6e0 con 0x7f5a4c06ab30 2024-06-13T07:07:28.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.346+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9c0 tx=0x7f5a3800ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.347+0000 7f5a43e00700 1 -- 172.21.15.7:0/2979274961 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800cbd0 con 0x7f5a4c06ab30 2024-06-13T07:07:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.347+0000 7f5a43e00700 1 -- 172.21.15.7:0/2979274961 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3800cd70 con 0x7f5a4c06ab30 2024-06-13T07:07:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.347+0000 7f5a43e00700 1 -- 172.21.15.7:0/2979274961 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38010430 con 0x7f5a4c06ab30 2024-06-13T07:07:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.348+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c1ab9d0 con 0x7f5a4c06ab30 2024-06-13T07:07:28.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.348+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1abec0 con 0x7f5a4c06ab30 2024-06-13T07:07:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.364+0000 7f5a43e00700 1 -- 172.21.15.7:0/2979274961 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5a38004da0 con 0x7f5a4c06ab30 2024-06-13T07:07:28.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.367+0000 7f5a43e00700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 0x7f5a300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.368+0000 7f5a43e00700 1 -- 172.21.15.7:0/2979274961 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5a380cf580 con 0x7f5a4c06ab30 2024-06-13T07:07:28.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.368+0000 7f5a4b400700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 0x7f5a300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.370+0000 7f5a4b400700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 0x7f5a300a51b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009680 tx=0x7f5a3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.374+0000 7f5a5d196b80 1 Processor -- start 2024-06-13T07:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 -- start start 2024-06-13T07:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d44ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d44f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e21d35da10 con 0x55e21d4554b0 2024-06-13T07:07:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d3a0b00 con 0x55e21d44f240 2024-06-13T07:07:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.375+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e21d35b8f0 con 0x55e21d454b90 2024-06-13T07:07:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:45544/0 (socket says 172.21.15.7:45544) 2024-06-13T07:07:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 -- 172.21.15.7:0/50532530 learned_addr learned my addr 172.21.15.7:0/50532530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4be00700 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d44f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 -- 172.21.15.7:0/50532530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 msgr2=0x55e21d44f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d44f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.376+0000 7f5a4b400700 1 -- 172.21.15.7:0/50532530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 msgr2=0x55e21d44ed00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.377+0000 7f5a4b400700 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d44ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.377+0000 7f5a4b400700 1 -- 172.21.15.7:0/50532530 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d348f90 con 0x55e21d454b90 2024-06-13T07:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.377+0000 7f5a4b400700 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c016000 tx=0x7f5a3c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f96d095ae7c9b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.377+0000 7f5a40c00700 1 -- 172.21.15.7:0/50532530 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c005040 con 0x55e21d454b90 2024-06-13T07:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.380+0000 7f5a40c00700 1 -- 172.21.15.7:0/50532530 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c0056c0 con 0x55e21d454b90 2024-06-13T07:07:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a40c00700 1 -- 172.21.15.7:0/50532530 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00b560 con 0x55e21d454b90 2024-06-13T07:07:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 -- 172.21.15.7:0/50532530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 msgr2=0x55e21d454f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c016000 tx=0x7f5a3c01c000 comp rx=0 tx=0).stop 2024-06-13T07:07:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 -- 172.21.15.7:0/50532530 shutdown_connections 2024-06-13T07:07:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d44f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d454b90 0x55e21d454f70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.381+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/50532530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d44ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.382+0000 7f5a5d196b80 1 -- 172.21.15.7:0/50532530 >> 172.21.15.7:0/50532530 conn(0x55e21d37b170 msgr2=0x55e21d449ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.383+0000 7f5a5d196b80 1 -- 172.21.15.7:0/50532530 shutdown_connections 2024-06-13T07:07:28.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.383+0000 7f5a5d196b80 1 -- 172.21.15.7:0/50532530 wait complete. 2024-06-13T07:07:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.386+0000 7f5a5d196b80 1 -- start start 2024-06-13T07:07:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.387+0000 7f5a5d196b80 1 Processor -- start 2024-06-13T07:07:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d44f240 0x55e21d45b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d4554b0 0x55e21d458a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e21d33cf60 con 0x55e21d44f240 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e21d39cac0 con 0x55e21d454b90 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e21d46ce80 con 0x55e21d4554b0 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a5d196b80 10 client.0 _ll_register_callbacks cb 0x55e21d336380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52326/0 (socket says 172.21.15.7:52326) 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.388+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1742752320 learned_addr learned my addr 172.21.15.7:0/1742752320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.389+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1742752320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d4554b0 msgr2=0x55e21d458a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.389+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d4554b0 0x55e21d458a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.389+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1742752320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d44f240 msgr2=0x55e21d45b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.389+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d44f240 0x55e21d45b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.389+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1742752320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d2ef8d0 con 0x55e21d454b90 2024-06-13T07:07:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.390+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a3809ba30 tx=0x7f5a38005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5193d2e8b9a94afb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.390+0000 7f5a35600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.391+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a4710 con 0x55e21d454b90 2024-06-13T07:07:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.391+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5a3809bd00 con 0x55e21d454b90 2024-06-13T07:07:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e21d3ae000 con 0x55e21d454b90 2024-06-13T07:07:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5a3809bf00 con 0x55e21d454b90 2024-06-13T07:07:28.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a5d196b80 10 client.27065 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e21d477420 con 0x55e21d454b90 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e21d3ae000 con 0x55e21d454b90 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.393+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f5a3809aef0 con 0x55e21d454b90 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 10 client.27065 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 10 client.27065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 20 client.27065 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d479a60 0x55e21d47be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 4 client.27065 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.394+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21d3a8580 con 0x55e21d479a60 2024-06-13T07:07:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.395+0000 7f5a4b400700 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d479a60 0x55e21d47be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.396+0000 7f5a4b400700 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d479a60 0x55e21d47be50 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4893c0f99676c5aa server_cookie=191670543bf81150 in_seq=0 out_seq=0 2024-06-13T07:07:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.396+0000 7f5a35600700 10 client.27065 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.396+0000 7f5a35600700 1 -- 172.21.15.7:0/1742752320 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e21d3a8580 con 0x55e21d479a60 2024-06-13T07:07:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.396+0000 7f5a35600700 10 client.27065 handle_command_reply: tid=0 2024-06-13T07:07:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.397+0000 7f5a5d196b80 1 client.27065 shutdown 2024-06-13T07:07:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.397+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 msgr2=0x55e21d4584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.397+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a3809ba30 tx=0x7f5a38005850 comp rx=0 tx=0).stop 2024-06-13T07:07:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 shutdown_connections 2024-06-13T07:07:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d479a60 0x55e21d47be50 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d4554b0 0x55e21d458a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d44f240 0x55e21d45b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a35600700 0 client.27065 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1742752320 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d454b90 0x55e21d4584d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 >> 172.21.15.7:0/1742752320 conn(0x55e21d37b170 msgr2=0x55e21d443790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 shutdown_connections 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1742752320 wait complete. 2024-06-13T07:07:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.399+0000 7f5a5d196b80 20 client.27065 trim_cache size 0 max 16384 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.437+0000 7f5a5d196b80 1 Processor -- start 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 -- start start 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d44f240 0x55e21d44ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d476050 0x55e21d44a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ae9c0 con 0x55e21d4554b0 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1ac250 con 0x55e21d476050 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.438+0000 7f5a5d196b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0725a0 con 0x55e21d44f240 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33392/0 (socket says 172.21.15.7:33392) 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1463010042 learned_addr learned my addr 172.21.15.7:0/1463010042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1463010042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d44f240 msgr2=0x55e21d44ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d44f240 0x55e21d44ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1463010042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d476050 msgr2=0x55e21d44a1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d476050 0x55e21d44a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.439+0000 7f5a4aa00700 1 -- 172.21.15.7:0/1463010042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21d4582a0 con 0x55e21d4554b0 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.440+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c5b0 tx=0x7f5a380051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6742cae50755059 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.440+0000 7f5a37e00700 1 -- 172.21.15.7:0/1463010042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a4710 con 0x55e21d4554b0 2024-06-13T07:07:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.441+0000 7f5a37e00700 1 -- 172.21.15.7:0/1463010042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a380b4070 con 0x55e21d4554b0 2024-06-13T07:07:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.441+0000 7f5a37e00700 1 -- 172.21.15.7:0/1463010042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3809aef0 con 0x55e21d4554b0 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.442+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1463010042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 msgr2=0x55e21d458800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.442+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c5b0 tx=0x7f5a380051c0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.443+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1463010042 shutdown_connections 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.443+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d476050 0x55e21d44a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.443+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d44f240 0x55e21d44ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.443+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/1463010042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d458800 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.443+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1463010042 >> 172.21.15.7:0/1463010042 conn(0x55e21d37b170 msgr2=0x55e21d449a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.445+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1463010042 shutdown_connections 2024-06-13T07:07:28.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.445+0000 7f5a5d196b80 1 -- 172.21.15.7:0/1463010042 wait complete. 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.446+0000 7f5a5d196b80 1 -- start start 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.446+0000 7f5a5d196b80 1 Processor -- start 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.447+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.447+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d472a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.447+0000 7f5a5d196b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d476050 0x55e21d480d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.448+0000 7f5a4b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.448+0000 7f5a4b400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52348/0 (socket says 172.21.15.7:52348) 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.449+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c072c10 con 0x55e21d4554b0 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.449+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c074080 con 0x55e21d44f240 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.449+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c0732e0 con 0x55e21d476050 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.449+0000 7f5a4b400700 1 -- 172.21.15.7:0/4065536869 learned_addr learned my addr 172.21.15.7:0/4065536869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.449+0000 7f5a5d196b80 10 client.0 _ll_register_callbacks cb 0x55e21d336380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4be00700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d476050 0x55e21d480d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d472a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4b400700 1 -- 172.21.15.7:0/4065536869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d476050 msgr2=0x55e21d480d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4b400700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d476050 0x55e21d480d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4b400700 1 -- 172.21.15.7:0/4065536869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 msgr2=0x55e21d472a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4b400700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d472a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.450+0000 7f5a4b400700 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21d4815b0 con 0x55e21d44f240 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.451+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d472a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.451+0000 7f5a4b400700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004e00 tx=0x7f5a3c00fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8e24a847db3f5d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.451+0000 7f5a35600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.452+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c039070 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.452+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a380b4070 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.452+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5a3c028040 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.452+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c005390 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.452+0000 7f5a5d196b80 10 client.27068 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.453+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e21d4f5270 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.453+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a380b4070 con 0x55e21d44f240 2024-06-13T07:07:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.454+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f5a3c035450 con 0x55e21d44f240 2024-06-13T07:07:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 10 client.27068 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 10 client.27068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 20 client.27068 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d4f5550 0x55e21d4f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 4 client.27068 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:07:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e21d222250 con 0x55e21d4f5550 2024-06-13T07:07:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.457+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d4f5550 0x55e21d4f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.458+0000 7f5a4aa00700 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d4f5550 0x55e21d4f7940 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706b0f5347cbe432 server_cookie=d736283b6c4180f1 in_seq=0 out_seq=0 2024-06-13T07:07:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.458+0000 7f5a35600700 10 client.27068 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.458+0000 7f5a35600700 1 -- 172.21.15.7:0/4065536869 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e21d222250 con 0x55e21d4f5550 2024-06-13T07:07:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.458+0000 7f5a35600700 10 client.27068 handle_command_reply: tid=0 2024-06-13T07:07:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.458+0000 7f5a5d196b80 1 client.27068 shutdown 2024-06-13T07:07:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 msgr2=0x55e21d4724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004e00 tx=0x7f5a3c00fb50 comp rx=0 tx=0).stop 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 shutdown_connections 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55e21d4f5550 0x55e21d4f7940 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21d476050 0x55e21d480d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e21d44f240 0x55e21d4724e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 --2- 172.21.15.7:0/4065536869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e21d4554b0 0x55e21d472a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a35600700 0 client.27068 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 >> 172.21.15.7:0/4065536869 conn(0x55e21d37b170 msgr2=0x55e21d455930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 shutdown_connections 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 1 -- 172.21.15.7:0/4065536869 wait complete. 2024-06-13T07:07:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.459+0000 7f5a5d196b80 20 client.27068 trim_cache size 0 max 16384 2024-06-13T07:07:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:07:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 msgr2=0x7f5a300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 0x7f5a300a51b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009680 tx=0x7f5a3c0058e0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 msgr2=0x7f5a4c10cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9c0 tx=0x7f5a3800ed80 comp rx=0 tx=0).stop 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 shutdown_connections 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a4c108500 0x7f5a4c10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c06a170 0x7f5a4c10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5a300a2cf0 0x7f5a300a51b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 --2- 172.21.15.7:0/2979274961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c06ab30 0x7f5a4c10cb60 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9c0 tx=0x7f5a3800ed80 comp rx=0 tx=0).stop 2024-06-13T07:07:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.461+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 >> 172.21.15.7:0/2979274961 conn(0x7f5a4c078340 msgr2=0x7f5a4c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.462+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 shutdown_connections 2024-06-13T07:07:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:28.462+0000 7f5a52800700 1 -- 172.21.15.7:0/2979274961 wait complete. 2024-06-13T07:07:29.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:29 smithi189 ceph-mon[36616]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 422 op/s 2024-06-13T07:07:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:29 smithi175 ceph-mon[36886]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 422 op/s 2024-06-13T07:07:29.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:29 smithi007 ceph-mon[32077]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 422 op/s 2024-06-13T07:07:31.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:07:31.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:07:31.498 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:31 smithi007 ceph-mon[32077]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 483 op/s 2024-06-13T07:07:31.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:31 smithi189 ceph-mon[36616]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 483 op/s 2024-06-13T07:07:31.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:31 smithi175 ceph-mon[36886]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 483 op/s 2024-06-13T07:07:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 690.0 s, 467.0 tps, lat 213.429 ms stddev 291.477 2024-06-13T07:07:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 -- 172.21.15.7:0/1446647862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c01035e0 msgr2=0x7fb0c0109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/1446647862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c01035e0 0x7fb0c0109e70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009ca0 tx=0x7fb0b802a8b0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 -- 172.21.15.7:0/1446647862 shutdown_connections 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/1446647862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c01035e0 0x7fb0c0109e70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/1446647862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/1446647862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c0101a30 0x7fb0c0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 -- 172.21.15.7:0/1446647862 >> 172.21.15.7:0/1446647862 conn(0x7fb0c00fcfc0 msgr2=0x7fb0c00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 -- 172.21.15.7:0/1446647862 shutdown_connections 2024-06-13T07:07:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.313+0000 7fb0c5e00640 1 -- 172.21.15.7:0/1446647862 wait complete. 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.314+0000 7fb0c5e00640 1 Processor -- start 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.314+0000 7fb0c5e00640 1 -- start start 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.314+0000 7fb0c5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 0x7fb0c01077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0c5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0c5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c01035e0 0x7fb0c0104320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0c5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010d6e0 con 0x7fb0c0102c20 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0c5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010d560 con 0x7fb0c01035e0 2024-06-13T07:07:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0c5e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c010d860 con 0x7fb0c0101a30 2024-06-13T07:07:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0be000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 0x7fb0c01077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0be000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41182/0 (socket says 172.21.15.7:41182) 2024-06-13T07:07:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0be000640 1 -- 172.21.15.7:0/720768596 learned_addr learned my addr 172.21.15.7:0/720768596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.315+0000 7fb0bf400640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c01035e0 0x7fb0c0104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 -- 172.21.15.7:0/720768596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 msgr2=0x7fb0c01077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 0x7fb0c01077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 -- 172.21.15.7:0/720768596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c01035e0 msgr2=0x7fb0c0104320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c01035e0 0x7fb0c0104320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 -- 172.21.15.7:0/720768596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c0104be0 con 0x7fb0c0102c20 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0bea00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 0x7fb0c01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0be000640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d950 tx=0x7fb0ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.316+0000 7fb0c4a00640 1 -- 172.21.15.7:0/720768596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac017070 con 0x7fb0c0102c20 2024-06-13T07:07:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.317+0000 7fb0c4a00640 1 -- 172.21.15.7:0/720768596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac009c30 con 0x7fb0c0102c20 2024-06-13T07:07:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.317+0000 7fb0c4a00640 1 -- 172.21.15.7:0/720768596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00f6b0 con 0x7fb0c0102c20 2024-06-13T07:07:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.317+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c01a6250 con 0x7fb0c0102c20 2024-06-13T07:07:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.317+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c01a6660 con 0x7fb0c0102c20 2024-06-13T07:07:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 Processor -- start 2024-06-13T07:07:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 -- start start 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c02d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c02fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d03bf8b420 con 0x55d03c02d760 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d03bf32c50 con 0x55d03c0285f0 2024-06-13T07:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.321+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d03bf873e0 con 0x55d03c027c00 2024-06-13T07:07:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52624/0 (socket says 172.21.15.7:52624) 2024-06-13T07:07:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 -- 172.21.15.7:0/3250334773 learned_addr learned my addr 172.21.15.7:0/3250334773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bea00640 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c02d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 -- 172.21.15.7:0/3250334773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 msgr2=0x55d03c02d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c02d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 -- 172.21.15.7:0/3250334773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 msgr2=0x55d03c02fbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c02fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 -- 172.21.15.7:0/3250334773 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03bc9f210 con 0x55d03c027c00 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bea00640 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c02d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.322+0000 7fb0bf400640 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009ca0 tx=0x7fb0b8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3ab3ed330bf05b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.323+0000 7fb0abe00640 1 -- 172.21.15.7:0/3250334773 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8030a30 con 0x55d03c027c00 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.323+0000 7fb0abe00640 1 -- 172.21.15.7:0/3250334773 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8036030 con 0x55d03c027c00 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.323+0000 7fb0abe00640 1 -- 172.21.15.7:0/3250334773 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8004600 con 0x55d03c027c00 2024-06-13T07:07:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.323+0000 7fb0c4a00640 1 -- 172.21.15.7:0/720768596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb0ac0109d0 con 0x7fb0c0102c20 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3250334773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 msgr2=0x55d03c028020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009ca0 tx=0x7fb0b8009a20 comp rx=0 tx=0).stop 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3250334773 shutdown_connections 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c02fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c02d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3250334773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c028020 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3250334773 >> 172.21.15.7:0/3250334773 conn(0x55d03bfeffb0 msgr2=0x55d03c023680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3250334773 shutdown_connections 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3250334773 wait complete. 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.324+0000 7fb0c4a00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 0x7fb0a40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.325+0000 7fb0c4a00640 1 -- 172.21.15.7:0/720768596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb0ac0ce820 con 0x7fb0c0102c20 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.325+0000 7fb0c8b75740 1 -- start start 2024-06-13T07:07:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.325+0000 7fb0bea00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 0x7fb0a40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.325+0000 7fb0c8b75740 1 Processor -- start 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0bea00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 0x7fb0a40a5100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55d03c024530 tx=0x7fb0b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c031fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c039520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d03c0306b0 con 0x55d03c02d760 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d03bf460b0 con 0x55d03c0285f0 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d03c030830 con 0x55d03c027c00 2024-06-13T07:07:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0c8b75740 10 client.0 _ll_register_callbacks cb 0x55d03bf31f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c031fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47076/0 (socket says 172.21.15.7:47076) 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.326+0000 7fb0be000640 1 -- 172.21.15.7:0/2895340055 learned_addr learned my addr 172.21.15.7:0/2895340055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 -- 172.21.15.7:0/2895340055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 msgr2=0x55d03c031fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c031fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 -- 172.21.15.7:0/2895340055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 msgr2=0x55d03c039520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c039520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 -- 172.21.15.7:0/2895340055 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03bd6b620 con 0x55d03c0285f0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0be000640 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac09aa90 tx=0x7fb0ac009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38360d2a685d4abe server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac020050 con 0x55d03c0285f0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb0ac09a1e0 con 0x55d03c0285f0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.327+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b8036030 con 0x55d03c0285f0 2024-06-13T07:07:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.328+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac0a3950 con 0x55d03c0285f0 2024-06-13T07:07:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.328+0000 7fb0c8b75740 10 client.27080 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.328+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d03c03c6f0 con 0x55d03c0285f0 2024-06-13T07:07:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.328+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8036030 con 0x55d03c0285f0 2024-06-13T07:07:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.328+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ac096ae0 con 0x55d03c0285f0 2024-06-13T07:07:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 10 client.27080 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 10 client.27080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 20 client.27080 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c03dd80 0x55d03c0401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 4 client.27080 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.329+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03bf98b50 con 0x55d03c03dd80 2024-06-13T07:07:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.330+0000 7fb0bea00640 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c03dd80 0x55d03c0401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.331+0000 7fb0bea00640 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c03dd80 0x55d03c0401b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9ae1bbc44bd9c9 server_cookie=e8ac64e1f9789021 in_seq=0 out_seq=0 2024-06-13T07:07:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.331+0000 7fb0a8c00640 10 client.27080 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.332+0000 7fb0a8c00640 1 -- 172.21.15.7:0/2895340055 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d03bf98b50 con 0x55d03c03dd80 2024-06-13T07:07:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.332+0000 7fb0a8c00640 10 client.27080 handle_command_reply: tid=0 2024-06-13T07:07:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.333+0000 7fb0c8b75740 1 client.27080 shutdown 2024-06-13T07:07:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 msgr2=0x55d03c0324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac09aa90 tx=0x7fb0ac009de0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 shutdown_connections 2024-06-13T07:07:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c03dd80 0x55d03c0401b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c039520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0324e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/2895340055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c031fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 >> 172.21.15.7:0/2895340055 conn(0x55d03bfeffb0 msgr2=0x55d03c022840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.334+0000 7fb0a8c00640 0 client.27080 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.335+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 shutdown_connections 2024-06-13T07:07:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.335+0000 7fb0c8b75740 1 -- 172.21.15.7:0/2895340055 wait complete. 2024-06-13T07:07:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.335+0000 7fb0c8b75740 20 client.27080 trim_cache size 0 max 16384 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.362+0000 7fb0c8b75740 1 Processor -- start 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 -- start start 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 0x55d03c023810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 0x55d03c026570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b3350 con 0x55d03c027c00 2024-06-13T07:07:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b0c20 con 0x55d03c0285f0 2024-06-13T07:07:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01aea70 con 0x55d03c02d760 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47088/0 (socket says 172.21.15.7:47088) 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.363+0000 7fb0be000640 1 -- 172.21.15.7:0/3018532253 learned_addr learned my addr 172.21.15.7:0/3018532253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0bf400640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 0x55d03c026570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0bea00640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 0x55d03c023810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 -- 172.21.15.7:0/3018532253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 msgr2=0x55d03c026570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 0x55d03c026570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 -- 172.21.15.7:0/3018532253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 msgr2=0x55d03c023810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 0x55d03c023810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 -- 172.21.15.7:0/3018532253 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03c028000 con 0x55d03c0285f0 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0bf400640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 0x55d03c026570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0bea00640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 0x55d03c023810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0be000640 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0971f0 tx=0x7fb0ac005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec52a75a9bac474 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0ab400640 1 -- 172.21.15.7:0/3018532253 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac014030 con 0x55d03c0285f0 2024-06-13T07:07:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0ab400640 1 -- 172.21.15.7:0/3018532253 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac097400 con 0x55d03c0285f0 2024-06-13T07:07:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.364+0000 7fb0ab400640 1 -- 172.21.15.7:0/3018532253 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac020070 con 0x55d03c0285f0 2024-06-13T07:07:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.366+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3018532253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 msgr2=0x55d03c0394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.366+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0971f0 tx=0x7fb0ac005a90 comp rx=0 tx=0).stop 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3018532253 shutdown_connections 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c02d760 0x55d03c026570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c0394e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/3018532253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c027c00 0x55d03c023810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3018532253 >> 172.21.15.7:0/3018532253 conn(0x55d03bfeffb0 msgr2=0x55d03c023100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.367+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3018532253 shutdown_connections 2024-06-13T07:07:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.368+0000 7fb0c8b75740 1 -- 172.21.15.7:0/3018532253 wait complete. 2024-06-13T07:07:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.368+0000 7fb0c8b75740 1 -- start start 2024-06-13T07:07:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.369+0000 7fb0c8b75740 1 Processor -- start 2024-06-13T07:07:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c0368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c037df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01af0e0 con 0x55d03c02d760 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47092/0 (socket says 172.21.15.7:47092) 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b0550 con 0x55d03c0285f0 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0be000640 1 -- 172.21.15.7:0/4020173865 learned_addr learned my addr 172.21.15.7:0/4020173865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01af7b0 con 0x55d03c027c00 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0bea00640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c0368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.370+0000 7fb0c8b75740 10 client.0 _ll_register_callbacks cb 0x7fb0ac00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 -- 172.21.15.7:0/4020173865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 msgr2=0x55d03c0368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c0368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 -- 172.21.15.7:0/4020173865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 msgr2=0x55d03c037df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c037df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03c0383e0 con 0x55d03c0285f0 2024-06-13T07:07:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0bea00640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c0368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0be000640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0a3420 tx=0x7fb0ac00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b47557a635cc28b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac014070 con 0x55d03c0285f0 2024-06-13T07:07:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ac097400 con 0x55d03c0285f0 2024-06-13T07:07:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.371+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb0ac09a3e0 con 0x55d03c0285f0 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac080050 con 0x55d03c0285f0 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 10 client.27086 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d03c039640 con 0x55d03c0285f0 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac097400 con 0x55d03c0285f0 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ac085440 con 0x55d03c0285f0 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 10 client.27086 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 10 client.27086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 20 client.27086 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c0a8d10 0x55d03c0ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 4 client.27086 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.372+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d03c031d90 con 0x55d03c0a8d10 2024-06-13T07:07:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.373+0000 7fb0bea00640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c0a8d10 0x55d03c0ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.380+0000 7fb0bea00640 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c0a8d10 0x55d03c0ab100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cc19aa72256ab0 server_cookie=2f38be93afed44df in_seq=0 out_seq=0 2024-06-13T07:07:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.380+0000 7fb0a8c00640 10 client.27086 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.524+0000 7fb0a8c00640 1 -- 172.21.15.7:0/4020173865 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d03c031d90 con 0x55d03c0a8d10 2024-06-13T07:07:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.524+0000 7fb0a8c00640 10 client.27086 handle_command_reply: tid=0 2024-06-13T07:07:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.524+0000 7fb0c8b75740 1 client.27086 shutdown 2024-06-13T07:07:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.524+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 msgr2=0x55d03c036de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.524+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0a3420 tx=0x7fb0ac00bf60 comp rx=0 tx=0).stop 2024-06-13T07:07:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 shutdown_connections 2024-06-13T07:07:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d03c0a8d10 0x55d03c0ab100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d03c02d760 0x55d03c037df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d03c0285f0 0x55d03c036de0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 --2- 172.21.15.7:0/4020173865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d03c027c00 0x55d03c0368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 >> 172.21.15.7:0/4020173865 conn(0x55d03bfeffb0 msgr2=0x55d03c026570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0a8c00640 0 client.27086 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 shutdown_connections 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.525+0000 7fb0c8b75740 1 -- 172.21.15.7:0/4020173865 wait complete. 2024-06-13T07:07:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.526+0000 7fb0c8b75740 20 client.27086 trim_cache size 0 max 16384 2024-06-13T07:07:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:07:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:07:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 msgr2=0x7fb0a40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 0x7fb0a40a5100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55d03c024530 tx=0x7fb0b0005f90 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 msgr2=0x7fb0c0107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d950 tx=0x7fb0ac00de10 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 shutdown_connections 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb0a40a2c40 0x7fb0a40a5100 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0c01035e0 0x7fb0c0104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0c0102c20 0x7fb0c0107cf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 --2- 172.21.15.7:0/720768596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c0101a30 0x7fb0c01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 >> 172.21.15.7:0/720768596 conn(0x7fb0c00fcfc0 msgr2=0x7fb0c0073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 shutdown_connections 2024-06-13T07:07:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:33.527+0000 7fb0c5e00640 1 -- 172.21.15.7:0/720768596 wait complete. 2024-06-13T07:07:33.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:33 smithi189 ceph-mon[36616]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-06-13T07:07:33.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:33 smithi175 ceph-mon[36886]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-06-13T07:07:33.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:33 smithi007 ceph-mon[32077]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-06-13T07:07:35.436 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:35 smithi007 ceph-mon[32077]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-06-13T07:07:35.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:35 smithi189 ceph-mon[36616]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-06-13T07:07:35.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:35 smithi175 ceph-mon[36886]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 485 op/s 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.830+0000 7fbadcc00700 1 -- 172.21.15.7:0/2687103453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109db0 msgr2=0x7fbad81150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.830+0000 7fbad4a00700 1 -- 172.21.15.7:0/2687103453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4004230 con 0x7fbad8109db0 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.830+0000 7fbadcc00700 1 --2- 172.21.15.7:0/2687103453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109db0 0x7fbad81150a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009990 tx=0x7fbac4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 -- 172.21.15.7:0/2687103453 shutdown_connections 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 --2- 172.21.15.7:0/2687103453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad81155e0 0x7fbad81179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 --2- 172.21.15.7:0/2687103453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad8109490 0x7fbad8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 --2- 172.21.15.7:0/2687103453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109db0 0x7fbad81150a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 -- 172.21.15.7:0/2687103453 >> 172.21.15.7:0/2687103453 conn(0x7fbad806d600 msgr2=0x7fbad806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 -- 172.21.15.7:0/2687103453 shutdown_connections 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.831+0000 7fbadcc00700 1 -- 172.21.15.7:0/2687103453 wait complete. 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.833+0000 7fbadcc00700 1 Processor -- start 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.833+0000 7fbadcc00700 1 -- start start 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbadcc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109490 0x7fbad81a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbadcc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbadcc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbad5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbad5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52648/0 (socket says 172.21.15.7:52648) 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.834+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47100/0 (socket says 172.21.15.7:47100) 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.835+0000 7fbadcc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbad811b130 con 0x7fbad8109490 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.835+0000 7fbadcc00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbad811afb0 con 0x7fbad81155e0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.835+0000 7fbadcc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbad811b2b0 con 0x7fbad8109db0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.835+0000 7fbad5400700 1 -- 172.21.15.7:0/1711494676 learned_addr learned my addr 172.21.15.7:0/1711494676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad5e00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109490 0x7fbad81a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad6800700 1 -- 172.21.15.7:0/1711494676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 msgr2=0x7fbad81a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad6800700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad6800700 1 -- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109490 msgr2=0x7fbad81a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad6800700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109490 0x7fbad81a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad6800700 1 -- 172.21.15.7:0/1711494676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad81a8530 con 0x7fbad81155e0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.836+0000 7fbad5400700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbad6800700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbad000bb00 tx=0x7fbad000bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbacea00700 1 -- 172.21.15.7:0/1711494676 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0019070 con 0x7fbad81155e0 2024-06-13T07:07:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad81a8820 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad81b00a0 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbacea00700 1 -- 172.21.15.7:0/1711494676 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad0014440 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.837+0000 7fbacea00700 1 -- 172.21.15.7:0/1711494676 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0019070 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.851+0000 7fbacea00700 1 -- 172.21.15.7:0/1711494676 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbad001c070 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.853+0000 7fbacea00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 0x7fbabc0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.853+0000 7fbacea00700 1 -- 172.21.15.7:0/1711494676 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbad00ce300 con 0x7fbad81155e0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.856+0000 7fbad5e00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 0x7fbabc0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.857+0000 7fbad5e00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 0x7fbabc0a5390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005fd0 tx=0x7fbac800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.871+0000 7fbae7616b80 1 Processor -- start 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.871+0000 7fbae7616b80 1 -- start start 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b2bfbea90 con 0x563b2c02bec0 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b2c001b80 con 0x563b2c0ba380 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.872+0000 7fbae7616b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b2bfbc970 con 0x563b2c0af630 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47114/0 (socket says 172.21.15.7:47114) 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 -- 172.21.15.7:0/952473313 learned_addr learned my addr 172.21.15.7:0/952473313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad5400700 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad5e00700 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 -- 172.21.15.7:0/952473313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 msgr2=0x563b2c0afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 -- 172.21.15.7:0/952473313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 msgr2=0x563b2c0af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.873+0000 7fbad6800700 1 -- 172.21.15.7:0/952473313 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2bfaa010 con 0x563b2c0ba380 2024-06-13T07:07:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.875+0000 7fbad6800700 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbad00a2f60 tx=0x7fbad009a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3c5a44ffa4bdcb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.875+0000 7fbac3400700 1 -- 172.21.15.7:0/952473313 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a3490 con 0x563b2c0ba380 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.875+0000 7fbac3400700 1 -- 172.21.15.7:0/952473313 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad00988c0 con 0x563b2c0ba380 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.876+0000 7fbac3400700 1 -- 172.21.15.7:0/952473313 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a3490 con 0x563b2c0ba380 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.877+0000 7fbae7616b80 1 -- 172.21.15.7:0/952473313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 msgr2=0x563b2c0bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.877+0000 7fbae7616b80 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbad00a2f60 tx=0x7fbad009a2b0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.880+0000 7fbae7616b80 1 -- 172.21.15.7:0/952473313 shutdown_connections 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0bc770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 --2- 172.21.15.7:0/952473313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 -- 172.21.15.7:0/952473313 >> 172.21.15.7:0/952473313 conn(0x563b2bfdc1f0 msgr2=0x563b2c0ab530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 -- 172.21.15.7:0/952473313 shutdown_connections 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.882+0000 7fbae7616b80 1 -- 172.21.15.7:0/952473313 wait complete. 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.883+0000 7fbae7616b80 1 -- start start 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.883+0000 7fbae7616b80 1 Processor -- start 2024-06-13T07:07:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b2bf9dfe0 con 0x563b2c0ba380 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b2bffdb40 con 0x563b2c0af630 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b2c0bd040 con 0x563b2c02bec0 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbae7616b80 10 client.0 _ll_register_callbacks cb 0x563b2bf9b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbad5400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbad5400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47118/0 (socket says 172.21.15.7:47118) 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.884+0000 7fbad5400700 1 -- 172.21.15.7:0/3757779158 learned_addr learned my addr 172.21.15.7:0/3757779158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad5400700 1 -- 172.21.15.7:0/3757779158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 msgr2=0x563b2c0b9a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad6800700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad5400700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0b9a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad5400700 1 -- 172.21.15.7:0/3757779158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 msgr2=0x563b2c0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad5400700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad5400700 1 -- 172.21.15.7:0/3757779158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2bf50950 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.885+0000 7fbad6800700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.886+0000 7fbab7e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.887+0000 7fbad5400700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b5c0 tx=0x7fbac40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159aee5e86c39aa1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.888+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40056c0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.888+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad00988c0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.888+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbac400b7d0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbac400b9d0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbae7616b80 10 client.27095 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563b2c0c77e0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad00988c0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbac40223f0 con 0x563b2c0af630 2024-06-13T07:07:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbae7616b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.889+0000 7fbae7616b80 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.890+0000 7fbae7616b80 20 client.27095 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.890+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0ca0d0 0x563b2c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.890+0000 7fbae7616b80 4 client.27095 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.890+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b2c00f080 con 0x563b2c0ca0d0 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.890+0000 7fbad5e00700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0ca0d0 0x563b2c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.892+0000 7fbad5e00700 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0ca0d0 0x563b2c0cc4c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e2c468fe1138b3 server_cookie=f6fb23ad67dde821 in_seq=0 out_seq=0 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.892+0000 7fbab7e00700 10 client.27095 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.893+0000 7fbab7e00700 1 -- 172.21.15.7:0/3757779158 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563b2c00f080 con 0x563b2c0ca0d0 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.893+0000 7fbab7e00700 10 client.27095 handle_command_reply: tid=0 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.893+0000 7fbae7616b80 1 client.27095 shutdown 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 msgr2=0x563b2c0b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b5c0 tx=0x7fbac40096a0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 shutdown_connections 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0ca0d0 0x563b2c0cc4c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 --2- 172.21.15.7:0/3757779158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0b6d90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 >> 172.21.15.7:0/3757779158 conn(0x563b2bfdc1f0 msgr2=0x563b2c0aa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.894+0000 7fbab7e00700 0 client.27095 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.895+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 shutdown_connections 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.895+0000 7fbae7616b80 1 -- 172.21.15.7:0/3757779158 wait complete. 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.895+0000 7fbae7616b80 20 client.27095 trim_cache size 0 max 16384 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.936+0000 7fbae7616b80 1 Processor -- start 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 -- start start 2024-06-13T07:07:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbab80112f0 con 0x563b2c0ba380 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbab800eb80 con 0x563b2c0af630 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.939+0000 7fbae7616b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbab800c970 con 0x563b2c02bec0 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52680/0 (socket says 172.21.15.7:52680) 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 -- 172.21.15.7:0/277311903 learned_addr learned my addr 172.21.15.7:0/277311903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5400700 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 -- 172.21.15.7:0/277311903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 msgr2=0x563b2c0aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 -- 172.21.15.7:0/277311903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 msgr2=0x563b2c0bb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.940+0000 7fbad5e00700 1 -- 172.21.15.7:0/277311903 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2c0ccb40 con 0x563b2c02bec0 2024-06-13T07:07:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.941+0000 7fbad5e00700 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x563b2c0ac430 tx=0x7fbac8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d404e9bfcd0c3b78 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.942+0000 7fbac2a00700 1 -- 172.21.15.7:0/277311903 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800eda0 con 0x563b2c02bec0 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.942+0000 7fbac2a00700 1 -- 172.21.15.7:0/277311903 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac802b070 con 0x563b2c02bec0 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.942+0000 7fbac2a00700 1 -- 172.21.15.7:0/277311903 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8025830 con 0x563b2c02bec0 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 -- 172.21.15.7:0/277311903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 msgr2=0x563b2c0ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x563b2c0ac430 tx=0x7fbac8014000 comp rx=0 tx=0).stop 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 -- 172.21.15.7:0/277311903 shutdown_connections 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c0ba380 0x563b2c0bb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c02bec0 0x563b2c0ab6a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 --2- 172.21.15.7:0/277311903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0af630 0x563b2c0aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 -- 172.21.15.7:0/277311903 >> 172.21.15.7:0/277311903 conn(0x563b2bfdc1f0 msgr2=0x563b2c0a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 -- 172.21.15.7:0/277311903 shutdown_connections 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.943+0000 7fbae7616b80 1 -- 172.21.15.7:0/277311903 wait complete. 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.944+0000 7fbae7616b80 1 -- start start 2024-06-13T07:07:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.944+0000 7fbae7616b80 1 Processor -- start 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.945+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.946+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.946+0000 7fbae7616b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.946+0000 7fbae7616b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbab800d040 con 0x563b2c02bec0 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:47132/0 (socket says 172.21.15.7:47132) 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad6800700 1 -- 172.21.15.7:0/2799507271 learned_addr learned my addr 172.21.15.7:0/2799507271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbab800e4b0 con 0x563b2c0ba380 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbab800d710 con 0x563b2c0af630 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbae7616b80 10 client.0 _ll_register_callbacks cb 0x563b2bf9b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad6800700 1 -- 172.21.15.7:0/2799507271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 msgr2=0x563b2c0c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.947+0000 7fbad5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:52694/0 (socket says 172.21.15.7:52694) 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad6800700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad6800700 1 -- 172.21.15.7:0/2799507271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 msgr2=0x563b2c0c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad6800700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad6800700 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2c0b3610 con 0x563b2c0ba380 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad6800700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbad00971f0 tx=0x7fbad0096be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d261bd80d80897 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbab7e00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbad5400700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.948+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad009aa80 con 0x563b2c0ba380 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.949+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbad009ac20 con 0x563b2c0ba380 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.949+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac802b070 con 0x563b2c0ba380 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.949+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbad00a3000 con 0x563b2c0ba380 2024-06-13T07:07:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.949+0000 7fbae7616b80 10 client.27101 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.949+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563b2c0b4910 con 0x563b2c0ba380 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac802b070 con 0x563b2c0ba380 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbad009bce0 con 0x563b2c0ba380 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 10 client.27101 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 20 client.27101 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0b4b40 0x563b2c137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 4 client.27101 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b2c009600 con 0x563b2c0b4b40 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.950+0000 7fbad5e00700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0b4b40 0x563b2c137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.952+0000 7fbad5e00700 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0b4b40 0x563b2c137dc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214170a5198fa41b server_cookie=76714a1035a79c02 in_seq=0 out_seq=0 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.952+0000 7fbab7e00700 10 client.27101 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.956+0000 7fbab7e00700 1 -- 172.21.15.7:0/2799507271 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b2c009600 con 0x563b2c0b4b40 2024-06-13T07:07:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.956+0000 7fbab7e00700 10 client.27101 handle_command_reply: tid=0 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.956+0000 7fbae7616b80 1 client.27101 shutdown 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.956+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 msgr2=0x563b2c0b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.956+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbad00971f0 tx=0x7fbad0096be0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 shutdown_connections 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563b2c0b4b40 0x563b2c137dc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b2c0ba380 0x563b2c0b3020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "39a6d39c-90c3-44c6-9c4c-6c809e4e4f45", 2024-06-13T07:07:36.185 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b2c02bec0 0x563b2c0c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbab7e00700 0 client.27101 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 --2- 172.21.15.7:0/2799507271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b2c0af630 0x563b2c0c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 >> 172.21.15.7:0/2799507271 conn(0x563b2bfdc1f0 msgr2=0x563b2c0b8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 shutdown_connections 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 1 -- 172.21.15.7:0/2799507271 wait complete. 2024-06-13T07:07:36.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.957+0000 7fbae7616b80 20 client.27101 trim_cache size 0 max 16384 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.961+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 msgr2=0x7fbabc0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.961+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 0x7fbabc0a5390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005fd0 tx=0x7fbac800a560 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.961+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 msgr2=0x7fbad81a7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.961+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbad000bb00 tx=0x7fbad000bec0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 shutdown_connections 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbabc0a2ed0 0x7fbabc0a5390 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad81155e0 0x7fbad81a7df0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad8109490 0x7fbad81a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 --2- 172.21.15.7:0/1711494676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbad8109db0 0x7fbad81a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 >> 172.21.15.7:0/1711494676 conn(0x7fbad806d600 msgr2=0x7fbad810b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 shutdown_connections 2024-06-13T07:07:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:35.963+0000 7fbadcc00700 1 -- 172.21.15.7:0/1711494676 wait complete. 2024-06-13T07:07:37.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:07:37.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:37 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-06-13T07:07:37.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:37 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 410 op/s 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:37 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:07:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 695.0 s, 463.0 tps, lat 214.969 ms stddev 259.540 2024-06-13T07:07:38.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:38 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:38.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:38 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:38.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:38 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:07:39.579 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:39 smithi007 ceph-mon[32077]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-06-13T07:07:39.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:39 smithi189 ceph-mon[36616]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-06-13T07:07:39.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:39 smithi175 ceph-mon[36886]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-06-13T07:07:41.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:41 smithi007 ceph-mon[32077]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-13T07:07:41.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:41 smithi189 ceph-mon[36616]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-13T07:07:41.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:41 smithi175 ceph-mon[36886]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.743+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1061142874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380a5b70 msgr2=0x7fbb380a9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.743+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1061142874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380a5b70 0x7fbb380a9bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0099c0 tx=0x7fbb2c009cd0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1061142874 shutdown_connections 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1061142874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380aa2a0 0x7fbb380b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1061142874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a51c0 0x7fbb380a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1061142874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380a5b70 0x7fbb380a9bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1061142874 >> 172.21.15.7:0/1061142874 conn(0x7fbb3801a690 msgr2=0x7fbb3801aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1061142874 shutdown_connections 2024-06-13T07:07:41.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.744+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1061142874 wait complete. 2024-06-13T07:07:41.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.746+0000 7fbb3fe00700 1 Processor -- start 2024-06-13T07:07:41.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.746+0000 7fbb3fe00700 1 -- start start 2024-06-13T07:07:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 0x7fbb380adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380aa2a0 0x7fbb3814f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52860/0 (socket says 172.21.15.7:52860) 2024-06-13T07:07:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.747+0000 7fbb3e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 0x7fbb380adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3ea00700 1 -- 172.21.15.7:0/1439032261 learned_addr learned my addr 172.21.15.7:0/1439032261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380b4ef0 con 0x7fbb380aa2a0 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380b4d70 con 0x7fbb380a51c0 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380b5070 con 0x7fbb380a5b70 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380aa2a0 0x7fbb3814f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3e000700 1 -- 172.21.15.7:0/1439032261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 msgr2=0x7fbb380ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3e000700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3e000700 1 -- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380aa2a0 msgr2=0x7fbb3814f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3e000700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380aa2a0 0x7fbb3814f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3e000700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3814fc30 con 0x7fbb380a5b70 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.748+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb3e000700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 0x7fbb380adba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098e0 tx=0x7fbb2c00b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb37400700 1 -- 172.21.15.7:0/1439032261 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c027070 con 0x7fbb380a5b70 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb37400700 1 -- 172.21.15.7:0/1439032261 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c0056d0 con 0x7fbb380a5b70 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3814fec0 con 0x7fbb380a5b70 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb37400700 1 -- 172.21.15.7:0/1439032261 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00f6c0 con 0x7fbb380a5b70 2024-06-13T07:07:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.749+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381502c0 con 0x7fbb380a5b70 2024-06-13T07:07:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.763+0000 7fbb37400700 1 -- 172.21.15.7:0/1439032261 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbb2c02a070 con 0x7fbb380a5b70 2024-06-13T07:07:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.766+0000 7fbb50463b80 1 Processor -- start 2024-06-13T07:07:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.766+0000 7fbb50463b80 1 -- start start 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.766+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.766+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb50463b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644242cfa10 con 0x5644243bfd60 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb50463b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564424312b00 con 0x5644243c0680 2024-06-13T07:07:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb50463b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644242cd8f0 con 0x5644243cb530 2024-06-13T07:07:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb37400700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 0x7fbb240a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.767+0000 7fbb3e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.768+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.768+0000 7fbb37400700 1 -- 172.21.15.7:0/1439032261 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbb2c0cfef0 con 0x7fbb380a5b70 2024-06-13T07:07:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.769+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34746/0 (socket says 172.21.15.7:34746) 2024-06-13T07:07:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.769+0000 7fbb3f400700 1 -- 172.21.15.7:0/490993538 learned_addr learned my addr 172.21.15.7:0/490993538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 -- 172.21.15.7:0/490993538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 msgr2=0x5644243caff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:07:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243caff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 -- 172.21.15.7:0/490993538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 msgr2=0x5644243c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 -- 172.21.15.7:0/490993538 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644242baf90 con 0x5644243cb530 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3e000700 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243cd920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0cd700 tx=0x7fbb2c098960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8612266e0aebbadd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.770+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 0x7fbb240a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.771+0000 7fbb2be00700 1 -- 172.21.15.7:0/490993538 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c003d00 con 0x5644243cb530 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.771+0000 7fbb2be00700 1 -- 172.21.15.7:0/490993538 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c09ccc0 con 0x5644243cb530 2024-06-13T07:07:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.771+0000 7fbb2be00700 1 -- 172.21.15.7:0/490993538 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c003d00 con 0x5644243cb530 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.772+0000 7fbb50463b80 1 -- 172.21.15.7:0/490993538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 msgr2=0x5644243cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.772+0000 7fbb50463b80 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243cd920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0cd700 tx=0x7fbb2c098960 comp rx=0 tx=0).stop 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.773+0000 7fbb50463b80 1 -- 172.21.15.7:0/490993538 shutdown_connections 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.773+0000 7fbb50463b80 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243cd920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.773+0000 7fbb50463b80 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.773+0000 7fbb50463b80 1 --2- 172.21.15.7:0/490993538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.773+0000 7fbb50463b80 1 -- 172.21.15.7:0/490993538 >> 172.21.15.7:0/490993538 conn(0x5644242ed170 msgr2=0x5644243bc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.774+0000 7fbb50463b80 1 -- 172.21.15.7:0/490993538 shutdown_connections 2024-06-13T07:07:41.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.774+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 0x7fbb240a51a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb40065600 tx=0x7fbb40068eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.776+0000 7fbb50463b80 1 -- 172.21.15.7:0/490993538 wait complete. 2024-06-13T07:07:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.776+0000 7fbb50463b80 1 -- start start 2024-06-13T07:07:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.777+0000 7fbb50463b80 1 Processor -- start 2024-06-13T07:07:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.777+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243bfd60 0x5644243caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.778+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.778+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.778+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.778+0000 7fbb3ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243bfd60 0x5644243caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33452/0 (socket says 172.21.15.7:33452) 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb3f400700 1 -- 172.21.15.7:0/1935984099 learned_addr learned my addr 172.21.15.7:0/1935984099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644242aef60 con 0x5644243c0680 2024-06-13T07:07:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56442430eac0 con 0x5644243bfd60 2024-06-13T07:07:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5644242cd8f0 con 0x5644243cb530 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb50463b80 10 client.0 _ll_register_callbacks cb 0x5644242a8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.779+0000 7fbb3e000700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.780+0000 7fbb3f400700 1 -- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243bfd60 msgr2=0x5644243caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.780+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243bfd60 0x5644243caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.780+0000 7fbb3f400700 1 -- 172.21.15.7:0/1935984099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 msgr2=0x5644243c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.781+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.781+0000 7fbb3f400700 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644242618d0 con 0x5644243cb530 2024-06-13T07:07:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.783+0000 7fbb3e000700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.783+0000 7fbb3f400700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb4002e2c0 tx=0x7fbb40068610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f63cfe4463dc00e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.783+0000 7fbb28c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.785+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4007b2d0 con 0x5644243cb530 2024-06-13T07:07:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.785+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbb40077730 con 0x5644243cb530 2024-06-13T07:07:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.785+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb2c09ccc0 con 0x5644243cb530 2024-06-13T07:07:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.787+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbb40077930 con 0x5644243cb530 2024-06-13T07:07:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.787+0000 7fbb50463b80 10 client.27019 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.787+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5644243d84c0 con 0x5644243cb530 2024-06-13T07:07:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.787+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c09ccc0 con 0x5644243cb530 2024-06-13T07:07:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbb40091b70 con 0x5644243cb530 2024-06-13T07:07:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 10 client.27019 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 10 client.27019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 20 client.27019 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644243daf00 0x5644243dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 4 client.27019 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.788+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564424320000 con 0x5644243daf00 2024-06-13T07:07:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.790+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644243daf00 0x5644243dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.793+0000 7fbb28c00700 10 client.27019 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.793+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644243daf00 0x5644243dd310 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80358f7d77eac02f server_cookie=70e07b8f801fe202 in_seq=0 out_seq=0 2024-06-13T07:07:41.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.796+0000 7fbb28c00700 1 -- 172.21.15.7:0/1935984099 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564424320000 con 0x5644243daf00 2024-06-13T07:07:41.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.796+0000 7fbb28c00700 10 client.27019 handle_command_reply: tid=0 2024-06-13T07:07:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.797+0000 7fbb50463b80 1 client.27019 shutdown 2024-06-13T07:07:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 msgr2=0x5644243c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb4002e2c0 tx=0x7fbb40068610 comp rx=0 tx=0).stop 2024-06-13T07:07:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 shutdown_connections 2024-06-13T07:07:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5644243daf00 0x5644243dd310 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb28c00700 0 client.27019 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243c84e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243bfd60 0x5644243caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 --2- 172.21.15.7:0/1935984099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 >> 172.21.15.7:0/1935984099 conn(0x5644242ed170 msgr2=0x5644243bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 shutdown_connections 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 1 -- 172.21.15.7:0/1935984099 wait complete. 2024-06-13T07:07:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.798+0000 7fbb50463b80 20 client.27019 trim_cache size 0 max 16384 2024-06-13T07:07:41.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.842+0000 7fbb50463b80 1 Processor -- start 2024-06-13T07:07:41.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.843+0000 7fbb50463b80 1 -- start start 2024-06-13T07:07:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.843+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.843+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb50463b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3815b6d0 con 0x5644243bfd60 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb50463b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38158f60 con 0x5644243c0680 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb50463b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38156db0 con 0x5644243cb530 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb3f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33460/0 (socket says 172.21.15.7:33460) 2024-06-13T07:07:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.844+0000 7fbb3f400700 1 -- 172.21.15.7:0/715762496 learned_addr learned my addr 172.21.15.7:0/715762496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3f400700 1 -- 172.21.15.7:0/715762496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 msgr2=0x5644243c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3f400700 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3f400700 1 -- 172.21.15.7:0/715762496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 msgr2=0x5644243c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3f400700 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.846+0000 7fbb3f400700 1 -- 172.21.15.7:0/715762496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644243dd990 con 0x5644243cb530 2024-06-13T07:07:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.847+0000 7fbb3f400700 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb40051a30 tx=0x7fbb40068b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d2ef022d4865ab server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.848+0000 7fbb2b400700 1 -- 172.21.15.7:0/715762496 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4007b0d0 con 0x5644243cb530 2024-06-13T07:07:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.848+0000 7fbb2b400700 1 -- 172.21.15.7:0/715762496 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4007be50 con 0x5644243cb530 2024-06-13T07:07:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.848+0000 7fbb2b400700 1 -- 172.21.15.7:0/715762496 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40075d00 con 0x5644243cb530 2024-06-13T07:07:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.849+0000 7fbb50463b80 1 -- 172.21.15.7:0/715762496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 msgr2=0x5644243ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.849+0000 7fbb50463b80 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb40051a30 tx=0x7fbb40068b70 comp rx=0 tx=0).stop 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 -- 172.21.15.7:0/715762496 shutdown_connections 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243cb530 0x5644243ca7e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243bfd60 0x5644243c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 --2- 172.21.15.7:0/715762496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243c0680 0x5644243c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 -- 172.21.15.7:0/715762496 >> 172.21.15.7:0/715762496 conn(0x5644242ed170 msgr2=0x5644243c35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 -- 172.21.15.7:0/715762496 shutdown_connections 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 -- 172.21.15.7:0/715762496 wait complete. 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.852+0000 7fbb50463b80 1 -- start start 2024-06-13T07:07:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.853+0000 7fbb50463b80 1 Processor -- start 2024-06-13T07:07:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243cb530 0x5644243cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38157420 con 0x5644243c0680 2024-06-13T07:07:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38158890 con 0x5644243cb530 2024-06-13T07:07:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38157af0 con 0x5644243bfd60 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.854+0000 7fbb50463b80 10 client.0 _ll_register_callbacks cb 0x5644242a8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.855+0000 7fbb3ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.855+0000 7fbb3ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33474/0 (socket says 172.21.15.7:33474) 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.855+0000 7fbb3ea00700 1 -- 172.21.15.7:0/3093569173 learned_addr learned my addr 172.21.15.7:0/3093569173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3f400700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243cb530 0x5644243cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3ea00700 1 -- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243cb530 msgr2=0x5644243cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243cb530 0x5644243cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3ea00700 1 -- 172.21.15.7:0/3093569173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 msgr2=0x5644243cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.856+0000 7fbb3ea00700 1 -- 172.21.15.7:0/3093569173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644243bd6c0 con 0x5644243bfd60 2024-06-13T07:07:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.859+0000 7fbb3ea00700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5644243ddbc0 tx=0x7fbb30014a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d211e6e3e2564ea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.859+0000 7fbb28c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:07:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.859+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30004250 con 0x5644243bfd60 2024-06-13T07:07:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.860+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbb300043f0 con 0x5644243bfd60 2024-06-13T07:07:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.860+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4007be50 con 0x5644243bfd60 2024-06-13T07:07:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.861+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbb30003680 con 0x5644243bfd60 2024-06-13T07:07:41.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 10 client.27025 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564424445e60 con 0x5644243bfd60 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4007be50 con 0x5644243bfd60 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fbb3001b830 con 0x5644243bfd60 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 10 client.27025 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 20 client.27025 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564424446570 0x564424448990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 4 client.27025 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.862+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56442431a580 con 0x564424446570 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.863+0000 7fbb3e000700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564424446570 0x564424448990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:41.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.864+0000 7fbb3e000700 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564424446570 0x564424448990 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a75805d86c98c7 server_cookie=abf72cc8b160490a in_seq=0 out_seq=0 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.864+0000 7fbb28c00700 10 client.27025 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.865+0000 7fbb28c00700 1 -- 172.21.15.7:0/3093569173 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56442431a580 con 0x564424446570 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.865+0000 7fbb28c00700 10 client.27025 handle_command_reply: tid=0 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.865+0000 7fbb50463b80 1 client.27025 shutdown 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.865+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 msgr2=0x5644243ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.865+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5644243ddbc0 tx=0x7fbb30014a30 comp rx=0 tx=0).stop 2024-06-13T07:07:41.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 shutdown_connections 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x564424446570 0x564424448990 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5644243cb530 0x5644243cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5644243bfd60 0x5644243ceda0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb28c00700 0 client.27025 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 --2- 172.21.15.7:0/3093569173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644243c0680 0x5644243cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 >> 172.21.15.7:0/3093569173 conn(0x5644242ed170 msgr2=0x5644243c2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.866+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 shutdown_connections 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.867+0000 7fbb50463b80 1 -- 172.21.15.7:0/3093569173 wait complete. 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.867+0000 7fbb50463b80 20 client.27025 trim_cache size 0 max 16384 2024-06-13T07:07:41.867 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "39a6d39c-90c3-44c6-9c4c-6c809e4e4f45": { 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "39a6d39c-90c3-44c6-9c4c-6c809e4e4f45", 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:07:41.868 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:07:41.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 msgr2=0x7fbb240a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 0x7fbb240a51a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb40065600 tx=0x7fbb40068eb0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 msgr2=0x7fbb380adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 0x7fbb380adba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098e0 tx=0x7fbb2c00b9c0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 shutdown_connections 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb380aa2a0 0x7fbb3814f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbb240a2ce0 0x7fbb240a51a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb380a51c0 0x7fbb380ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 --2- 172.21.15.7:0/1439032261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb380a5b70 0x7fbb380adba0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 >> 172.21.15.7:0/1439032261 conn(0x7fbb3801a690 msgr2=0x7fbb380a2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 shutdown_connections 2024-06-13T07:07:41.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:07:41.868+0000 7fbb3fe00700 1 -- 172.21.15.7:0/1439032261 wait complete. 2024-06-13T07:07:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 700.0 s, 555.8 tps, lat 181.020 ms stddev 240.330 2024-06-13T07:07:43.512 INFO:tasks.cephfs.filesystem:scrub status for tag:39a6d39c-90c3-44c6-9c4c-6c809e4e4f45 - {'path': '/', 'tag': '39a6d39c-90c3-44c6-9c4c-6c809e4e4f45', 'options': 'recursive,force'} 2024-06-13T07:07:43.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:43 smithi189 ceph-mon[36616]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.8 MiB/s wr, 564 op/s 2024-06-13T07:07:43.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:07:43.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:07:43.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:43 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:07:43.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:43 smithi175 ceph-mon[36886]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.8 MiB/s wr, 564 op/s 2024-06-13T07:07:43.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:07:43.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:07:43.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:43 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:07:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:43 smithi007 ceph-mon[32077]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.8 MiB/s wr, 564 op/s 2024-06-13T07:07:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:07:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:07:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:43 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:07:45.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:45 smithi175 ceph-mon[36886]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-06-13T07:07:45.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:45 smithi007 ceph-mon[32077]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-06-13T07:07:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:45 smithi189 ceph-mon[36616]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.9 MiB/s wr, 608 op/s 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 -- 172.21.15.7:0/3820219615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b60104e10 msgr2=0x7f1b60107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 --2- 172.21.15.7:0/3820219615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b60104e10 0x7f1b60107200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009860 tx=0x7f1b4c02a340 comp rx=0 tx=0).stop 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 -- 172.21.15.7:0/3820219615 shutdown_connections 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 --2- 172.21.15.7:0/3820219615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b60109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 --2- 172.21.15.7:0/3820219615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b60104e10 0x7f1b60107200 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 --2- 172.21.15.7:0/3820219615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b601024e0 0x7f1b601048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 -- 172.21.15.7:0/3820219615 >> 172.21.15.7:0/3820219615 conn(0x7f1b600fbe70 msgr2=0x7f1b600fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 -- 172.21.15.7:0/3820219615 shutdown_connections 2024-06-13T07:07:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.763+0000 7f1b68200640 1 -- 172.21.15.7:0/3820219615 wait complete. 2024-06-13T07:07:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.765+0000 7f1b68200640 1 Processor -- start 2024-06-13T07:07:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.765+0000 7f1b68200640 1 -- start start 2024-06-13T07:07:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.765+0000 7f1b68200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 0x7f1b60195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.765+0000 7f1b68200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b60104e10 0x7f1b601960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.765+0000 7f1b68200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b6019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b68200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6010f8d0 con 0x7f1b60104e10 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b68200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6010f750 con 0x7f1b601024e0 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b68200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6010fa50 con 0x7f1b60107740 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b65800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 0x7f1b60195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b65800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 0x7f1b60195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52896/0 (socket says 172.21.15.7:52896) 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b64e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b60104e10 0x7f1b601960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b65800640 1 -- 172.21.15.7:0/3680316591 learned_addr learned my addr 172.21.15.7:0/3680316591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.766+0000 7f1b66200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b6019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 -- 172.21.15.7:0/3680316591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 msgr2=0x7f1b60195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 0x7f1b60195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 -- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b60104e10 msgr2=0x7f1b601960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b60104e10 0x7f1b601960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 -- 172.21.15.7:0/3680316591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b601ae7d0 con 0x7f1b60107740 2024-06-13T07:07:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.767+0000 7f1b66200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b6019d0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800be20 tx=0x7f1b5800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.768+0000 7f1b5e000640 1 -- 172.21.15.7:0/3680316591 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58019070 con 0x7f1b60107740 2024-06-13T07:07:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.768+0000 7f1b5e000640 1 -- 172.21.15.7:0/3680316591 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b58014920 con 0x7f1b60107740 2024-06-13T07:07:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.768+0000 7f1b5e000640 1 -- 172.21.15.7:0/3680316591 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58012920 con 0x7f1b60107740 2024-06-13T07:07:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.768+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b601aea60 con 0x7f1b60107740 2024-06-13T07:07:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.769+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601aefb0 con 0x7f1b60107740 2024-06-13T07:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.778+0000 7f1b5e000640 1 -- 172.21.15.7:0/3680316591 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1b58014ad0 con 0x7f1b60107740 2024-06-13T07:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.779+0000 7f1b6affe740 1 Processor -- start 2024-06-13T07:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.779+0000 7f1b6affe740 1 -- start start 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91d9ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c91d950420 con 0x55c91d9ecc00 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c91d8f7c50 con 0x55c91d9f2760 2024-06-13T07:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b6affe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c91d94c3e0 con 0x55c91d9ed5f0 2024-06-13T07:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b65800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b65800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33496/0 (socket says 172.21.15.7:33496) 2024-06-13T07:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b65800640 1 -- 172.21.15.7:0/1668834457 learned_addr learned my addr 172.21.15.7:0/1668834457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b64e00640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.780+0000 7f1b64e00640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91d9ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 -- 172.21.15.7:0/1668834457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 msgr2=0x55c91d9f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 -- 172.21.15.7:0/1668834457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 msgr2=0x55c91d9ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91d9ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 -- 172.21.15.7:0/1668834457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91d664210 con 0x55c91d9ed5f0 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b64e00640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91d9ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b5e000640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b65800640 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004630 tx=0x7f1b540117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f8503402d48ec6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.781+0000 7f1b5e000640 1 -- 172.21.15.7:0/3680316591 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1b580cee70 con 0x7f1b60107740 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.782+0000 7f1b52a00640 1 -- 172.21.15.7:0/1668834457 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54012450 con 0x55c91d9ed5f0 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.782+0000 7f1b64e00640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.782+0000 7f1b52a00640 1 -- 172.21.15.7:0/1668834457 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b54012ad0 con 0x55c91d9ed5f0 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 -- 172.21.15.7:0/1668834457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 msgr2=0x55c91d9f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004630 tx=0x7f1b540117b0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 -- 172.21.15.7:0/1668834457 shutdown_connections 2024-06-13T07:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ed5f0 0x55c91d9f20f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 --2- 172.21.15.7:0/1668834457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91d9ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b6affe740 1 -- 172.21.15.7:0/1668834457 >> 172.21.15.7:0/1668834457 conn(0x55c91d9b4fb0 msgr2=0x55c91d9e8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.783+0000 7f1b64e00640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02a930 tx=0x7f1b4c02a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.784+0000 7f1b6affe740 1 -- 172.21.15.7:0/1668834457 shutdown_connections 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.784+0000 7f1b6affe740 1 -- 172.21.15.7:0/1668834457 wait complete. 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.784+0000 7f1b6affe740 1 -- start start 2024-06-13T07:07:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 Processor -- start 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9f2760 0x55c91d9f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c91d9f57b0 con 0x55c91d9f2760 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c91d90b0b0 con 0x55c91d9fc440 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c91d9f5930 con 0x55c91d9ecc00 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.785+0000 7f1b6affe740 10 client.0 _ll_register_callbacks cb 0x55c91d8f6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52914/0 (socket says 172.21.15.7:52914) 2024-06-13T07:07:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b64e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9f2760 0x55c91d9f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 -- 172.21.15.7:0/4280329655 learned_addr learned my addr 172.21.15.7:0/4280329655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b65800640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 -- 172.21.15.7:0/4280329655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 msgr2=0x55c91d9f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 -- 172.21.15.7:0/4280329655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9f2760 msgr2=0x55c91d9f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9f2760 0x55c91d9f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 -- 172.21.15.7:0/4280329655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91d730620 con 0x55c91d9fc440 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b66200640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b580977f0 tx=0x7f1b5809bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=653ee70574684830 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.786+0000 7f1b47400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.787+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580a3d70 con 0x55c91d9fc440 2024-06-13T07:07:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.787+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1b5809a7e0 con 0x55c91d9fc440 2024-06-13T07:07:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.788+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c91d95db50 con 0x55c91d9fc440 2024-06-13T07:07:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.788+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1b580a4000 con 0x55c91d9fc440 2024-06-13T07:07:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.788+0000 7f1b6affe740 10 client.27116 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.788+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c91da01c10 con 0x55c91d9fc440 2024-06-13T07:07:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.790+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c91d95db50 con 0x55c91d9fc440 2024-06-13T07:07:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.790+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f1b580970e0 con 0x55c91d9fc440 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 10 client.27116 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 10 client.27116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 20 client.27116 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da032d0 0x55c91da05760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 4 client.27116 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91d957ea0 con 0x55c91da032d0 2024-06-13T07:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.791+0000 7f1b65800640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da032d0 0x55c91da05760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.794+0000 7f1b65800640 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da032d0 0x55c91da05760 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe33cf25fe31b54 server_cookie=854d8a4b3630aef5 in_seq=0 out_seq=0 2024-06-13T07:07:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.794+0000 7f1b47400640 10 client.27116 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.795+0000 7f1b47400640 1 -- 172.21.15.7:0/4280329655 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c91d957ea0 con 0x55c91da032d0 2024-06-13T07:07:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.795+0000 7f1b47400640 10 client.27116 handle_command_reply: tid=0 2024-06-13T07:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.795+0000 7f1b6affe740 1 client.27116 shutdown 2024-06-13T07:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 msgr2=0x55c91d9fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1b580977f0 tx=0x7f1b5809bbd0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 shutdown_connections 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da032d0 0x55c91da05760 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91d9fe830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9f2760 0x55c91d9f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 --2- 172.21.15.7:0/4280329655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b47400640 0 client.27116 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.796+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 >> 172.21.15.7:0/4280329655 conn(0x55c91d9b4fb0 msgr2=0x55c91d9e7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.797+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 shutdown_connections 2024-06-13T07:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.797+0000 7f1b6affe740 1 -- 172.21.15.7:0/4280329655 wait complete. 2024-06-13T07:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.797+0000 7f1b6affe740 20 client.27116 trim_cache size 0 max 16384 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.829+0000 7f1b6affe740 1 Processor -- start 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 -- start start 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601bbb10 con 0x55c91d9fc440 2024-06-13T07:07:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601b93e0 con 0x55c91d9f2760 2024-06-13T07:07:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b6affe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601b7230 con 0x55c91d9ecc00 2024-06-13T07:07:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.831+0000 7f1b64e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.830+0000 7f1b65800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.831+0000 7f1b66200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.832+0000 7f1b66200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34838/0 (socket says 172.21.15.7:34838) 2024-06-13T07:07:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.832+0000 7f1b66200640 1 -- 172.21.15.7:0/3770061901 learned_addr learned my addr 172.21.15.7:0/3770061901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.832+0000 7f1b66200640 1 -- 172.21.15.7:0/3770061901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 msgr2=0x55c91d9e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.833+0000 7f1b66200640 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.833+0000 7f1b66200640 1 -- 172.21.15.7:0/3770061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 msgr2=0x55c91d9e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.833+0000 7f1b66200640 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9e8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.833+0000 7f1b66200640 1 -- 172.21.15.7:0/3770061901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91d9f62c0 con 0x55c91d9fc440 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.833+0000 7f1b65800640 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.834+0000 7f1b64e00640 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.834+0000 7f1b66200640 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1b5809b320 tx=0x7f1b58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bef99f4cf5bd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.835+0000 7f1b52000640 1 -- 172.21.15.7:0/3770061901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b4250 con 0x55c91d9fc440 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.835+0000 7f1b52000640 1 -- 172.21.15.7:0/3770061901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b580cce50 con 0x55c91d9fc440 2024-06-13T07:07:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.835+0000 7f1b52000640 1 -- 172.21.15.7:0/3770061901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5809aa40 con 0x55c91d9fc440 2024-06-13T07:07:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.836+0000 7f1b6affe740 1 -- 172.21.15.7:0/3770061901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 msgr2=0x55c91d9eb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.836+0000 7f1b6affe740 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1b5809b320 tx=0x7f1b58004a40 comp rx=0 tx=0).stop 2024-06-13T07:07:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.838+0000 7f1b6affe740 1 -- 172.21.15.7:0/3770061901 shutdown_connections 2024-06-13T07:07:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.838+0000 7f1b6affe740 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9fc440 0x55c91d9eb5e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.838+0000 7f1b6affe740 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9f2760 0x55c91d9e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.838+0000 7f1b6affe740 1 --2- 172.21.15.7:0/3770061901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9ecc00 0x55c91d9e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.838+0000 7f1b6affe740 1 -- 172.21.15.7:0/3770061901 >> 172.21.15.7:0/3770061901 conn(0x55c91d9b4fb0 msgr2=0x55c91d9ee770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.840+0000 7f1b6affe740 1 -- 172.21.15.7:0/3770061901 shutdown_connections 2024-06-13T07:07:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.840+0000 7f1b6affe740 1 -- 172.21.15.7:0/3770061901 wait complete. 2024-06-13T07:07:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.840+0000 7f1b6affe740 1 -- start start 2024-06-13T07:07:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.841+0000 7f1b6affe740 1 Processor -- start 2024-06-13T07:07:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.842+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.842+0000 7f1b65800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b65800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34852/0 (socket says 172.21.15.7:34852) 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b64e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b66200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b66200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52930/0 (socket says 172.21.15.7:52930) 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b64e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:33518/0 (socket says 172.21.15.7:33518) 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601b78a0 con 0x55c91d9ecc00 2024-06-13T07:07:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601b8d10 con 0x55c91d9fc440 2024-06-13T07:07:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b601b7f70 con 0x55c91d9f2760 2024-06-13T07:07:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b6affe740 10 client.0 _ll_register_callbacks cb 0x7f1b58097630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:07:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.845+0000 7f1b65800640 1 -- 172.21.15.7:0/142259269 learned_addr learned my addr 172.21.15.7:0/142259269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b66200640 1 -- 172.21.15.7:0/142259269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 msgr2=0x55c91da14b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b66200640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b66200640 1 -- 172.21.15.7:0/142259269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 msgr2=0x55c91da145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b66200640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b66200640 1 -- 172.21.15.7:0/142259269 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91da6c3f0 con 0x55c91d9fc440 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b65800640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.846+0000 7f1b64e00640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.847+0000 7f1b66200640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1b580977f0 tx=0x7f1b580998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0bdf430eb02d90 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.847+0000 7f1b47400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:07:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.847+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b8070 con 0x55c91d9fc440 2024-06-13T07:07:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.847+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1b5809a9e0 con 0x55c91d9fc440 2024-06-13T07:07:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.847+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b580cce50 con 0x55c91d9fc440 2024-06-13T07:07:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.848+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1b580a3e00 con 0x55c91d9fc440 2024-06-13T07:07:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.849+0000 7f1b6affe740 10 client.27119 fetch_fsmap learned FSMap version 38 2024-06-13T07:07:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.849+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c91da6d670 con 0x55c91d9fc440 2024-06-13T07:07:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b580cce50 con 0x55c91d9fc440 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f1b5807d070 con 0x55c91d9fc440 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 10 client.27119 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 10 client.27119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 20 client.27119 populate_metadata read hostname 'smithi007' 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da6dea0 0x55c91da702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 4 client.27119 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.851+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c91d8d57a0 con 0x55c91da6dea0 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.852+0000 7f1b65800640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da6dea0 0x55c91da702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.852+0000 7f1b65800640 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da6dea0 0x55c91da702c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1ad997c91d10f6 server_cookie=ca7deb1d77b96f39 in_seq=0 out_seq=0 2024-06-13T07:07:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.854+0000 7f1b47400640 10 client.27119 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.873+0000 7f1b47400640 1 -- 172.21.15.7:0/142259269 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c91d8d57a0 con 0x55c91da6dea0 2024-06-13T07:07:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.873+0000 7f1b47400640 10 client.27119 handle_command_reply: tid=0 2024-06-13T07:07:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.873+0000 7f1b6affe740 1 client.27119 shutdown 2024-06-13T07:07:46.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 msgr2=0x55c91da6beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1b580977f0 tx=0x7f1b580998a0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 shutdown_connections 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c91da6dea0 0x55c91da702c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c91d9fc440 0x55c91da6beb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c91d9f2760 0x55c91da14b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 --2- 172.21.15.7:0/142259269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c91d9ecc00 0x55c91da145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 >> 172.21.15.7:0/142259269 conn(0x55c91d9b4fb0 msgr2=0x55c91d9eb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b47400640 0 client.27119 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 shutdown_connections 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 1 -- 172.21.15.7:0/142259269 wait complete. 2024-06-13T07:07:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.874+0000 7f1b6affe740 20 client.27119 trim_cache size 0 max 16384 2024-06-13T07:07:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:07:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:07:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.875+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 msgr2=0x7f1b480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.875+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02a930 tx=0x7f1b4c02a340 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.875+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 msgr2=0x7f1b6019d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.875+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b6019d0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800be20 tx=0x7f1b5800bf10 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 shutdown_connections 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1b480a2cb0 0x7f1b480a5170 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b60107740 0x7f1b6019d0d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b60104e10 0x7f1b601960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 --2- 172.21.15.7:0/3680316591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b601024e0 0x7f1b60195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 >> 172.21.15.7:0/3680316591 conn(0x7f1b600fbe70 msgr2=0x7f1b60102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 shutdown_connections 2024-06-13T07:07:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:07:46.876+0000 7f1b68200640 1 -- 172.21.15.7:0/3680316591 wait complete. 2024-06-13T07:07:47.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:47 smithi175 ceph-mon[36886]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 534 op/s 2024-06-13T07:07:47.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:47 smithi007 ceph-mon[32077]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 534 op/s 2024-06-13T07:07:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:47 smithi189 ceph-mon[36616]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 534 op/s 2024-06-13T07:07:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 705.0 s, 505.2 tps, lat 200.432 ms stddev 243.850 2024-06-13T07:07:49.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:49 smithi175 ceph-mon[36886]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.3 MiB/s wr, 564 op/s 2024-06-13T07:07:49.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:49 smithi007 ceph-mon[32077]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.3 MiB/s wr, 564 op/s 2024-06-13T07:07:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:49 smithi189 ceph-mon[36616]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.3 MiB/s wr, 564 op/s 2024-06-13T07:07:51.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:51 smithi007 ceph-mon[32077]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 613 op/s 2024-06-13T07:07:51.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:51 smithi175 ceph-mon[36886]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 613 op/s 2024-06-13T07:07:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:51 smithi189 ceph-mon[36616]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 613 op/s 2024-06-13T07:07:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 710.0 s, 441.2 tps, lat 222.791 ms stddev 259.415 2024-06-13T07:07:53.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:53 smithi007 ceph-mon[32077]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 530 op/s 2024-06-13T07:07:53.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:53 smithi175 ceph-mon[36886]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 530 op/s 2024-06-13T07:07:53.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:53 smithi189 ceph-mon[36616]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 530 op/s 2024-06-13T07:07:55.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:55 smithi007 ceph-mon[32077]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-06-13T07:07:55.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:55 smithi189 ceph-mon[36616]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-06-13T07:07:55.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:55 smithi175 ceph-mon[36886]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-06-13T07:07:57.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:57 smithi007 ceph-mon[32077]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-06-13T07:07:57.668 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:57 smithi175 ceph-mon[36886]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-06-13T07:07:57.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:57 smithi189 ceph-mon[36616]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-06-13T07:07:57.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 715.0 s, 476.7 tps, lat 208.953 ms stddev 244.168 2024-06-13T07:07:59.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:59 smithi007 ceph-mon[32077]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:07:59.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:07:59 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:07:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:59 smithi189 ceph-mon[36616]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:07:59.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:07:59 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:07:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:59 smithi175 ceph-mon[36886]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:07:59.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:07:59 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:08:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.103+0000 7f2b04200640 1 -- 172.21.15.7:0/3498125862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc104e10 msgr2=0x7f2afc105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.103+0000 7f2b04200640 1 --2- 172.21.15.7:0/3498125862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc104e10 0x7f2afc105290 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009860 tx=0x7f2ae802a360 comp rx=0 tx=0).stop 2024-06-13T07:08:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.104+0000 7f2b04200640 1 -- 172.21.15.7:0/3498125862 shutdown_connections 2024-06-13T07:08:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.104+0000 7f2b04200640 1 --2- 172.21.15.7:0/3498125862 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc1057d0 0x7f2afc10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.104+0000 7f2b04200640 1 --2- 172.21.15.7:0/3498125862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc104e10 0x7f2afc105290 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009860 tx=0x7f2ae802a360 comp rx=0 tx=0).stop 2024-06-13T07:08:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.104+0000 7f2b04200640 1 --2- 172.21.15.7:0/3498125862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc103c20 0x7f2afc104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.104+0000 7f2b04200640 1 -- 172.21.15.7:0/3498125862 >> 172.21.15.7:0/3498125862 conn(0x7f2afc0ff190 msgr2=0x7f2afc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.106+0000 7f2b04200640 1 -- 172.21.15.7:0/3498125862 shutdown_connections 2024-06-13T07:08:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.107+0000 7f2b04200640 1 -- 172.21.15.7:0/3498125862 wait complete. 2024-06-13T07:08:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.110+0000 7f2b04200640 1 Processor -- start 2024-06-13T07:08:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 -- start start 2024-06-13T07:08:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 0x7f2afc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 0x7f2afc1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10f910 con 0x7f2afc19f6c0 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10f790 con 0x7f2afc103c20 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.111+0000 7f2b04200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc10fa90 con 0x7f2afc1057d0 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.112+0000 7f2b01800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.112+0000 7f2b01800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54572/0 (socket says 172.21.15.7:54572) 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.112+0000 7f2b01800640 1 -- 172.21.15.7:0/4178151034 learned_addr learned my addr 172.21.15.7:0/4178151034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.113+0000 7f2b00e00640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 0x7f2afc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.113+0000 7f2b01800640 1 -- 172.21.15.7:0/4178151034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 msgr2=0x7f2afc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.113+0000 7f2b02200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 0x7f2afc1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b01800640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 0x7f2afc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b01800640 1 -- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 msgr2=0x7f2afc1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b01800640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 0x7f2afc1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b01800640 1 -- 172.21.15.7:0/4178151034 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1a2120 con 0x7f2afc103c20 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b02200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 0x7f2afc1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.114+0000 7f2b00e00640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 0x7f2afc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.115+0000 7f2b01800640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2aec01ca50 tx=0x7f2aec01cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.116+0000 7f2afa000640 1 -- 172.21.15.7:0/4178151034 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec02a070 con 0x7f2afc103c20 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.116+0000 7f2afa000640 1 -- 172.21.15.7:0/4178151034 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aec01dbe0 con 0x7f2afc103c20 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.116+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc06b420 con 0x7f2afc103c20 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.116+0000 7f2afa000640 1 -- 172.21.15.7:0/4178151034 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0186c0 con 0x7f2afc103c20 2024-06-13T07:08:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.116+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc06b990 con 0x7f2afc103c20 2024-06-13T07:08:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.121+0000 7f2afa000640 1 -- 172.21.15.7:0/4178151034 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2aec02d070 con 0x7f2afc103c20 2024-06-13T07:08:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.124+0000 7f2afa000640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 0x7f2ae00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.124+0000 7f2afa000640 1 -- 172.21.15.7:0/4178151034 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2aec0e09b0 con 0x7f2afc103c20 2024-06-13T07:08:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.125+0000 7f2b00e00640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 0x7f2ae00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.127+0000 7f2b00e00640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 0x7f2ae00a5170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ae802a950 tx=0x7f2ae802ae00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.127+0000 7f2b06f90740 1 Processor -- start 2024-06-13T07:08:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.127+0000 7f2b06f90740 1 -- start start 2024-06-13T07:08:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c531644420 con 0x55c5316e6760 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5315ebc50 con 0x55c5316e15f0 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.128+0000 7f2b06f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c5316403e0 con 0x55c5316e0c00 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b01800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b01800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316e1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54064/0 (socket says 172.21.15.7:54064) 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b01800640 1 -- 172.21.15.7:0/1643324783 learned_addr learned my addr 172.21.15.7:0/1643324783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 -- 172.21.15.7:0/1643324783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 msgr2=0x55c5316e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316e1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 -- 172.21.15.7:0/1643324783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 msgr2=0x55c5316e8bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316e8bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.129+0000 7f2b00e00640 1 -- 172.21.15.7:0/1643324783 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c531358210 con 0x55c5316e15f0 2024-06-13T07:08:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.131+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316e60f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8008000 tx=0x7f2ae803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d71118d7eab2d50 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.133+0000 7f2af2a00640 1 -- 172.21.15.7:0/1643324783 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae804d070 con 0x55c5316e15f0 2024-06-13T07:08:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.133+0000 7f2af2a00640 1 -- 172.21.15.7:0/1643324783 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8044040 con 0x55c5316e15f0 2024-06-13T07:08:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.133+0000 7f2af2a00640 1 -- 172.21.15.7:0/1643324783 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8037040 con 0x55c5316e15f0 2024-06-13T07:08:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.134+0000 7f2b06f90740 1 -- 172.21.15.7:0/1643324783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 msgr2=0x55c5316e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.134+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316e60f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8008000 tx=0x7f2ae803b000 comp rx=0 tx=0).stop 2024-06-13T07:08:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.135+0000 7f2b06f90740 1 -- 172.21.15.7:0/1643324783 shutdown_connections 2024-06-13T07:08:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.135+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.135+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316e60f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.135+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1643324783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.135+0000 7f2b06f90740 1 -- 172.21.15.7:0/1643324783 >> 172.21.15.7:0/1643324783 conn(0x55c5316a8fb0 msgr2=0x55c5316dc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.139+0000 7f2b06f90740 1 -- 172.21.15.7:0/1643324783 shutdown_connections 2024-06-13T07:08:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.139+0000 7f2b06f90740 1 -- 172.21.15.7:0/1643324783 wait complete. 2024-06-13T07:08:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.139+0000 7f2b06f90740 1 -- start start 2024-06-13T07:08:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.140+0000 7f2b06f90740 1 Processor -- start 2024-06-13T07:08:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.142+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.142+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.142+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b06f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c5316f1ec0 con 0x55c5316e6760 2024-06-13T07:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b06f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5315ff0b0 con 0x55c5316e15f0 2024-06-13T07:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b06f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c5316f2040 con 0x55c5316e0c00 2024-06-13T07:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b01800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b01800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54066/0 (socket says 172.21.15.7:54066) 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b01800640 1 -- 172.21.15.7:0/1853239948 learned_addr learned my addr 172.21.15.7:0/1853239948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b06f90740 10 client.0 _ll_register_callbacks cb 0x55c5315eaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 -- 172.21.15.7:0/1853239948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 msgr2=0x55c5316f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 -- 172.21.15.7:0/1853239948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 msgr2=0x55c5316fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b00e00640 1 -- 172.21.15.7:0/1853239948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c531424620 con 0x55c5316e15f0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.143+0000 7f2b01800640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.144+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae8044040 con 0x55c5316e15f0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.144+0000 7f2b00e00640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f3d10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f3584156847fc server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.144+0000 7f2ae7400640 10 client.27131 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.145+0000 7f2ae7400640 1 -- 172.21.15.7:0/1853239948 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae804d070 con 0x55c5316e15f0 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.145+0000 7f2ae7400640 1 -- 172.21.15.7:0/1853239948 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2ae80037f0 con 0x55c5316e15f0 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.145+0000 7f2ae7400640 1 -- 172.21.15.7:0/1853239948 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2ae8037040 con 0x55c5316e15f0 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.145+0000 7f2b06f90740 10 client.27131 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.145+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55c5316e9410 con 0x55c5316e15f0 2024-06-13T07:08:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2ae7400640 1 -- 172.21.15.7:0/1853239948 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f2ae806b030 con 0x55c5316e15f0 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 10 client.27131 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 10 client.27131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 20 client.27131 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c5316ea760 0x55c5316ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 4 client.27131 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.146+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c531651b50 con 0x55c5316ea760 2024-06-13T07:08:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.147+0000 7f2b01800640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c5316ea760 0x55c5316ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.149+0000 7f2b01800640 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c5316ea760 0x55c5316ecbd0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c5cc08f698f302 server_cookie=6e831c6e89747d0b in_seq=0 out_seq=0 2024-06-13T07:08:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.149+0000 7f2ae7400640 10 client.27131 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.150+0000 7f2ae7400640 1 -- 172.21.15.7:0/1853239948 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c531651b50 con 0x55c5316ea760 2024-06-13T07:08:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.150+0000 7f2ae7400640 10 client.27131 handle_command_reply: tid=0 2024-06-13T07:08:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 client.27131 shutdown 2024-06-13T07:08:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 msgr2=0x55c5316f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f3d10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8005770 comp rx=0 tx=0).stop 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 shutdown_connections 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c5316ea760 0x55c5316ecbd0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f3d10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 --2- 172.21.15.7:0/1853239948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2ae7400640 0 client.27131 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.151+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 >> 172.21.15.7:0/1853239948 conn(0x55c5316a8fb0 msgr2=0x55c5316db840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.152+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 shutdown_connections 2024-06-13T07:08:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.152+0000 7f2b06f90740 1 -- 172.21.15.7:0/1853239948 wait complete. 2024-06-13T07:08:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.152+0000 7f2b06f90740 20 client.27131 trim_cache size 0 max 16384 2024-06-13T07:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 Processor -- start 2024-06-13T07:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 -- start start 2024-06-13T07:08:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e0c00 0x55c5316f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1a28b0 con 0x55c5316e6760 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc071a60 con 0x55c5316e0c00 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.189+0000 7f2b06f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06f8b0 con 0x55c5316e15f0 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:54082/0 (socket says 172.21.15.7:54082) 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 -- 172.21.15.7:0/3654132578 learned_addr learned my addr 172.21.15.7:0/3654132578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 -- 172.21.15.7:0/3654132578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e0c00 msgr2=0x55c5316f2df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e0c00 0x55c5316f2df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 -- 172.21.15.7:0/3654132578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 msgr2=0x55c5316dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.190+0000 7f2b00e00640 1 -- 172.21.15.7:0/3654132578 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5316e5cc0 con 0x55c5316e15f0 2024-06-13T07:08:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.191+0000 7f2b00e00640 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8030bb0 tx=0x7f2ae8061000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b86954c134f1491 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.191+0000 7f2af2000640 1 -- 172.21.15.7:0/3654132578 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80037f0 con 0x55c5316e15f0 2024-06-13T07:08:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.191+0000 7f2af2000640 1 -- 172.21.15.7:0/3654132578 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8044040 con 0x55c5316e15f0 2024-06-13T07:08:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.191+0000 7f2af2000640 1 -- 172.21.15.7:0/3654132578 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80037f0 con 0x55c5316e15f0 2024-06-13T07:08:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.192+0000 7f2b06f90740 1 -- 172.21.15.7:0/3654132578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 msgr2=0x55c5316fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.192+0000 7f2b06f90740 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8030bb0 tx=0x7f2ae8061000 comp rx=0 tx=0).stop 2024-06-13T07:08:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.194+0000 7f2b06f90740 1 -- 172.21.15.7:0/3654132578 shutdown_connections 2024-06-13T07:08:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e15f0 0x55c5316fad10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 --2- 172.21.15.7:0/3654132578 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e0c00 0x55c5316f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 -- 172.21.15.7:0/3654132578 >> 172.21.15.7:0/3654132578 conn(0x55c5316a8fb0 msgr2=0x55c5316f37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 -- 172.21.15.7:0/3654132578 shutdown_connections 2024-06-13T07:08:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.195+0000 7f2b06f90740 1 -- 172.21.15.7:0/3654132578 wait complete. 2024-06-13T07:08:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.196+0000 7f2b06f90740 1 -- start start 2024-06-13T07:08:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.196+0000 7f2b06f90740 1 Processor -- start 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06ff20 con 0x55c5316e6760 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc071390 con 0x55c5316e15f0 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0705f0 con 0x55c5316e0c00 2024-06-13T07:08:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.197+0000 7f2b06f90740 10 client.0 _ll_register_callbacks cb 0x7f2ae804b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.198+0000 7f2b02200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.198+0000 7f2b02200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35872/0 (socket says 172.21.15.7:35872) 2024-06-13T07:08:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.198+0000 7f2b02200640 1 -- 172.21.15.7:0/2009764855 learned_addr learned my addr 172.21.15.7:0/2009764855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.198+0000 7f2b01800640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b00e00640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b02200640 1 -- 172.21.15.7:0/2009764855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 msgr2=0x55c5316fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b02200640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b02200640 1 -- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 msgr2=0x55c5316f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b02200640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b02200640 1 -- 172.21.15.7:0/2009764855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5316f7fb0 con 0x55c5316e6760 2024-06-13T07:08:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.199+0000 7f2b00e00640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.200+0000 7f2b02200640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2af4010620 tx=0x7f2af4010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997070bae448ff7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.200+0000 7f2ae7400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.200+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4012920 con 0x55c5316e6760 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.200+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f2af4012ac0 con 0x55c5316e6760 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.200+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae8044040 con 0x55c5316e6760 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.201+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2af4012d00 con 0x55c5316e6760 2024-06-13T07:08:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.201+0000 7f2b06f90740 10 client.16404 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.201+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c531761070 con 0x55c5316e6760 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.201+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8044040 con 0x55c5316e6760 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f2af400f040 con 0x55c5316e6760 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 10 client.16404 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 20 client.16404 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c531761950 0x55c531763d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 4 client.16404 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.202+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5316f35c0 con 0x55c531761950 2024-06-13T07:08:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.204+0000 7f2b01800640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c531761950 0x55c531763d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.205+0000 7f2b01800640 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c531761950 0x55c531763d70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5d4e4c460a8ae2 server_cookie=8c86f1c82cbd9780 in_seq=0 out_seq=0 2024-06-13T07:08:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.205+0000 7f2ae7400640 10 client.16404 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.221+0000 7f2ae7400640 1 -- 172.21.15.7:0/2009764855 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c5316f35c0 con 0x55c531761950 2024-06-13T07:08:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.221+0000 7f2ae7400640 10 client.16404 handle_command_reply: tid=0 2024-06-13T07:08:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.221+0000 7f2b06f90740 1 client.16404 shutdown 2024-06-13T07:08:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.222+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 msgr2=0x55c5316f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.222+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2af4010620 tx=0x7f2af4010a20 comp rx=0 tx=0).stop 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.222+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 shutdown_connections 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.222+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c531761950 0x55c531763d70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.223+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5316e6760 0x55c5316f79f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.223+0000 7f2ae7400640 0 client.16404 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.223+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5316e15f0 0x55c5316f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.223+0000 7f2b06f90740 1 --2- 172.21.15.7:0/2009764855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c5316e0c00 0x55c5316fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.223+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 >> 172.21.15.7:0/2009764855 conn(0x55c5316a8fb0 msgr2=0x55c5316f37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.224+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 shutdown_connections 2024-06-13T07:08:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.224+0000 7f2b06f90740 1 -- 172.21.15.7:0/2009764855 wait complete. 2024-06-13T07:08:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.225+0000 7f2b06f90740 20 client.16404 trim_cache size 0 max 16384 2024-06-13T07:08:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:08:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:08:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 msgr2=0x7f2ae00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 0x7f2ae00a5170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ae802a950 tx=0x7f2ae802ae00 comp rx=0 tx=0).stop 2024-06-13T07:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 msgr2=0x7f2afc19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2aec01ca50 tx=0x7f2aec01cf10 comp rx=0 tx=0).stop 2024-06-13T07:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 shutdown_connections 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f2ae00a2cb0 0x7f2ae00a5170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc19f6c0 0x7f2afc1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2afc1057d0 0x7f2afc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 --2- 172.21.15.7:0/4178151034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc103c20 0x7f2afc19a120 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 >> 172.21.15.7:0/4178151034 conn(0x7f2afc0ff190 msgr2=0x7f2afc108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 shutdown_connections 2024-06-13T07:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:00.227+0000 7f2b04200640 1 -- 172.21.15.7:0/4178151034 wait complete. 2024-06-13T07:08:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:01 smithi007 ceph-mon[32077]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-06-13T07:08:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:08:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:01 smithi189 ceph-mon[36616]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-06-13T07:08:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:01.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:01.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:08:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:01 smithi175 ceph-mon[36886]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-06-13T07:08:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:08:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:08:01.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:08:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 720.0 s, 498.3 tps, lat 203.250 ms stddev 264.388 2024-06-13T07:08:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:03 smithi007 ceph-mon[32077]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 461 op/s 2024-06-13T07:08:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:03 smithi189 ceph-mon[36616]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 461 op/s 2024-06-13T07:08:03.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:03 smithi175 ceph-mon[36886]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.8 MiB/s wr, 461 op/s 2024-06-13T07:08:05.667 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:05 smithi007 ceph-mon[32077]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 492 op/s 2024-06-13T07:08:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:05 smithi189 ceph-mon[36616]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 492 op/s 2024-06-13T07:08:05.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:05 smithi175 ceph-mon[36886]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 492 op/s 2024-06-13T07:08:07.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:07 smithi007 ceph-mon[32077]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-06-13T07:08:07.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:07 smithi189 ceph-mon[36616]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-06-13T07:08:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:07 smithi175 ceph-mon[36886]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 427 op/s 2024-06-13T07:08:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 725.0 s, 463.8 tps, lat 219.969 ms stddev 301.067 2024-06-13T07:08:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:09 smithi189 ceph-mon[36616]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-06-13T07:08:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:09 smithi175 ceph-mon[36886]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-06-13T07:08:09.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:09 smithi007 ceph-mon[32077]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-06-13T07:08:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:11 smithi189 ceph-mon[36616]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-06-13T07:08:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:11 smithi175 ceph-mon[36886]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-06-13T07:08:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:11 smithi007 ceph-mon[32077]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-06-13T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 730.0 s, 458.6 tps, lat 214.772 ms stddev 256.151 2024-06-13T07:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.462+0000 7f3998a00640 1 -- 172.21.15.7:0/928444608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994107740 msgr2=0x7f3994109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.462+0000 7f3998a00640 1 --2- 172.21.15.7:0/928444608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994107740 0x7f3994109b30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3984009860 tx=0x7f398402a340 comp rx=0 tx=0).stop 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 -- 172.21.15.7:0/928444608 shutdown_connections 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 --2- 172.21.15.7:0/928444608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994107740 0x7f3994109b30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 --2- 172.21.15.7:0/928444608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f3994107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 --2- 172.21.15.7:0/928444608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39941024e0 0x7f39941048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 -- 172.21.15.7:0/928444608 >> 172.21.15.7:0/928444608 conn(0x7f39940fbe70 msgr2=0x7f39940fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 -- 172.21.15.7:0/928444608 shutdown_connections 2024-06-13T07:08:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.464+0000 7f3998a00640 1 -- 172.21.15.7:0/928444608 wait complete. 2024-06-13T07:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.465+0000 7f3998a00640 1 Processor -- start 2024-06-13T07:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 -- start start 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39941024e0 0x7f3994199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 0x7f39941a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f399410d680 con 0x7f39941024e0 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f399410d500 con 0x7f3994104e10 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3998a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f399410d800 con 0x7f3994107740 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3991400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.466+0000 7f3992800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 0x7f39941a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37168/0 (socket says 172.21.15.7:37168) 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 -- 172.21.15.7:0/1362469706 learned_addr learned my addr 172.21.15.7:0/1362469706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991e00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39941024e0 0x7f3994199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 -- 172.21.15.7:0/1362469706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 msgr2=0x7f39941a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 0x7f39941a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 -- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39941024e0 msgr2=0x7f3994199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39941024e0 0x7f3994199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 -- 172.21.15.7:0/1362469706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941a1970 con 0x7f3994104e10 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3992800640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 0x7f39941a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.467+0000 7f3991400640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f398000c8f0 tx=0x7f398000cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.469+0000 7f398aa00640 1 -- 172.21.15.7:0/1362469706 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980007bb0 con 0x7f3994104e10 2024-06-13T07:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.469+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941a1c60 con 0x7f3994104e10 2024-06-13T07:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.469+0000 7f398aa00640 1 -- 172.21.15.7:0/1362469706 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980007d50 con 0x7f3994104e10 2024-06-13T07:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.469+0000 7f398aa00640 1 -- 172.21.15.7:0/1362469706 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39800187e0 con 0x7f3994104e10 2024-06-13T07:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.469+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941a21b0 con 0x7f3994104e10 2024-06-13T07:08:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.474+0000 7f399b802740 1 Processor -- start 2024-06-13T07:08:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.482+0000 7f399b802740 1 -- start start 2024-06-13T07:08:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.482+0000 7f399b802740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.484+0000 7f398aa00640 1 -- 172.21.15.7:0/1362469706 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f398000f450 con 0x7f3994104e10 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.484+0000 7f399b802740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e03785f0 0x55c4e037d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.484+0000 7f3992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f3992800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37182/0 (socket says 172.21.15.7:37182) 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f399b802740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e037fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f3992800640 1 -- 172.21.15.7:0/4144441242 learned_addr learned my addr 172.21.15.7:0/4144441242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c4e02db420 con 0x55c4e03785f0 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c4e0282c50 con 0x55c4e0377c00 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.485+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c4e02d73e0 con 0x55c4e037d760 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.486+0000 7f3992800640 1 -- 172.21.15.7:0/4144441242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 msgr2=0x55c4e037fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.486+0000 7f3992800640 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e037fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.486+0000 7f3992800640 1 -- 172.21.15.7:0/4144441242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e03785f0 msgr2=0x55c4e037d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.486+0000 7f3992800640 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e03785f0 0x55c4e037d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.486+0000 7f3992800640 1 -- 172.21.15.7:0/4144441242 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dffef210 con 0x55c4e0377c00 2024-06-13T07:08:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.487+0000 7f3992800640 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3984031000 tx=0x7f3984030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaaa480af819306e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.487+0000 7f397f000640 1 -- 172.21.15.7:0/4144441242 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984041020 con 0x55c4e0377c00 2024-06-13T07:08:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.487+0000 7f397f000640 1 -- 172.21.15.7:0/4144441242 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39840035f0 con 0x55c4e0377c00 2024-06-13T07:08:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.487+0000 7f397f000640 1 -- 172.21.15.7:0/4144441242 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984037040 con 0x55c4e0377c00 2024-06-13T07:08:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.489+0000 7f398aa00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 0x7f39780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.489+0000 7f398aa00640 1 -- 172.21.15.7:0/1362469706 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f39800cf420 con 0x7f3994104e10 2024-06-13T07:08:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.490+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 msgr2=0x55c4e0378020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.490+0000 7f399b802740 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3984031000 tx=0x7f3984030650 comp rx=0 tx=0).stop 2024-06-13T07:08:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.490+0000 7f3991e00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 0x7f39780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.491+0000 7f3991e00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 0x7f39780a51f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f398c007350 tx=0x7f398c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 shutdown_connections 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e037fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e03785f0 0x55c4e037d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 --2- 172.21.15.7:0/4144441242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e0377c00 0x55c4e0378020 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 >> 172.21.15.7:0/4144441242 conn(0x55c4e033ffb0 msgr2=0x55c4e0373680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.494+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 shutdown_connections 2024-06-13T07:08:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.495+0000 7f399b802740 1 -- 172.21.15.7:0/4144441242 wait complete. 2024-06-13T07:08:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.496+0000 7f399b802740 1 -- start start 2024-06-13T07:08:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.496+0000 7f399b802740 1 Processor -- start 2024-06-13T07:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.499+0000 7f399b802740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.499+0000 7f399b802740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.499+0000 7f399b802740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0389490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f399b802740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c4e03806b0 con 0x55c4e0377c00 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f399b802740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c4e02960b0 con 0x55c4e03785f0 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37198/0 (socket says 172.21.15.7:37198) 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3992800640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0389490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 -- 172.21.15.7:0/4058977151 learned_addr learned my addr 172.21.15.7:0/4058977151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f399b802740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c4e0380830 con 0x55c4e037d760 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f399b802740 10 client.0 _ll_register_callbacks cb 0x55c4e0281f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991e00640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 -- 172.21.15.7:0/4058977151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 msgr2=0x55c4e0389490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0389490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.500+0000 7f3991400640 1 -- 172.21.15.7:0/4058977151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 msgr2=0x55c4e0381f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.501+0000 7f3991400640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.501+0000 7f3991400640 1 -- 172.21.15.7:0/4058977151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e00bb620 con 0x55c4e03785f0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.501+0000 7f3991e00640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.501+0000 7f3992800640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0389490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f3991400640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39800a4020 tx=0x7f398009b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9234fa675f9f450 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f3973400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398009fc90 con 0x55c4e03785f0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f39800cf140 con 0x55c4e03785f0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39840035f0 con 0x55c4e03785f0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f39800b9050 con 0x55c4e03785f0 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f399b802740 10 client.27143 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.502+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c4e038c870 con 0x55c4e03785f0 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39840035f0 con 0x55c4e03785f0 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f39800860b0 con 0x55c4e03785f0 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 10 client.27143 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 10 client.27143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 20 client.27143 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e038df00 0x55c4e03902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 4 client.27143 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.504+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e02e8b50 con 0x55c4e038df00 2024-06-13T07:08:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.508+0000 7f3991e00640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e038df00 0x55c4e03902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.508+0000 7f3991e00640 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e038df00 0x55c4e03902f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d49a72287e5476 server_cookie=1b7f0b8975a1101a in_seq=0 out_seq=0 2024-06-13T07:08:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.509+0000 7f3973400640 10 client.27143 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.509+0000 7f3973400640 1 -- 172.21.15.7:0/4058977151 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c4e02e8b50 con 0x55c4e038df00 2024-06-13T07:08:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.509+0000 7f3973400640 10 client.27143 handle_command_reply: tid=0 2024-06-13T07:08:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.509+0000 7f399b802740 1 client.27143 shutdown 2024-06-13T07:08:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.511+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 msgr2=0x55c4e0382450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.511+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39800a4020 tx=0x7f398009b370 comp rx=0 tx=0).stop 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 shutdown_connections 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e038df00 0x55c4e03902f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0389490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e0382450 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 --2- 172.21.15.7:0/4058977151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f3973400640 0 client.27143 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.512+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 >> 172.21.15.7:0/4058977151 conn(0x55c4e033ffb0 msgr2=0x55c4e0372840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:08:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.513+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 shutdown_connections 2024-06-13T07:08:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.513+0000 7f399b802740 1 -- 172.21.15.7:0/4058977151 wait complete. 2024-06-13T07:08:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.513+0000 7f399b802740 20 client.27143 trim_cache size 0 max 16384 2024-06-13T07:08:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.573+0000 7f399b802740 1 Processor -- start 2024-06-13T07:08:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.573+0000 7f399b802740 1 -- start start 2024-06-13T07:08:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.573+0000 7f399b802740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.573+0000 7f399b802740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e03785f0 0x55c4e0389450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.573+0000 7f399b802740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e037d760 0x55c4e0376c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.574+0000 7f399b802740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39941aede0 con 0x55c4e0377c00 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.574+0000 7f399b802740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39941ac6b0 con 0x55c4e037d760 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.574+0000 7f399b802740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39941aa500 con 0x55c4e03785f0 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.574+0000 7f3991e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.574+0000 7f3991e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54274/0 (socket says 172.21.15.7:54274) 2024-06-13T07:08:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 -- 172.21.15.7:0/1000466849 learned_addr learned my addr 172.21.15.7:0/1000466849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 -- 172.21.15.7:0/1000466849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e03785f0 msgr2=0x55c4e0389450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e03785f0 0x55c4e0389450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 -- 172.21.15.7:0/1000466849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e037d760 msgr2=0x55c4e0376c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e037d760 0x55c4e0376c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.575+0000 7f3991e00640 1 -- 172.21.15.7:0/1000466849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e0378e80 con 0x55c4e0377c00 2024-06-13T07:08:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.576+0000 7f3991e00640 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f398c00efa0 tx=0x7f398c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a48867a172521 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.577+0000 7f397e600640 1 -- 172.21.15.7:0/1000466849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c02a070 con 0x55c4e0377c00 2024-06-13T07:08:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.577+0000 7f397e600640 1 -- 172.21.15.7:0/1000466849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f398c01e0d0 con 0x55c4e0377c00 2024-06-13T07:08:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.578+0000 7f399b802740 1 -- 172.21.15.7:0/1000466849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 msgr2=0x55c4e0381530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.578+0000 7f399b802740 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f398c00efa0 tx=0x7f398c00c000 comp rx=0 tx=0).stop 2024-06-13T07:08:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.578+0000 7f399b802740 1 -- 172.21.15.7:0/1000466849 shutdown_connections 2024-06-13T07:08:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.578+0000 7f399b802740 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e037d760 0x55c4e0376c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.578+0000 7f399b802740 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e03785f0 0x55c4e0389450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.579+0000 7f399b802740 1 --2- 172.21.15.7:0/1000466849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0381530 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.579+0000 7f399b802740 1 -- 172.21.15.7:0/1000466849 >> 172.21.15.7:0/1000466849 conn(0x55c4e033ffb0 msgr2=0x55c4e0376570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.579+0000 7f399b802740 1 -- 172.21.15.7:0/1000466849 shutdown_connections 2024-06-13T07:08:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.579+0000 7f399b802740 1 -- 172.21.15.7:0/1000466849 wait complete. 2024-06-13T07:08:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.580+0000 7f399b802740 1 -- start start 2024-06-13T07:08:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.580+0000 7f399b802740 1 Processor -- start 2024-06-13T07:08:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.581+0000 7f399b802740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0386b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.582+0000 7f3991e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0386b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.582+0000 7f399b802740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e03870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.582+0000 7f399b802740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0388010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.582+0000 7f3991e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0386b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54280/0 (socket says 172.21.15.7:54280) 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f3991e00640 1 -- 172.21.15.7:0/441565913 learned_addr learned my addr 172.21.15.7:0/441565913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39941aab70 con 0x55c4e0377c00 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f3992800640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0388010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f3991400640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e03870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39941abfe0 con 0x55c4e03785f0 2024-06-13T07:08:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.583+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39941ab240 con 0x55c4e037d760 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f3992800640 1 -- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 msgr2=0x55c4e03870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f3992800640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e03870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f3992800640 1 -- 172.21.15.7:0/441565913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 msgr2=0x55c4e0386b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f3992800640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0386b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f3992800640 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e0388600 con 0x55c4e037d760 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f399b802740 10 client.0 _ll_register_callbacks cb 0x7f398000fd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.584+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f398c01e0d0 con 0x55c4e037d760 2024-06-13T07:08:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.587+0000 7f3973400640 10 client.27058 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.587+0000 7f3992800640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0388010 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3984009830 tx=0x7f3984030a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f7439fb914c055 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.588+0000 7f3973400640 1 -- 172.21.15.7:0/441565913 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984041040 con 0x55c4e037d760 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.588+0000 7f3973400640 1 -- 172.21.15.7:0/441565913 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f3984003b40 con 0x55c4e037d760 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.588+0000 7f3973400640 1 -- 172.21.15.7:0/441565913 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3984037040 con 0x55c4e037d760 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.588+0000 7f399b802740 10 client.27058 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.588+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55c4e03897d0 con 0x55c4e037d760 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.589+0000 7f3973400640 1 -- 172.21.15.7:0/441565913 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f398403d480 con 0x55c4e037d760 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.590+0000 7f399b802740 10 client.27058 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.590+0000 7f399b802740 10 client.27058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.590+0000 7f399b802740 20 client.27058 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.590+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e04015b0 0x55c4e04039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.590+0000 7f399b802740 4 client.27058 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.591+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4e0381d00 con 0x55c4e04015b0 2024-06-13T07:08:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.591+0000 7f3991e00640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e04015b0 0x55c4e04039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.593+0000 7f3991e00640 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e04015b0 0x55c4e04039a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0421962c6278d7 server_cookie=f3d3ae3bcd5cdd96 in_seq=0 out_seq=0 2024-06-13T07:08:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.593+0000 7f3973400640 10 client.27058 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.770+0000 7f3973400640 1 -- 172.21.15.7:0/441565913 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c4e0381d00 con 0x55c4e04015b0 2024-06-13T07:08:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.770+0000 7f3973400640 10 client.27058 handle_command_reply: tid=0 2024-06-13T07:08:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.770+0000 7f399b802740 1 client.27058 shutdown 2024-06-13T07:08:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 msgr2=0x55c4e0388010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0388010 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3984009830 tx=0x7f3984030a80 comp rx=0 tx=0).stop 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 shutdown_connections 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55c4e04015b0 0x55c4e04039a0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4e037d760 0x55c4e0388010 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c4e03785f0 0x55c4e03870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 --2- 172.21.15.7:0/441565913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4e0377c00 0x55c4e0386b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 >> 172.21.15.7:0/441565913 conn(0x55c4e033ffb0 msgr2=0x55c4e0371fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f3973400640 0 client.27058 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 shutdown_connections 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 1 -- 172.21.15.7:0/441565913 wait complete. 2024-06-13T07:08:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.772+0000 7f399b802740 20 client.27058 trim_cache size 0 max 16384 2024-06-13T07:08:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:08:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:08:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.773+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 msgr2=0x7f39780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.773+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 0x7f39780a51f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f398c007350 tx=0x7f398c007400 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.773+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 msgr2=0x7f399419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.773+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f398000c8f0 tx=0x7f398000cdb0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 shutdown_connections 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f39780a2d30 0x7f39780a51f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3994107740 0x7f39941a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3994104e10 0x7f399419a290 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 --2- 172.21.15.7:0/1362469706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39941024e0 0x7f3994199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 >> 172.21.15.7:0/1362469706 conn(0x7f39940fbe70 msgr2=0x7f3994100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 shutdown_connections 2024-06-13T07:08:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:13.774+0000 7f3998a00640 1 -- 172.21.15.7:0/1362469706 wait complete. 2024-06-13T07:08:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:13 smithi189 ceph-mon[36616]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 451 op/s 2024-06-13T07:08:13.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:13 smithi175 ceph-mon[36886]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 451 op/s 2024-06-13T07:08:13.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:13 smithi007 ceph-mon[32077]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 451 op/s 2024-06-13T07:08:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:15 smithi189 ceph-mon[36616]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:08:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:15 smithi175 ceph-mon[36886]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:08:15.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:15 smithi007 ceph-mon[32077]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 507 op/s 2024-06-13T07:08:17.624 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:17 smithi007 ceph-mon[32077]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-06-13T07:08:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:17 smithi189 ceph-mon[36616]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-06-13T07:08:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:17 smithi175 ceph-mon[36886]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 422 op/s 2024-06-13T07:08:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 735.0 s, 477.8 tps, lat 200.950 ms stddev 225.034 2024-06-13T07:08:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 -- 172.21.15.7:0/850905885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2650108500 msgr2=0x7f26501088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 --2- 172.21.15.7:0/850905885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2650108500 0x7f26501088e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f263c009990 tx=0x7f263c009ca0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 -- 172.21.15.7:0/850905885 shutdown_connections 2024-06-13T07:08:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 --2- 172.21.15.7:0/850905885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265006ab30 0x7f265010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 --2- 172.21.15.7:0/850905885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2650108500 0x7f26501088e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 --2- 172.21.15.7:0/850905885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265006a170 0x7f265006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.029+0000 7f2655000700 1 -- 172.21.15.7:0/850905885 >> 172.21.15.7:0/850905885 conn(0x7f2650078340 msgr2=0x7f2650078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.030+0000 7f2655000700 1 -- 172.21.15.7:0/850905885 shutdown_connections 2024-06-13T07:08:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.030+0000 7f2655000700 1 -- 172.21.15.7:0/850905885 wait complete. 2024-06-13T07:08:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.034+0000 7f2655000700 1 Processor -- start 2024-06-13T07:08:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.034+0000 7f2655000700 1 -- start start 2024-06-13T07:08:19.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.035+0000 7f2655000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265006a170 0x7f265019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f2655000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f2655000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2650108500 0x7f265010d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f2655000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2650112830 con 0x7f2650108500 2024-06-13T07:08:19.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f2655000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26501126b0 con 0x7f265006ab30 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f2655000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26501129b0 con 0x7f265006a170 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f264d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f264d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37232/0 (socket says 172.21.15.7:37232) 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f264d400700 1 -- 172.21.15.7:0/1320806133 learned_addr learned my addr 172.21.15.7:0/1320806133 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.036+0000 7f264e800700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2650108500 0x7f265010d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 -- 172.21.15.7:0/1320806133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265006a170 msgr2=0x7f265019b740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265006a170 0x7f265019b740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 -- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2650108500 msgr2=0x7f265010d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2650108500 0x7f265010d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 -- 172.21.15.7:0/1320806133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265010dae0 con 0x7f265006ab30 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f264d400700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f264000b530 tx=0x7f264000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.037+0000 7f2646a00700 1 -- 172.21.15.7:0/1320806133 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640015020 con 0x7f265006ab30 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.038+0000 7f2646a00700 1 -- 172.21.15.7:0/1320806133 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2640011d10 con 0x7f265006ab30 2024-06-13T07:08:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.038+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265010ddd0 con 0x7f265006ab30 2024-06-13T07:08:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.038+0000 7f2646a00700 1 -- 172.21.15.7:0/1320806133 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640010e50 con 0x7f265006ab30 2024-06-13T07:08:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.038+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265019f620 con 0x7f265006ab30 2024-06-13T07:08:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.042+0000 7f2646a00700 1 -- 172.21.15.7:0/1320806133 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2640018020 con 0x7f265006ab30 2024-06-13T07:08:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.046+0000 7f265f8e3b80 1 Processor -- start 2024-06-13T07:08:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.046+0000 7f265f8e3b80 1 -- start start 2024-06-13T07:08:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc64b0 0x560adfdbfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560adfccea10 con 0x560adfdc5b90 2024-06-13T07:08:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560adfd11b00 con 0x560adfdc0240 2024-06-13T07:08:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560adfccc8f0 con 0x560adfdc64b0 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f264d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f264d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:37242/0 (socket says 172.21.15.7:37242) 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f264d400700 1 -- 172.21.15.7:0/3829311029 learned_addr learned my addr 172.21.15.7:0/3829311029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.047+0000 7f264de00700 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc64b0 0x560adfdbfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f264d400700 1 -- 172.21.15.7:0/3829311029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc64b0 msgr2=0x560adfdbfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f264d400700 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc64b0 0x560adfdbfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f264d400700 1 -- 172.21.15.7:0/3829311029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 msgr2=0x560adfdc5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f264d400700 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f264d400700 1 -- 172.21.15.7:0/3829311029 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adfcb9f90 con 0x560adfdc0240 2024-06-13T07:08:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.048+0000 7f2646a00700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 0x7f26340a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.049+0000 7f264d400700 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26400d2d00 tx=0x7f26400d7540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99bafc811f79783 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.049+0000 7f2646a00700 1 -- 172.21.15.7:0/1320806133 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f26400ceb60 con 0x7f265006ab30 2024-06-13T07:08:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.049+0000 7f264e800700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 0x7f26340a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.051+0000 7f263b400700 1 -- 172.21.15.7:0/3829311029 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640097320 con 0x560adfdc0240 2024-06-13T07:08:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.051+0000 7f263b400700 1 -- 172.21.15.7:0/3829311029 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400d2e50 con 0x560adfdc0240 2024-06-13T07:08:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.051+0000 7f263b400700 1 -- 172.21.15.7:0/3829311029 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264009a9e0 con 0x560adfdc0240 2024-06-13T07:08:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f264e800700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 0x7f26340a5300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f264800e1b0 tx=0x7f2648009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 -- 172.21.15.7:0/3829311029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 msgr2=0x560adfdc0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26400d2d00 tx=0x7f26400d7540 comp rx=0 tx=0).stop 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 -- 172.21.15.7:0/3829311029 shutdown_connections 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc0240 0x560adfdc0680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/3829311029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc64b0 0x560adfdbfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.054+0000 7f265f8e3b80 1 -- 172.21.15.7:0/3829311029 >> 172.21.15.7:0/3829311029 conn(0x560adfcec170 msgr2=0x560adfdbaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.055+0000 7f265f8e3b80 1 -- 172.21.15.7:0/3829311029 shutdown_connections 2024-06-13T07:08:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.055+0000 7f265f8e3b80 1 -- 172.21.15.7:0/3829311029 wait complete. 2024-06-13T07:08:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.055+0000 7f265f8e3b80 1 -- start start 2024-06-13T07:08:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.056+0000 7f265f8e3b80 1 Processor -- start 2024-06-13T07:08:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.057+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.057+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc64b0 0x560adfdc99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f264de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f264e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc64b0 0x560adfdc99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560adfcadf60 con 0x560adfdc5b90 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560adfd0dac0 con 0x560adfdc64b0 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560adfddddb0 con 0x560adfdc0240 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f264de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41378/0 (socket says 172.21.15.7:41378) 2024-06-13T07:08:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f264de00700 1 -- 172.21.15.7:0/1161532429 learned_addr learned my addr 172.21.15.7:0/1161532429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.058+0000 7f265f8e3b80 10 client.0 _ll_register_callbacks cb 0x560adfca7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.059+0000 7f264de00700 1 -- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc64b0 msgr2=0x560adfdc99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.059+0000 7f264de00700 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc64b0 0x560adfdc99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.059+0000 7f264de00700 1 -- 172.21.15.7:0/1161532429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 msgr2=0x560adfdc9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.059+0000 7f264de00700 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.059+0000 7f264de00700 1 -- 172.21.15.7:0/1161532429 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adfc608d0 con 0x560adfdc0240 2024-06-13T07:08:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.060+0000 7f264de00700 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f263c003c30 tx=0x7f263c003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a63a3d65da5e54a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.060+0000 7f262fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.060+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c0054d0 con 0x560adfdc0240 2024-06-13T07:08:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.060+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f263c005670 con 0x560adfdc0240 2024-06-13T07:08:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.060+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26400d2e50 con 0x560adfdc0240 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.061+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f263c00b7d0 con 0x560adfdc0240 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.061+0000 7f265f8e3b80 10 client.27064 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.061+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560adfde84c0 con 0x560adfdc0240 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.061+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400d2e50 con 0x560adfdc0240 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f263c00f950 con 0x560adfdc0240 2024-06-13T07:08:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 10 client.27064 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 20 client.27064 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdeab50 0x560adfdecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 4 client.27064 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.062+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560adfd1f000 con 0x560adfdeab50 2024-06-13T07:08:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.063+0000 7f264d400700 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdeab50 0x560adfdecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.064+0000 7f264d400700 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdeab50 0x560adfdecfa0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0dc70dc6d87806 server_cookie=8091dc18cdcc7801 in_seq=0 out_seq=0 2024-06-13T07:08:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.064+0000 7f262fe00700 10 client.27064 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.065+0000 7f262fe00700 1 -- 172.21.15.7:0/1161532429 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560adfd1f000 con 0x560adfdeab50 2024-06-13T07:08:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.065+0000 7f262fe00700 10 client.27064 handle_command_reply: tid=0 2024-06-13T07:08:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.065+0000 7f265f8e3b80 1 client.27064 shutdown 2024-06-13T07:08:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.065+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 msgr2=0x560adfdcc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.065+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f263c003c30 tx=0x7f263c003c60 comp rx=0 tx=0).stop 2024-06-13T07:08:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 shutdown_connections 2024-06-13T07:08:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdeab50 0x560adfdecfa0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc64b0 0x560adfdc99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdcc170 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1161532429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfdc5b90 0x560adfdc9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 >> 172.21.15.7:0/1161532429 conn(0x560adfcec170 msgr2=0x560adfdb4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f262fe00700 0 client.27064 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.066+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 shutdown_connections 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.067+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1161532429 wait complete. 2024-06-13T07:08:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.067+0000 7f265f8e3b80 20 client.27064 trim_cache size 0 max 16384 2024-06-13T07:08:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 Processor -- start 2024-06-13T07:08:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 -- start start 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfdbfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdc6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26501b7170 con 0x560adfcec170 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2650073bc0 con 0x560adfdc5b90 2024-06-13T07:08:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.108+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26500719d0 con 0x560adfdc0240 2024-06-13T07:08:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.109+0000 7f264d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.109+0000 7f264d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41388/0 (socket says 172.21.15.7:41388) 2024-06-13T07:08:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.109+0000 7f264d400700 1 -- 172.21.15.7:0/1499955783 learned_addr learned my addr 172.21.15.7:0/1499955783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 -- 172.21.15.7:0/1499955783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 msgr2=0x560adfdc6fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:08:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdc6fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 -- 172.21.15.7:0/1499955783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 msgr2=0x560adfdbfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfdbfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 -- 172.21.15.7:0/1499955783 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adfdc7a00 con 0x560adfdc0240 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f264d400700 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26400d7890 tx=0x7f26400d2640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6eadabc6de9b14 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f263aa00700 1 -- 172.21.15.7:0/1499955783 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264009abc0 con 0x560adfdc0240 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.110+0000 7f263aa00700 1 -- 172.21.15.7:0/1499955783 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400b4070 con 0x560adfdc0240 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.111+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1499955783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 msgr2=0x560adfdb5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.111+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26400d7890 tx=0x7f26400d2640 comp rx=0 tx=0).stop 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1499955783 shutdown_connections 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfdbfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/1499955783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfdb5130 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1499955783 >> 172.21.15.7:0/1499955783 conn(0x560adfdc0bd0 msgr2=0x560adfdc6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1499955783 shutdown_connections 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.112+0000 7f265f8e3b80 1 -- 172.21.15.7:0/1499955783 wait complete. 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.113+0000 7f265f8e3b80 1 -- start start 2024-06-13T07:08:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.113+0000 7f265f8e3b80 1 Processor -- start 2024-06-13T07:08:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfde3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdf1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2650072080 con 0x560adfcec170 2024-06-13T07:08:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26500734f0 con 0x560adfdc5b90 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2650072750 con 0x560adfdc0240 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.115+0000 7f265f8e3b80 10 client.0 _ll_register_callbacks cb 0x560adfca7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.116+0000 7f264d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.116+0000 7f264de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfde3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.116+0000 7f264d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41390/0 (socket says 172.21.15.7:41390) 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.116+0000 7f264d400700 1 -- 172.21.15.7:0/768667103 learned_addr learned my addr 172.21.15.7:0/768667103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.116+0000 7f264e800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdf1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264d400700 1 -- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 msgr2=0x560adfdf1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264d400700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdf1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264d400700 1 -- 172.21.15.7:0/768667103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 msgr2=0x560adfde3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264d400700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfde3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264d400700 1 -- 172.21.15.7:0/768667103 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adfdf2760 con 0x560adfdc0240 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.117+0000 7f264de00700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfde3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.118+0000 7f264d400700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f26400d2d00 tx=0x7f26400d25a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43ee9a98bc4b1b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.118+0000 7f262fe00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.118+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264009abc0 con 0x560adfdc0240 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.118+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f26400104f0 con 0x560adfdc0240 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.118+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26400b4070 con 0x560adfdc0240 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.119+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f264009b780 con 0x560adfdc0240 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.119+0000 7f265f8e3b80 10 client.27070 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.119+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560adfe55470 con 0x560adfdc0240 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.119+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400b4070 con 0x560adfdc0240 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.119+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f264009f0b0 con 0x560adfdc0240 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 10 client.27070 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 10 client.27070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 20 client.27070 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdc64b0 0x560adfe576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 4 client.27070 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560adfd19580 con 0x560adfdc64b0 2024-06-13T07:08:19.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.120+0000 7f264de00700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdc64b0 0x560adfe576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:19.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.127+0000 7f264de00700 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdc64b0 0x560adfe576f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54125371ae29d99 server_cookie=b4b046f2ef45a1c6 in_seq=0 out_seq=0 2024-06-13T07:08:19.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.127+0000 7f262fe00700 10 client.27070 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:19.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f262fe00700 1 -- 172.21.15.7:0/768667103 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560adfd19580 con 0x560adfdc64b0 2024-06-13T07:08:19.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f262fe00700 10 client.27070 handle_command_reply: tid=0 2024-06-13T07:08:19.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 client.27070 shutdown 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 msgr2=0x560adfde3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f26400d2d00 tx=0x7f26400d25a0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 shutdown_connections 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560adfdc64b0 0x560adfe576f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560adfdc5b90 0x560adfdf1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560adfcec170 0x560adfde3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 --2- 172.21.15.7:0/768667103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560adfdc0240 0x560adfde3880 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 >> 172.21.15.7:0/768667103 conn(0x560adfdc0bd0 msgr2=0x560adfdc8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.130+0000 7f262fe00700 0 client.27070 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:19.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.131+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 shutdown_connections 2024-06-13T07:08:19.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.131+0000 7f265f8e3b80 1 -- 172.21.15.7:0/768667103 wait complete. 2024-06-13T07:08:19.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.131+0000 7f265f8e3b80 20 client.27070 trim_cache size 0 max 16384 2024-06-13T07:08:19.133 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:08:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:08:19.133 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:08:19.133 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:08:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 msgr2=0x7f26340a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 0x7f26340a5300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f264800e1b0 tx=0x7f2648009040 comp rx=0 tx=0).stop 2024-06-13T07:08:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 msgr2=0x7f265019bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f264000b530 tx=0x7f264000b8f0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 shutdown_connections 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2650108500 0x7f265010d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265006a170 0x7f265019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f26340a2e40 0x7f26340a5300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 --2- 172.21.15.7:0/1320806133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f265006ab30 0x7f265019bc80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 >> 172.21.15.7:0/1320806133 conn(0x7f2650078340 msgr2=0x7f2650101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.135+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 shutdown_connections 2024-06-13T07:08:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:19.136+0000 7f2655000700 1 -- 172.21.15.7:0/1320806133 wait complete. 2024-06-13T07:08:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:19 smithi189 ceph-mon[36616]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-06-13T07:08:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:19 smithi175 ceph-mon[36886]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-06-13T07:08:19.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:19 smithi007 ceph-mon[32077]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-06-13T07:08:20.635 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:08:20.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:08:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:21.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:21 smithi007 ceph-mon[32077]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2024-06-13T07:08:21.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:21 smithi189 ceph-mon[36616]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2024-06-13T07:08:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:21 smithi175 ceph-mon[36886]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2024-06-13T07:08:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 740.0 s, 414.8 tps, lat 243.674 ms stddev 317.442 2024-06-13T07:08:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:23 smithi189 ceph-mon[36616]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-06-13T07:08:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:23 smithi175 ceph-mon[36886]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-06-13T07:08:23.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:23 smithi007 ceph-mon[32077]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-06-13T07:08:25.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.159+0000 7f9996800700 1 -- 172.21.15.7:0/1251986457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f999010ef90 msgr2=0x7f999010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.159+0000 7f9996800700 1 --2- 172.21.15.7:0/1251986457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f999010ef90 0x7f999010f370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9984009990 tx=0x7f9984009ca0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.167+0000 7f9996800700 1 -- 172.21.15.7:0/1251986457 shutdown_connections 2024-06-13T07:08:25.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.168+0000 7f9996800700 1 --2- 172.21.15.7:0/1251986457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f9990115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.168+0000 7f9996800700 1 --2- 172.21.15.7:0/1251986457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f999010ef90 0x7f999010f370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.168+0000 7f9996800700 1 --2- 172.21.15.7:0/1251986457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9990109490 0x7f9990109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.168+0000 7f9996800700 1 -- 172.21.15.7:0/1251986457 >> 172.21.15.7:0/1251986457 conn(0x7f999006d470 msgr2=0x7f999006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:25.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.170+0000 7f9996800700 1 -- 172.21.15.7:0/1251986457 shutdown_connections 2024-06-13T07:08:25.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.170+0000 7f9996800700 1 -- 172.21.15.7:0/1251986457 wait complete. 2024-06-13T07:08:25.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.172+0000 7f9996800700 1 Processor -- start 2024-06-13T07:08:25.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.174+0000 7f9996800700 1 -- start start 2024-06-13T07:08:25.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.177+0000 7f9996800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.177+0000 7f9996800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.177+0000 7f9996800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.178+0000 7f9995400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.178+0000 7f9995400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57482/0 (socket says 172.21.15.7:57482) 2024-06-13T07:08:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.178+0000 7f9995e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.178+0000 7f9995e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40450/0 (socket says 172.21.15.7:40450) 2024-06-13T07:08:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.179+0000 7f9994a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.179+0000 7f9994a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40776/0 (socket says 172.21.15.7:40776) 2024-06-13T07:08:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.179+0000 7f9995400700 1 -- 172.21.15.7:0/3680985561 learned_addr learned my addr 172.21.15.7:0/3680985561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.179+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99901193e0 con 0x7f9990109e50 2024-06-13T07:08:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.179+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9990119260 con 0x7f999010ef90 2024-06-13T07:08:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.180+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9990119560 con 0x7f9990109490 2024-06-13T07:08:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.181+0000 7f9995e00700 1 -- 172.21.15.7:0/3680985561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 msgr2=0x7f99901a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.181+0000 7f9995e00700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.181+0000 7f9995e00700 1 -- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 msgr2=0x7f99901a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.181+0000 7f9995e00700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.181+0000 7f9995e00700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999006e270 con 0x7f999010ef90 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.182+0000 7f9994a00700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.183+0000 7f9995e00700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f998800c9c0 tx=0x7f998800ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.183+0000 7f9995400700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.183+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988012870 con 0x7f999010ef90 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.183+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999019fc00 con 0x7f999010ef90 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.183+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99901a0110 con 0x7f999010ef90 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.184+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9988007740 con 0x7f999010ef90 2024-06-13T07:08:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.184+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988012870 con 0x7f999010ef90 2024-06-13T07:08:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.188+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9970005330 con 0x7f999010ef90 2024-06-13T07:08:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.188+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f99880078f0 con 0x7f999010ef90 2024-06-13T07:08:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.191+0000 7f998e000700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 0x7f99780a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.191+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f99880cea40 con 0x7f999010ef90 2024-06-13T07:08:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.192+0000 7f9995400700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 0x7f99780a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.193+0000 7f9995400700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 0x7f99780a5420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9984009750 tx=0x7f998400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:25.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.196+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f99880d0e20 con 0x7f999010ef90 2024-06-13T07:08:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.699+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9970005620 con 0x7f999010ef90 2024-06-13T07:08:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.703+0000 7f998e000700 1 -- 172.21.15.7:0/3680985561 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11868 (secure 0 0 0) 0x7f998801f070 con 0x7f999010ef90 2024-06-13T07:08:25.705 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:08:25.705 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:04:39.673793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T07:08:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 msgr2=0x7f99780a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 0x7f99780a5420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9984009750 tx=0x7f998400b540 comp rx=0 tx=0).stop 2024-06-13T07:08:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 msgr2=0x7f999019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f998800c9c0 tx=0x7f998800ccd0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 shutdown_connections 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f99780a2f60 0x7f99780a5420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f999010ef90 0x7f999019f600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9990109490 0x7f99901a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 --2- 172.21.15.7:0/3680985561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9990109e50 0x7f99901a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.707+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 >> 172.21.15.7:0/3680985561 conn(0x7f999006d470 msgr2=0x7f999010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.708+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 shutdown_connections 2024-06-13T07:08:25.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:25.708+0000 7f9996800700 1 -- 172.21.15.7:0/3680985561 wait complete. 2024-06-13T07:08:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-06-13T07:08:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:25 smithi189 ceph-mon[36616]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-06-13T07:08:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:25 smithi175 ceph-mon[36886]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-06-13T07:08:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:25 smithi007 ceph-mon[32077]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-06-13T07:08:26.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:26 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3680985561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:08:26.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:26 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3680985561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:08:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:26 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3680985561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:08:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.021+0000 7f58fb200640 1 -- 172.21.15.7:0/717207312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc102a50 msgr2=0x7f58fc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.021+0000 7f58fb200640 1 --2- 172.21.15.7:0/717207312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc102a50 0x7f58fc102ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009860 tx=0x7f58e802a380 comp rx=0 tx=0).stop 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.022+0000 7f58fb200640 1 -- 172.21.15.7:0/717207312 shutdown_connections 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.022+0000 7f58fb200640 1 --2- 172.21.15.7:0/717207312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc103410 0x7f58fc109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.022+0000 7f58fb200640 1 --2- 172.21.15.7:0/717207312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc102a50 0x7f58fc102ed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.022+0000 7f58fb200640 1 --2- 172.21.15.7:0/717207312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc101860 0x7f58fc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.022+0000 7f58fb200640 1 -- 172.21.15.7:0/717207312 >> 172.21.15.7:0/717207312 conn(0x7f58fc0fcdd0 msgr2=0x7f58fc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.027+0000 7f58fb200640 1 -- 172.21.15.7:0/717207312 shutdown_connections 2024-06-13T07:08:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.029+0000 7f58fb200640 1 -- 172.21.15.7:0/717207312 wait complete. 2024-06-13T07:08:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.032+0000 7f58fb200640 1 Processor -- start 2024-06-13T07:08:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.032+0000 7f58fb200640 1 -- start start 2024-06-13T07:08:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc102a50 0x7f58fc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc103410 0x7f58fc1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10d5a0 con 0x7f58fc103410 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10d420 con 0x7f58fc102a50 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58fb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10d720 con 0x7f58fc101860 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57498/0 (socket says 172.21.15.7:57498) 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.033+0000 7f58f9e00640 1 -- 172.21.15.7:0/3335435260 learned_addr learned my addr 172.21.15.7:0/3335435260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.034+0000 7f58f9400640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc102a50 0x7f58fc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.034+0000 7f58f9e00640 1 -- 172.21.15.7:0/3335435260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc102a50 msgr2=0x7f58fc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.034+0000 7f58f9e00640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc102a50 0x7f58fc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.034+0000 7f58f9e00640 1 -- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc103410 msgr2=0x7f58fc1a1400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.035+0000 7f58fa800640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc103410 0x7f58fc1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.035+0000 7f58f9e00640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc103410 0x7f58fc1a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.035+0000 7f58f9e00640 1 -- 172.21.15.7:0/3335435260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc1a1a70 con 0x7f58fc101860 2024-06-13T07:08:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.037+0000 7f58f9e00640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b470 tx=0x7f58ec00b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.038+0000 7f58f2a00640 1 -- 172.21.15.7:0/3335435260 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec015020 con 0x7f58fc101860 2024-06-13T07:08:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.038+0000 7f58f2a00640 1 -- 172.21.15.7:0/3335435260 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec011d70 con 0x7f58fc101860 2024-06-13T07:08:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.038+0000 7f58f2a00640 1 -- 172.21.15.7:0/3335435260 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00f340 con 0x7f58fc101860 2024-06-13T07:08:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.038+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc1a1d60 con 0x7f58fc101860 2024-06-13T07:08:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.039+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc1a21d0 con 0x7f58fc101860 2024-06-13T07:08:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.051+0000 7f5903fc2740 1 Processor -- start 2024-06-13T07:08:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.051+0000 7f58f2a00640 1 -- 172.21.15.7:0/3335435260 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f58ec018020 con 0x7f58fc101860 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 -- start start 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb27599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55deb26b4420 con 0x55deb275f130 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55deb265bc50 con 0x55deb2759fc0 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.052+0000 7f5903fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55deb26b03e0 con 0x55deb27595d0 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.053+0000 7f58f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.053+0000 7f58f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40812/0 (socket says 172.21.15.7:40812) 2024-06-13T07:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.053+0000 7f58f9400640 1 -- 172.21.15.7:0/4010433137 learned_addr learned my addr 172.21.15.7:0/4010433137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.055+0000 7f58f9e00640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.055+0000 7f58fa800640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb27599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58f9400640 1 -- 172.21.15.7:0/4010433137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 msgr2=0x55deb27599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58f9400640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb27599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58f9400640 1 -- 172.21.15.7:0/4010433137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 msgr2=0x55deb275eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58f9400640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58f9400640 1 -- 172.21.15.7:0/4010433137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb23c8210 con 0x55deb275f130 2024-06-13T07:08:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.056+0000 7f58fa800640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb27599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.057+0000 7f58f2a00640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 0x7f58e00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.057+0000 7f58f2a00640 1 -- 172.21.15.7:0/3335435260 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f58ec0cee70 con 0x7f58fc101860 2024-06-13T07:08:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.057+0000 7f58fa800640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 0x7f58e00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.057+0000 7f58f9e00640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.058+0000 7f58f9400640 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58e802a970 tx=0x7f58e8005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7868d6b6746b39c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.059+0000 7f58fa800640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 0x7f58e00a5100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55deb275a820 tx=0x7f58f40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.059+0000 7f58e7400640 1 -- 172.21.15.7:0/4010433137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8037040 con 0x55deb275f130 2024-06-13T07:08:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.060+0000 7f58e7400640 1 -- 172.21.15.7:0/4010433137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e8005030 con 0x55deb275f130 2024-06-13T07:08:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.061+0000 7f58e7400640 1 -- 172.21.15.7:0/4010433137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8004a20 con 0x55deb275f130 2024-06-13T07:08:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 -- 172.21.15.7:0/4010433137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 msgr2=0x55deb27615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58e802a970 tx=0x7f58e8005cb0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 -- 172.21.15.7:0/4010433137 shutdown_connections 2024-06-13T07:08:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb27615c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 --2- 172.21.15.7:0/4010433137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb27599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 -- 172.21.15.7:0/4010433137 >> 172.21.15.7:0/4010433137 conn(0x55deb2718fb0 msgr2=0x55deb274c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 -- 172.21.15.7:0/4010433137 shutdown_connections 2024-06-13T07:08:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.062+0000 7f5903fc2740 1 -- 172.21.15.7:0/4010433137 wait complete. 2024-06-13T07:08:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.063+0000 7f5903fc2740 1 -- start start 2024-06-13T07:08:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.064+0000 7f5903fc2740 1 Processor -- start 2024-06-13T07:08:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.066+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 0x55deb2752710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.066+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb2759fc0 0x55deb2752c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.066+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.066+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 0x55deb2752710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 0x55deb2752710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40488/0 (socket says 172.21.15.7:40488) 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f5903fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55deb2750ed0 con 0x55deb2759fc0 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f5903fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55deb266f0b0 con 0x55deb27595d0 2024-06-13T07:08:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f5903fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55deb2751050 con 0x55deb275f130 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f5903fc2740 10 client.0 _ll_register_callbacks cb 0x55deb265af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f58fa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f58fa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57518/0 (socket says 172.21.15.7:57518) 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.067+0000 7f58f9e00640 1 -- 172.21.15.7:0/623721591 learned_addr learned my addr 172.21.15.7:0/623721591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58fa800640 1 -- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 msgr2=0x55deb2752710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58fa800640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 0x55deb2752710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58fa800640 1 -- 172.21.15.7:0/623721591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb2759fc0 msgr2=0x55deb2752c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58f9400640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb2759fc0 0x55deb2752c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58fa800640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb2759fc0 0x55deb2752c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.068+0000 7f58fa800640 1 -- 172.21.15.7:0/623721591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb2494620 con 0x55deb275f130 2024-06-13T07:08:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.069+0000 7f58fa800640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f91594460b43a1c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.069+0000 7f58dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.069+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f402a060 con 0x55deb275f130 2024-06-13T07:08:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.071+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f58f4015030 con 0x55deb275f130 2024-06-13T07:08:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.071+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e8005030 con 0x55deb275f130 2024-06-13T07:08:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.071+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f58f4009bb0 con 0x55deb275f130 2024-06-13T07:08:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.071+0000 7f5903fc2740 10 client.27085 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.071+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55deb2776a30 con 0x55deb275f130 2024-06-13T07:08:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e8005030 con 0x55deb275f130 2024-06-13T07:08:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f58f400b040 con 0x55deb275f130 2024-06-13T07:08:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 10 client.27085 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 10 client.27085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 20 client.27085 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb2778170 0x55deb277a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 4 client.27085 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.072+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb26c1b50 con 0x55deb2778170 2024-06-13T07:08:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.073+0000 7f58f9e00640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb2778170 0x55deb277a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.076+0000 7f58f9e00640 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb2778170 0x55deb277a5f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b540736660c857 server_cookie=bc3298984ab8f3d1 in_seq=0 out_seq=0 2024-06-13T07:08:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.076+0000 7f58dbe00640 10 client.27085 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.077+0000 7f58dbe00640 1 -- 172.21.15.7:0/623721591 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55deb26c1b50 con 0x55deb2778170 2024-06-13T07:08:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.077+0000 7f58dbe00640 10 client.27085 handle_command_reply: tid=0 2024-06-13T07:08:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.077+0000 7f5903fc2740 1 client.27085 shutdown 2024-06-13T07:08:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 msgr2=0x55deb2757c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f58f4005fd0 tx=0x7f58f4004ab0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 shutdown_connections 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb2778170 0x55deb277a5f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb275f130 0x55deb2757c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb2759fc0 0x55deb2752c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 --2- 172.21.15.7:0/623721591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb27595d0 0x55deb2752710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 >> 172.21.15.7:0/623721591 conn(0x55deb2718fb0 msgr2=0x55deb274b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.079+0000 7f58dbe00640 0 client.27085 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.080+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 shutdown_connections 2024-06-13T07:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.080+0000 7f5903fc2740 1 -- 172.21.15.7:0/623721591 wait complete. 2024-06-13T07:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.080+0000 7f5903fc2740 20 client.27085 trim_cache size 0 max 16384 2024-06-13T07:08:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.112+0000 7f5903fc2740 1 Processor -- start 2024-06-13T07:08:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 -- start start 2024-06-13T07:08:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb274f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1aaa80 con 0x55deb275f130 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1a8350 con 0x55deb2759fc0 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.116+0000 7f5903fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1a61c0 con 0x55deb27595d0 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:57530/0 (socket says 172.21.15.7:57530) 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 -- 172.21.15.7:0/1902366508 learned_addr learned my addr 172.21.15.7:0/1902366508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9400640 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 -- 172.21.15.7:0/1902366508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 msgr2=0x55deb275b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 -- 172.21.15.7:0/1902366508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 msgr2=0x55deb274f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb274f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 -- 172.21.15.7:0/1902366508 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb275a3a0 con 0x55deb27595d0 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.117+0000 7f58f9e00640 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f58ec010500 tx=0x7f58ec0d2950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c7f5f250a8220a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.118+0000 7f58e6a00640 1 -- 172.21.15.7:0/1902366508 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec09acf0 con 0x55deb27595d0 2024-06-13T07:08:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.118+0000 7f58e6a00640 1 -- 172.21.15.7:0/1902366508 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec09ba90 con 0x55deb27595d0 2024-06-13T07:08:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.119+0000 7f5903fc2740 1 -- 172.21.15.7:0/1902366508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 msgr2=0x55deb275b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.119+0000 7f5903fc2740 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f58ec010500 tx=0x7f58ec0d2950 comp rx=0 tx=0).stop 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 -- 172.21.15.7:0/1902366508 shutdown_connections 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb275f130 0x55deb274f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb2759fc0 0x55deb275b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 --2- 172.21.15.7:0/1902366508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb27595d0 0x55deb275b280 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 -- 172.21.15.7:0/1902366508 >> 172.21.15.7:0/1902366508 conn(0x55deb2718fb0 msgr2=0x55deb2752710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 -- 172.21.15.7:0/1902366508 shutdown_connections 2024-06-13T07:08:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 -- 172.21.15.7:0/1902366508 wait complete. 2024-06-13T07:08:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.122+0000 7f5903fc2740 1 -- start start 2024-06-13T07:08:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 Processor -- start 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb27595d0 0x55deb2789ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb2759fc0 0x55deb278a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1a6810 con 0x55deb27595d0 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1a7c80 con 0x55deb275f130 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc1a6ee0 con 0x55deb2759fc0 2024-06-13T07:08:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.123+0000 7f5903fc2740 10 client.0 _ll_register_callbacks cb 0x7f58f400fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.124+0000 7f58fa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.124+0000 7f58fa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40506/0 (socket says 172.21.15.7:40506) 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.124+0000 7f58fa800640 1 -- 172.21.15.7:0/2755703976 learned_addr learned my addr 172.21.15.7:0/2755703976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.124+0000 7f58f9400640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb2759fc0 0x55deb278a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.125+0000 7f58fa800640 1 -- 172.21.15.7:0/2755703976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb2759fc0 msgr2=0x55deb278a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.125+0000 7f58fa800640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb2759fc0 0x55deb278a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.125+0000 7f58fa800640 1 -- 172.21.15.7:0/2755703976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb27595d0 msgr2=0x55deb2789ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.125+0000 7f58fa800640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb27595d0 0x55deb2789ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.125+0000 7f58fa800640 1 -- 172.21.15.7:0/2755703976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb278b620 con 0x55deb275f130 2024-06-13T07:08:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.126+0000 7f58fa800640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f58f4016040 tx=0x7f58f4004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b48bf9d3d271be server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.126+0000 7f58dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.126+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f401dc20 con 0x55deb275f130 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.126+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f58f401ddc0 con 0x55deb275f130 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.127+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ec09ba90 con 0x55deb275f130 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.127+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f58f403a050 con 0x55deb275f130 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.127+0000 7f5903fc2740 10 client.27179 fetch_fsmap learned FSMap version 38 2024-06-13T07:08:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.127+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55deb27e2d50 con 0x55deb275f130 2024-06-13T07:08:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.128+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec09ba90 con 0x55deb275f130 2024-06-13T07:08:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.128+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f58f4025740 con 0x55deb275f130 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.128+0000 7f5903fc2740 10 client.27179 fetch_fsmap finished waiting for FSMap version 38 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.128+0000 7f5903fc2740 10 client.27179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.128+0000 7f5903fc2740 20 client.27179 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.129+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb27e3660 0x55deb27e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.129+0000 7f5903fc2740 4 client.27179 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.129+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deb26bbea0 con 0x55deb27e3660 2024-06-13T07:08:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.130+0000 7f58f9e00640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb27e3660 0x55deb27e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.131+0000 7f58dbe00640 10 client.27179 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.131+0000 7f58f9e00640 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb27e3660 0x55deb27e5a80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1ae90bc3089d462 server_cookie=5b80039574d7ee2b in_seq=0 out_seq=0 2024-06-13T07:08:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.385+0000 7f58dbe00640 1 -- 172.21.15.7:0/2755703976 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55deb26bbea0 con 0x55deb27e3660 2024-06-13T07:08:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.385+0000 7f58dbe00640 10 client.27179 handle_command_reply: tid=0 2024-06-13T07:08:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.385+0000 7f5903fc2740 1 client.27179 shutdown 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 msgr2=0x55deb278b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f58f4016040 tx=0x7f58f4004d50 comp rx=0 tx=0).stop 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 shutdown_connections 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55deb27e3660 0x55deb27e5a80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55deb275f130 0x55deb278b030 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55deb2759fc0 0x55deb278a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 --2- 172.21.15.7:0/2755703976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55deb27595d0 0x55deb2789ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 >> 172.21.15.7:0/2755703976 conn(0x55deb2718fb0 msgr2=0x55deb2752710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f58dbe00640 0 client.27179 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 shutdown_connections 2024-06-13T07:08:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 1 -- 172.21.15.7:0/2755703976 wait complete. 2024-06-13T07:08:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.386+0000 7f5903fc2740 20 client.27179 trim_cache size 0 max 16384 2024-06-13T07:08:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:08:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:08:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.387+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 msgr2=0x7f58e00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.387+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 0x7f58e00a5100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55deb275a820 tx=0x7f58f40095c0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.387+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 msgr2=0x7f58fc199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.387+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b470 tx=0x7f58ec00b930 comp rx=0 tx=0).stop 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 shutdown_connections 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f58e00a2c40 0x7f58e00a5100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc103410 0x7f58fc1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc102a50 0x7f58fc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 --2- 172.21.15.7:0/3335435260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58fc101860 0x7f58fc199e50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 >> 172.21.15.7:0/3335435260 conn(0x7f58fc0fcdd0 msgr2=0x7f58fc073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 shutdown_connections 2024-06-13T07:08:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:27.388+0000 7f58fb200640 1 -- 172.21.15.7:0/3335435260 wait complete. 2024-06-13T07:08:27.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:08:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:27 smithi189 ceph-mon[36616]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-06-13T07:08:27.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:27 smithi175 ceph-mon[36886]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-06-13T07:08:27.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 745.0 s, 437.4 tps, lat 237.830 ms stddev 291.023 2024-06-13T07:08:27.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:27 smithi007 ceph-mon[32077]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.3 MiB/s wr, 412 op/s 2024-06-13T07:08:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:29 smithi189 ceph-mon[36616]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.6 MiB/s wr, 439 op/s 2024-06-13T07:08:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:29 smithi175 ceph-mon[36886]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.6 MiB/s wr, 439 op/s 2024-06-13T07:08:29.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:29 smithi007 ceph-mon[32077]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.6 MiB/s wr, 439 op/s 2024-06-13T07:08:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:31 smithi189 ceph-mon[36616]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:08:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:31 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:31 smithi175 ceph-mon[36886]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:08:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:31 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:31.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:31 smithi007 ceph-mon[32077]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-06-13T07:08:31.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:31 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.434+0000 7f9fe5400700 1 --2- 172.21.15.7:0/1556224224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe010da90 0x7f9fe0111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.435+0000 7f9fe8800700 1 -- 172.21.15.7:0/1556224224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe01121c0 msgr2=0x7f9fe0119d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.435+0000 7f9fe8800700 1 --2- 172.21.15.7:0/1556224224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe01121c0 0x7f9fe0119d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b0d0 tx=0x7f9fdc00b3e0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.436+0000 7f9fe8800700 1 -- 172.21.15.7:0/1556224224 shutdown_connections 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.436+0000 7f9fe8800700 1 --2- 172.21.15.7:0/1556224224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe01121c0 0x7f9fe0119d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.436+0000 7f9fe8800700 1 --2- 172.21.15.7:0/1556224224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010d0e0 0x7f9fe010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.436+0000 7f9fe8800700 1 --2- 172.21.15.7:0/1556224224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe010da90 0x7f9fe0111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.436+0000 7f9fe8800700 1 -- 172.21.15.7:0/1556224224 >> 172.21.15.7:0/1556224224 conn(0x7f9fe006d650 msgr2=0x7f9fe010b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.437+0000 7f9fe8800700 1 -- 172.21.15.7:0/1556224224 shutdown_connections 2024-06-13T07:08:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.437+0000 7f9fe8800700 1 -- 172.21.15.7:0/1556224224 wait complete. 2024-06-13T07:08:32.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.439+0000 7f9fe8800700 1 Processor -- start 2024-06-13T07:08:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.440+0000 7f9fe8800700 1 -- start start 2024-06-13T07:08:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe8800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 0x7f9fe01a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe8800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 0x7f9fe01a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe8800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 0x7f9fe01a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52518/0 (socket says 172.21.15.7:52518) 2024-06-13T07:08:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.441+0000 7f9fe6800700 1 -- 172.21.15.7:0/4067330396 learned_addr learned my addr 172.21.15.7:0/4067330396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 0x7f9fe01a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011d250 con 0x7f9fe010d0e0 2024-06-13T07:08:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011d0d0 con 0x7f9fe01121c0 2024-06-13T07:08:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011d3d0 con 0x7f9fe010da90 2024-06-13T07:08:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe6800700 1 -- 172.21.15.7:0/4067330396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 msgr2=0x7f9fe01a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.442+0000 7f9fe6800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 0x7f9fe01a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe6800700 1 -- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 msgr2=0x7f9fe01a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe6800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 0x7f9fe01a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe6800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe01a8580 con 0x7f9fe01121c0 2024-06-13T07:08:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe5400700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 0x7f9fe01a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:32.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 0x7f9fe01a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.443+0000 7f9fe6800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b770 tx=0x7f9fdc0079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.444+0000 7f9fdaa00700 1 -- 172.21.15.7:0/4067330396 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003c00 con 0x7f9fe01121c0 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.444+0000 7f9fdaa00700 1 -- 172.21.15.7:0/4067330396 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fdc007e10 con 0x7f9fe01121c0 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.444+0000 7f9fdaa00700 1 -- 172.21.15.7:0/4067330396 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc01b880 con 0x7f9fe01121c0 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.444+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe01a8810 con 0x7f9fe01121c0 2024-06-13T07:08:32.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.444+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01b00f0 con 0x7f9fe01121c0 2024-06-13T07:08:32.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.454+0000 7f9fdaa00700 1 -- 172.21.15.7:0/4067330396 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9fdc026450 con 0x7f9fe01121c0 2024-06-13T07:08:32.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.455+0000 7f9ff30d7b80 1 Processor -- start 2024-06-13T07:08:32.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.456+0000 7f9ff30d7b80 1 -- start start 2024-06-13T07:08:32.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.456+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 0x55cf9bee2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 0x55cf9bedc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9bedce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf9bde2a10 con 0x55cf9bee23a0 2024-06-13T07:08:32.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cf9be25b00 con 0x55cf9bedca50 2024-06-13T07:08:32.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf9bde08f0 con 0x55cf9bee2cc0 2024-06-13T07:08:32.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.457+0000 7f9fdaa00700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 0x7f9fc80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.458+0000 7f9fe6800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 0x7f9fc80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.458+0000 7f9fe5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 0x55cf9bee2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.458+0000 7f9fdaa00700 1 -- 172.21.15.7:0/4067330396 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fdc0d0270 con 0x7f9fe01121c0 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.458+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 0x55cf9bedc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 0x55cf9bedc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53784/0 (socket says 172.21.15.7:53784) 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5e00700 1 -- 172.21.15.7:0/405658139 learned_addr learned my addr 172.21.15.7:0/405658139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5400700 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9bedce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5400700 1 -- 172.21.15.7:0/405658139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 msgr2=0x55cf9bedc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5400700 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 0x55cf9bedc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5400700 1 -- 172.21.15.7:0/405658139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 msgr2=0x55cf9bedce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.459+0000 7f9fe5400700 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9bedce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.460+0000 7f9fe5400700 1 -- 172.21.15.7:0/405658139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9bdcdf90 con 0x55cf9bee23a0 2024-06-13T07:08:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.460+0000 7f9fe6800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 0x7f9fc80a52b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc09c7d0 tx=0x7f9fdc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.460+0000 7f9fe5400700 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 0x55cf9bee2780 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fe010fd80 tx=0x7f9fd0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dcbc2fd01dbea3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.461+0000 7f9fcf400700 1 -- 172.21.15.7:0/405658139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0013070 con 0x55cf9bee23a0 2024-06-13T07:08:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.461+0000 7f9fcf400700 1 -- 172.21.15.7:0/405658139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd0004510 con 0x55cf9bee23a0 2024-06-13T07:08:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.461+0000 7f9fcf400700 1 -- 172.21.15.7:0/405658139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0005280 con 0x55cf9bee23a0 2024-06-13T07:08:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.462+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/405658139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 msgr2=0x55cf9bee2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.462+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 0x55cf9bee2780 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fe010fd80 tx=0x7f9fd0009260 comp rx=0 tx=0).stop 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.463+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/405658139 shutdown_connections 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.463+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9bedce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.463+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bee23a0 0x55cf9bee2780 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.463+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/405658139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee2cc0 0x55cf9bedc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.463+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/405658139 >> 172.21.15.7:0/405658139 conn(0x55cf9be00170 msgr2=0x55cf9beceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.464+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/405658139 shutdown_connections 2024-06-13T07:08:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.464+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/405658139 wait complete. 2024-06-13T07:08:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.465+0000 7f9ff30d7b80 1 -- start start 2024-06-13T07:08:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.465+0000 7f9ff30d7b80 1 Processor -- start 2024-06-13T07:08:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9bee8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee2cc0 0x55cf9bee62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf9bdc1f60 con 0x55cf9bedca50 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55cf9be21ac0 con 0x55cf9bee2cc0 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf9bee9640 con 0x55cf9bee23a0 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9ff30d7b80 10 client.0 _ll_register_callbacks cb 0x55cf9bdbb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53792/0 (socket says 172.21.15.7:53792) 2024-06-13T07:08:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.466+0000 7f9fe5e00700 1 -- 172.21.15.7:0/3977544131 learned_addr learned my addr 172.21.15.7:0/3977544131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee2cc0 0x55cf9bee62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5e00700 1 -- 172.21.15.7:0/3977544131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee2cc0 msgr2=0x55cf9bee62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee2cc0 0x55cf9bee62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5e00700 1 -- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 msgr2=0x55cf9bee8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9bee8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.467+0000 7f9fe5e00700 1 -- 172.21.15.7:0/3977544131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9bd748d0 con 0x55cf9bee23a0 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.468+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4010f10 tx=0x7f9fd400f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae219981876ea69 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.469+0000 7f9fc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.469+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400fea0 con 0x55cf9bee23a0 2024-06-13T07:08:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.469+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd0004510 con 0x55cf9bee23a0 2024-06-13T07:08:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.469+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fd4016070 con 0x55cf9bee23a0 2024-06-13T07:08:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.470+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9fd400fea0 con 0x55cf9bee23a0 2024-06-13T07:08:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.471+0000 7f9ff30d7b80 10 client.27100 fetch_fsmap learned FSMap version 39 2024-06-13T07:08:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.471+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cf9bef3ee0 con 0x55cf9bee23a0 2024-06-13T07:08:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.471+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd0004510 con 0x55cf9bee23a0 2024-06-13T07:08:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.471+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9fd4019410 con 0x55cf9bee23a0 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.472+0000 7f9ff30d7b80 10 client.27100 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.472+0000 7f9ff30d7b80 10 client.27100 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.472+0000 7f9ff30d7b80 20 client.27100 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.473+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bef6470 0x55cf9bef8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.473+0000 7f9ff30d7b80 4 client.27100 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.473+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9be33000 con 0x55cf9bef6470 2024-06-13T07:08:32.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.473+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bef6470 0x55cf9bef8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.475+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bef6470 0x55cf9bef8860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62e66bbc7fbe47ae server_cookie=331e7800de37eb06 in_seq=0 out_seq=0 2024-06-13T07:08:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.475+0000 7f9fc3e00700 10 client.27100 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.476+0000 7f9fc3e00700 1 -- 172.21.15.7:0/3977544131 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf9be33000 con 0x55cf9bef6470 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.476+0000 7f9fc3e00700 10 client.27100 handle_command_reply: tid=0 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.476+0000 7f9ff30d7b80 1 client.27100 shutdown 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 msgr2=0x55cf9bee5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4010f10 tx=0x7f9fd400f700 comp rx=0 tx=0).stop 2024-06-13T07:08:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 shutdown_connections 2024-06-13T07:08:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bef6470 0x55cf9bef8860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee2cc0 0x55cf9bee62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9bee8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3977544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bee5d70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 >> 172.21.15.7:0/3977544131 conn(0x55cf9be00170 msgr2=0x55cf9bec8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9fc3e00700 0 client.27100 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.477+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 shutdown_connections 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.478+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3977544131 wait complete. 2024-06-13T07:08:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.478+0000 7f9ff30d7b80 20 client.27100 trim_cache size 0 max 16384 2024-06-13T07:08:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.513+0000 7f9ff30d7b80 1 Processor -- start 2024-06-13T07:08:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.513+0000 7f9ff30d7b80 1 -- start start 2024-06-13T07:08:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.513+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9be00170 0x55cf9bedc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.513+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bec9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b6ea0 con 0x55cf9be00170 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b4700 con 0x55cf9bedca50 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0072150 con 0x55cf9bee23a0 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52530/0 (socket says 172.21.15.7:52530) 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.514+0000 7f9fe5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bec9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.515+0000 7f9fe5e00700 1 -- 172.21.15.7:0/3445370953 learned_addr learned my addr 172.21.15.7:0/3445370953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:32.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.516+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9be00170 0x55cf9bedc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5400700 1 -- 172.21.15.7:0/3445370953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 msgr2=0x55cf9becf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5400700 1 -- 172.21.15.7:0/3445370953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9be00170 msgr2=0x55cf9bedc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9be00170 0x55cf9bedc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5400700 1 -- 172.21.15.7:0/3445370953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9bee5b40 con 0x55cf9bee23a0 2024-06-13T07:08:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.518+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:32.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.519+0000 7f9fe5400700 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bec9130 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55cf9bef9160 tx=0x7f9fd0004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2391fa557b4cc500 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.519+0000 7f9fcea00700 1 -- 172.21.15.7:0/3445370953 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00049b0 con 0x55cf9bee23a0 2024-06-13T07:08:32.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.520+0000 7f9fcea00700 1 -- 172.21.15.7:0/3445370953 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd00130d0 con 0x55cf9bee23a0 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.521+0000 7f9fcea00700 1 -- 172.21.15.7:0/3445370953 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00049b0 con 0x55cf9bee23a0 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.521+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3445370953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 msgr2=0x55cf9bec9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.521+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bec9130 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55cf9bef9160 tx=0x7f9fd0004bc0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.522+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3445370953 shutdown_connections 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.522+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9bee23a0 0x55cf9bec9130 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.522+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9be00170 0x55cf9bedc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.522+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/3445370953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bedca50 0x55cf9becf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.522+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3445370953 >> 172.21.15.7:0/3445370953 conn(0x55cf9bedd3e0 msgr2=0x55cf9becea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.523+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3445370953 shutdown_connections 2024-06-13T07:08:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.523+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/3445370953 wait complete. 2024-06-13T07:08:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.524+0000 7f9ff30d7b80 1 -- start start 2024-06-13T07:08:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.524+0000 7f9ff30d7b80 1 Processor -- start 2024-06-13T07:08:32.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 0x55cf9befe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9beee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee23a0 0x55cf9beeee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b2bc0 con 0x55cf9bedca50 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b4030 con 0x55cf9bee23a0 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b3290 con 0x55cf9be00170 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.525+0000 7f9ff30d7b80 10 client.0 _ll_register_callbacks cb 0x55cf9bdbb380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.526+0000 7f9fe5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9beee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.526+0000 7f9fe5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 0x55cf9befe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.526+0000 7f9fe5400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 0x55cf9befe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53818/0 (socket says 172.21.15.7:53818) 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.526+0000 7f9fe5400700 1 -- 172.21.15.7:0/1914784500 learned_addr learned my addr 172.21.15.7:0/1914784500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.526+0000 7f9fe5400700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee23a0 0x55cf9beeee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.527+0000 7f9fe5e00700 1 -- 172.21.15.7:0/1914784500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 msgr2=0x55cf9befe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.527+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 0x55cf9befe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.527+0000 7f9fe5e00700 1 -- 172.21.15.7:0/1914784500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee23a0 msgr2=0x55cf9beeee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.527+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee23a0 0x55cf9beeee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.527+0000 7f9fe5e00700 1 -- 172.21.15.7:0/1914784500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9befd1b0 con 0x55cf9bedca50 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.528+0000 7f9fc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:08:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.528+0000 7f9fe5e00700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9beee930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4019da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59857624ac0a781f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:32.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.529+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4031060 con 0x55cf9bedca50 2024-06-13T07:08:32.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.529+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9fd4016040 con 0x55cf9bedca50 2024-06-13T07:08:32.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.529+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd00130d0 con 0x55cf9bedca50 2024-06-13T07:08:32.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.531+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9fd4005240 con 0x55cf9bedca50 2024-06-13T07:08:32.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.531+0000 7f9ff30d7b80 10 client.16446 fetch_fsmap learned FSMap version 39 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.531+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cf9bf60340 con 0x55cf9bedca50 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.533+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd00130d0 con 0x55cf9bedca50 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.533+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9fd402c480 con 0x55cf9bedca50 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.533+0000 7f9ff30d7b80 10 client.16446 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.534+0000 7f9ff30d7b80 10 client.16446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.534+0000 7f9ff30d7b80 20 client.16446 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.534+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bee2cc0 0x55cf9bf62a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.534+0000 7f9ff30d7b80 4 client.16446 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:08:32.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.534+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf9be2d580 con 0x55cf9bee2cc0 2024-06-13T07:08:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.535+0000 7f9fe5400700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bee2cc0 0x55cf9bf62a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.539+0000 7f9fe5400700 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bee2cc0 0x55cf9bf62a40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b50a12f57c9fe25b server_cookie=f1a15dd7ebae726d in_seq=0 out_seq=0 2024-06-13T07:08:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.539+0000 7f9fc3e00700 10 client.16446 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:08:32.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.539+0000 7f9fc3e00700 1 -- 172.21.15.7:0/1914784500 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf9be2d580 con 0x55cf9bee2cc0 2024-06-13T07:08:32.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.539+0000 7f9fc3e00700 10 client.16446 handle_command_reply: tid=0 2024-06-13T07:08:32.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.540+0000 7f9ff30d7b80 1 client.16446 shutdown 2024-06-13T07:08:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 msgr2=0x55cf9beee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9beee930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4019da0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 shutdown_connections 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55cf9bee2cc0 0x55cf9bf62a40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55cf9bee23a0 0x55cf9beeee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf9be00170 0x55cf9befe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 --2- 172.21.15.7:0/1914784500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf9bedca50 0x55cf9beee930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 >> 172.21.15.7:0/1914784500 conn(0x55cf9bedd3e0 msgr2=0x55cf9bee5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.543+0000 7f9fc3e00700 0 client.16446 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.544+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 shutdown_connections 2024-06-13T07:08:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.544+0000 7f9ff30d7b80 1 -- 172.21.15.7:0/1914784500 wait complete. 2024-06-13T07:08:32.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.546+0000 7f9ff30d7b80 20 client.16446 trim_cache size 0 max 16384 2024-06-13T07:08:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:08:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 msgr2=0x7f9fc80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 0x7f9fc80a52b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc09c7d0 tx=0x7f9fdc009040 comp rx=0 tx=0).stop 2024-06-13T07:08:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 msgr2=0x7f9fe01a7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b770 tx=0x7f9fdc0079a0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 shutdown_connections 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9fc80a2df0 0x7f9fc80a52b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe01121c0 0x7f9fe01a7e40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe010d0e0 0x7f9fe01a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 --2- 172.21.15.7:0/4067330396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe010da90 0x7f9fe01a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.547+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 >> 172.21.15.7:0/4067330396 conn(0x7f9fe006d650 msgr2=0x7f9fe0110850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.548+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 shutdown_connections 2024-06-13T07:08:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:32.548+0000 7f9fe8800700 1 -- 172.21.15.7:0/4067330396 wait complete. 2024-06-13T07:08:32.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 750.0 s, 585.2 tps, lat 165.065 ms stddev 178.675 2024-06-13T07:08:33.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:08:33.666 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:33 smithi007 ceph-mon[32077]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:08:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:33 smithi189 ceph-mon[36616]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:08:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:33 smithi175 ceph-mon[36886]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-06-13T07:08:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:35 smithi189 ceph-mon[36616]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.0 MiB/s wr, 496 op/s 2024-06-13T07:08:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:35 smithi175 ceph-mon[36886]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.0 MiB/s wr, 496 op/s 2024-06-13T07:08:35.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:35 smithi007 ceph-mon[32077]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.0 MiB/s wr, 496 op/s 2024-06-13T07:08:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.705+0000 7f9c08600700 1 -- 172.21.15.7:0/1416685060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 msgr2=0x7f9c00101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.705+0000 7f9c08600700 1 --2- 172.21.15.7:0/1416685060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c00101070 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b0a0 tx=0x7f9bf800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 -- 172.21.15.7:0/1416685060 shutdown_connections 2024-06-13T07:08:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 --2- 172.21.15.7:0/1416685060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c00101070 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 --2- 172.21.15.7:0/1416685060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c00106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 --2- 172.21.15.7:0/1416685060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c00069490 0x7f9c001006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 -- 172.21.15.7:0/1416685060 >> 172.21.15.7:0/1416685060 conn(0x7f9c00077590 msgr2=0x7f9c000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 -- 172.21.15.7:0/1416685060 shutdown_connections 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.706+0000 7f9c08600700 1 -- 172.21.15.7:0/1416685060 wait complete. 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.707+0000 7f9c08600700 1 Processor -- start 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.707+0000 7f9c08600700 1 -- start start 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c00069490 0x7f9c0019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c0019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00111730 con 0x7f9c001066b0 2024-06-13T07:08:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001115b0 con 0x7f9c00100bf0 2024-06-13T07:08:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c08600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c001118b0 con 0x7f9c00069490 2024-06-13T07:08:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c05200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c06600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c0019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c05200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52556/0 (socket says 172.21.15.7:52556) 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.708+0000 7f9c05200700 1 -- 172.21.15.7:0/3504082301 learned_addr learned my addr 172.21.15.7:0/3504082301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 -- 172.21.15.7:0/3504082301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c00069490 msgr2=0x7f9c0019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c00069490 0x7f9c0019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 -- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 msgr2=0x7f9c0019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c0019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 -- 172.21.15.7:0/3504082301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0019fe30 con 0x7f9c00100bf0 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c06600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c0019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.709+0000 7f9c05200700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.710+0000 7f9bfe000700 1 -- 172.21.15.7:0/3504082301 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00cbd0 con 0x7f9c00100bf0 2024-06-13T07:08:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.710+0000 7f9bfe000700 1 -- 172.21.15.7:0/3504082301 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec00cd70 con 0x7f9c00100bf0 2024-06-13T07:08:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.710+0000 7f9bfe000700 1 -- 172.21.15.7:0/3504082301 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec010430 con 0x7f9c00100bf0 2024-06-13T07:08:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.710+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c001a0120 con 0x7f9c00100bf0 2024-06-13T07:08:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.710+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c001a79a0 con 0x7f9c00100bf0 2024-06-13T07:08:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.714+0000 7f9bfe000700 1 -- 172.21.15.7:0/3504082301 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9bec004750 con 0x7f9c00100bf0 2024-06-13T07:08:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.717+0000 7f9bfe000700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 0x7f9be80a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.717+0000 7f9c12ee3b80 1 Processor -- start 2024-06-13T07:08:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.721+0000 7f9bfe000700 1 -- 172.21.15.7:0/3504082301 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9bec0cf5f0 con 0x7f9c00100bf0 2024-06-13T07:08:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.721+0000 7f9c12ee3b80 1 -- start start 2024-06-13T07:08:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.721+0000 7f9c05c00700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 0x7f9be80a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2d3b10 0x55f78d2d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f78d1cfa10 con 0x55f78d23ce40 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f78d212b00 con 0x55f78d2c05b0 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.722+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f78d1cd8f0 con 0x55f78d2d3b10 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.723+0000 7f9c05200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.723+0000 7f9c05200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52558/0 (socket says 172.21.15.7:52558) 2024-06-13T07:08:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.723+0000 7f9c05200700 1 -- 172.21.15.7:0/3671945622 learned_addr learned my addr 172.21.15.7:0/3671945622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.724+0000 7f9c05200700 1 -- 172.21.15.7:0/3671945622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2d3b10 msgr2=0x55f78d2d5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.724+0000 7f9c05200700 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2d3b10 0x55f78d2d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.724+0000 7f9c05200700 1 -- 172.21.15.7:0/3671945622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 msgr2=0x55f78d2c0070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.724+0000 7f9c05200700 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2c0070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.724+0000 7f9c05200700 1 -- 172.21.15.7:0/3671945622 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d1baf90 con 0x55f78d2c05b0 2024-06-13T07:08:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.726+0000 7f9c05200700 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9bec09ba40 tx=0x7f9bec09b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4edf1b4f7778eafb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.728+0000 7f9bf2a00700 1 -- 172.21.15.7:0/3671945622 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec09bab0 con 0x55f78d2c05b0 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.728+0000 7f9bf2a00700 1 -- 172.21.15.7:0/3671945622 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec0b4070 con 0x55f78d2c05b0 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.728+0000 7f9bf2a00700 1 -- 172.21.15.7:0/3671945622 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0cce60 con 0x55f78d2c05b0 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.729+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3671945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 msgr2=0x55f78d2c0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.729+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9bec09ba40 tx=0x7f9bec09b900 comp rx=0 tx=0).stop 2024-06-13T07:08:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.730+0000 7f9c05c00700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 0x7f9be80a51e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55f78d2c13f0 tx=0x7f9bf40096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.733+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3671945622 shutdown_connections 2024-06-13T07:08:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.733+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2d3b10 0x55f78d2d5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.733+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.733+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3671945622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2c0a30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.733+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3671945622 >> 172.21.15.7:0/3671945622 conn(0x55f78d1ed170 msgr2=0x55f78d2bc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.734+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3671945622 shutdown_connections 2024-06-13T07:08:36.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.735+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3671945622 wait complete. 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.735+0000 7f9c12ee3b80 1 -- start start 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.736+0000 7f9c12ee3b80 1 Processor -- start 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.736+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f78d1aef60 con 0x55f78d23ce40 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c05c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c06600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c06600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52570/0 (socket says 172.21.15.7:52570) 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c06600700 1 -- 172.21.15.7:0/3708033878 learned_addr learned my addr 172.21.15.7:0/3708033878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f78d20eac0 con 0x55f78d2d3b10 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.737+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f78d2d6770 con 0x55f78d2c05b0 2024-06-13T07:08:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c12ee3b80 10 client.0 _ll_register_callbacks cb 0x55f78d1ac720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c06600700 1 -- 172.21.15.7:0/3708033878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 msgr2=0x55f78d2d0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c06600700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c06600700 1 -- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 msgr2=0x55f78d2d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c06600700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.738+0000 7f9c06600700 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d1618d0 con 0x55f78d2d3b10 2024-06-13T07:08:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.739+0000 7f9c05c00700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.739+0000 7f9be7400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:36.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.739+0000 7f9c06600700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009a20 tx=0x7f9bf8003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926e447fc0efaebf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.749+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80078f0 con 0x55f78d2d3b10 2024-06-13T07:08:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.749+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec0b4070 con 0x55f78d2d3b10 2024-06-13T07:08:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.749+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9bf80076f0 con 0x55f78d2d3b10 2024-06-13T07:08:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.751+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bf80078f0 con 0x55f78d2d3b10 2024-06-13T07:08:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.751+0000 7f9c12ee3b80 10 client.27209 fetch_fsmap learned FSMap version 39 2024-06-13T07:08:36.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.751+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f78d2d7f70 con 0x55f78d2d3b10 2024-06-13T07:08:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec0b4070 con 0x55f78d2d3b10 2024-06-13T07:08:36.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9bf8011070 con 0x55f78d2d3b10 2024-06-13T07:08:36.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 10 client.27209 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:08:36.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 10 client.27209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:08:36.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 20 client.27209 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:36.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d2da500 0x55f78d2dc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 4 client.27209 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:36.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.752+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78d220000 con 0x55f78d2da500 2024-06-13T07:08:36.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.754+0000 7f9c05c00700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d2da500 0x55f78d2dc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.763+0000 7f9c05c00700 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d2da500 0x55f78d2dc980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55d56e58872a8db9 server_cookie=16dd4dbd5ac0ca62 in_seq=0 out_seq=0 2024-06-13T07:08:36.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.764+0000 7f9be7400700 10 client.27209 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:08:36.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.768+0000 7f9be7400700 1 -- 172.21.15.7:0/3708033878 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f78d220000 con 0x55f78d2da500 2024-06-13T07:08:36.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.768+0000 7f9be7400700 10 client.27209 handle_command_reply: tid=0 2024-06-13T07:08:36.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.768+0000 7f9c12ee3b80 1 client.27209 shutdown 2024-06-13T07:08:36.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.769+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 msgr2=0x55f78d2d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.769+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009a20 tx=0x7f9bf8003ce0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.769+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 shutdown_connections 2024-06-13T07:08:36.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.769+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2d3b10 0x55f78d2d0bb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.770+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d23ce40 0x55f78d2d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.770+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d2da500 0x55f78d2dc980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.770+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/3708033878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.770+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 >> 172.21.15.7:0/3708033878 conn(0x55f78d1ed170 msgr2=0x55f78d2bb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.770+0000 7f9be7400700 0 client.27209 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:08:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.771+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 shutdown_connections 2024-06-13T07:08:36.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.771+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/3708033878 wait complete. 2024-06-13T07:08:36.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.771+0000 7f9c12ee3b80 20 client.27209 trim_cache size 0 max 16384 2024-06-13T07:08:36.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.805+0000 7f9c12ee3b80 1 Processor -- start 2024-06-13T07:08:36.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.805+0000 7f9c12ee3b80 1 -- start start 2024-06-13T07:08:36.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d23ce40 0x55f78d2f0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00072ce0 con 0x55f78d2d3b10 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c00070540 con 0x55f78d23ce40 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c12ee3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0006e360 con 0x55f78d2c05b0 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c05200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c05200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53850/0 (socket says 172.21.15.7:53850) 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c05200700 1 -- 172.21.15.7:0/2161664301 learned_addr learned my addr 172.21.15.7:0/2161664301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.806+0000 7f9c06600700 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05c00700 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d23ce40 0x55f78d2f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05200700 1 -- 172.21.15.7:0/2161664301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d23ce40 msgr2=0x55f78d2f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05200700 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d23ce40 0x55f78d2f0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05200700 1 -- 172.21.15.7:0/2161664301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 msgr2=0x55f78d2d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05200700 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2d4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.807+0000 7f9c05200700 1 -- 172.21.15.7:0/2161664301 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d2d0440 con 0x55f78d2c05b0 2024-06-13T07:08:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.808+0000 7f9c05200700 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9bec098480 tx=0x7f9bec005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7abec5e69a6c2fed server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.809+0000 7f9bf2000700 1 -- 172.21.15.7:0/2161664301 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b9070 con 0x55f78d2c05b0 2024-06-13T07:08:36.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.809+0000 7f9bf2000700 1 -- 172.21.15.7:0/2161664301 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bec0b4070 con 0x55f78d2c05b0 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.810+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/2161664301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 msgr2=0x55f78d2d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.810+0000 7f9bf2000700 1 -- 172.21.15.7:0/2161664301 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b9070 con 0x55f78d2c05b0 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.810+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9bec098480 tx=0x7f9bec005770 comp rx=0 tx=0).stop 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/2161664301 shutdown_connections 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d23ce40 0x55f78d2f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/2161664301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d2c05b0 0x55f78d2d2eb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/2161664301 >> 172.21.15.7:0/2161664301 conn(0x55f78d1ed170 msgr2=0x55f78d2bbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/2161664301 shutdown_connections 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.811+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/2161664301 wait complete. 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.812+0000 7f9c12ee3b80 1 -- start start 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.812+0000 7f9c12ee3b80 1 Processor -- start 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.813+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.813+0000 7f9c12ee3b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c05c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c05c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53866/0 (socket says 172.21.15.7:53866) 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c05c00700 1 -- 172.21.15.7:0/4004561516 learned_addr learned my addr 172.21.15.7:0/4004561516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c05200700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0006ea00 con 0x55f78d2d3b10 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0006fe70 con 0x55f78d2c05b0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0006f0d0 con 0x55f78d23ce40 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.814+0000 7f9c06600700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c12ee3b80 10 client.0 _ll_register_callbacks cb 0x55f78d1ac720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c05200700 1 -- 172.21.15.7:0/4004561516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 msgr2=0x55f78d2e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c05200700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c05200700 1 -- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 msgr2=0x55f78d2dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c05200700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.815+0000 7f9c05200700 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d2ddca0 con 0x55f78d2c05b0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec0b4070 con 0x55f78d2c05b0 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9c06600700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9c05200700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2dd220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f78d2bd860 tx=0x7f9bec005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ecc370767581a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9be7400700 10 client.27215 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9c05c00700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9be7400700 1 -- 172.21.15.7:0/4004561516 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b9070 con 0x55f78d2c05b0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9be7400700 1 -- 172.21.15.7:0/4004561516 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f9bec099db0 con 0x55f78d2c05b0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.816+0000 7f9be7400700 1 -- 172.21.15.7:0/4004561516 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bec0b9070 con 0x55f78d2c05b0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 10 client.27215 fetch_fsmap learned FSMap version 39 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55f78d34d870 con 0x55f78d2c05b0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9be7400700 1 -- 172.21.15.7:0/4004561516 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f9bec0ba260 con 0x55f78d2c05b0 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 10 client.27215 fetch_fsmap finished waiting for FSMap version 39 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 10 client.27215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 20 client.27215 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d34da40 0x55f78d34fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 4 client.27215 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:08:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.817+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f78d21a580 con 0x55f78d34da40 2024-06-13T07:08:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.818+0000 7f9c05c00700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d34da40 0x55f78d34fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.835+0000 7f9c05c00700 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d34da40 0x55f78d34fe30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb9c16811aa59391 server_cookie=35eb57a083df3080 in_seq=0 out_seq=0 2024-06-13T07:08:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.835+0000 7f9be7400700 10 client.27215 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:08:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.836+0000 7f9be7400700 1 -- 172.21.15.7:0/4004561516 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f78d21a580 con 0x55f78d34da40 2024-06-13T07:08:36.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.836+0000 7f9be7400700 10 client.27215 handle_command_reply: tid=0 2024-06-13T07:08:36.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.836+0000 7f9c12ee3b80 1 client.27215 shutdown 2024-06-13T07:08:36.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 msgr2=0x55f78d2dd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2dd220 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f78d2bd860 tx=0x7f9bec005d30 comp rx=0 tx=0).stop 2024-06-13T07:08:36.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 shutdown_connections 2024-06-13T07:08:36.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f78d2d3b10 0x55f78d2dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x55f78d34da40 0x55f78d34fe30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f78d23ce40 0x55f78d2e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 --2- 172.21.15.7:0/4004561516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f78d2c05b0 0x55f78d2dd220 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.837+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 >> 172.21.15.7:0/4004561516 conn(0x55f78d1ed170 msgr2=0x55f78d2cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.838+0000 7f9be7400700 0 client.27215 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:08:36.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.839+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 shutdown_connections 2024-06-13T07:08:36.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.839+0000 7f9c12ee3b80 1 -- 172.21.15.7:0/4004561516 wait complete. 2024-06-13T07:08:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.840+0000 7f9c12ee3b80 20 client.27215 trim_cache size 0 max 16384 2024-06-13T07:08:36.840 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:08:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 msgr2=0x7f9be80a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 0x7f9be80a51e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55f78d2c13f0 tx=0x7f9bf40096c0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 msgr2=0x7f9c0019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00ed00 comp rx=0 tx=0).stop 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 shutdown_connections 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c001066b0 0x7f9c0019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c00069490 0x7f9c0019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f9be80a2d20 0x7f9be80a51e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 --2- 172.21.15.7:0/3504082301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9c00100bf0 0x7f9c0019b360 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:36.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.843+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 >> 172.21.15.7:0/3504082301 conn(0x7f9c00077590 msgr2=0x7f9c00103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:36.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.844+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 shutdown_connections 2024-06-13T07:08:36.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:36.844+0000 7f9c08600700 1 -- 172.21.15.7:0/3504082301 wait complete. 2024-06-13T07:08:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:37 smithi189 ceph-mon[36616]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-06-13T07:08:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:37 smithi175 ceph-mon[36886]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-06-13T07:08:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 755.0 s, 438.0 tps, lat 231.325 ms stddev 278.657 2024-06-13T07:08:37.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:37 smithi007 ceph-mon[32077]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-06-13T07:08:39.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:08:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:39 smithi175 ceph-mon[36886]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-13T07:08:39.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:39 smithi007 ceph-mon[32077]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-13T07:08:40.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:39 smithi189 ceph-mon[36616]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.0 MiB/s wr, 498 op/s 2024-06-13T07:08:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 -- 172.21.15.7:0/1402077630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103c20 msgr2=0x7efd0c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 --2- 172.21.15.7:0/1402077630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103c20 0x7efd0c104040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009860 tx=0x7efcfc02a340 comp rx=0 tx=0).stop 2024-06-13T07:08:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 -- 172.21.15.7:0/1402077630 shutdown_connections 2024-06-13T07:08:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 --2- 172.21.15.7:0/1402077630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0c1057d0 0x7efd0c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 --2- 172.21.15.7:0/1402077630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c104e10 0x7efd0c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 --2- 172.21.15.7:0/1402077630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c103c20 0x7efd0c104040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.593+0000 7efd11c00640 1 -- 172.21.15.7:0/1402077630 >> 172.21.15.7:0/1402077630 conn(0x7efd0c0ff190 msgr2=0x7efd0c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.594+0000 7efd11c00640 1 -- 172.21.15.7:0/1402077630 shutdown_connections 2024-06-13T07:08:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.594+0000 7efd11c00640 1 -- 172.21.15.7:0/1402077630 wait complete. 2024-06-13T07:08:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.595+0000 7efd11c00640 1 Processor -- start 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.595+0000 7efd11c00640 1 -- start start 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0c103c20 0x7efd0c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c104e10 0x7efd0c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10f870 con 0x7efd0c103c20 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10f6f0 con 0x7efd0c104e10 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd11c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c10f9f0 con 0x7efd0c1057d0 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.596+0000 7efd0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53874/0 (socket says 172.21.15.7:53874) 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 -- 172.21.15.7:0/807235093 learned_addr learned my addr 172.21.15.7:0/807235093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 -- 172.21.15.7:0/807235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c104e10 msgr2=0x7efd0c19e800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c104e10 0x7efd0c19e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 -- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0c103c20 msgr2=0x7efd0c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0c103c20 0x7efd0c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.597+0000 7efd0b400640 1 -- 172.21.15.7:0/807235093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c06b4f0 con 0x7efd0c1057d0 2024-06-13T07:08:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.598+0000 7efd0b400640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd04009fd0 tx=0x7efd0400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.599+0000 7efd03400640 1 -- 172.21.15.7:0/807235093 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004b20 con 0x7efd0c1057d0 2024-06-13T07:08:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.599+0000 7efd03400640 1 -- 172.21.15.7:0/807235093 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd04003e50 con 0x7efd0c1057d0 2024-06-13T07:08:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.599+0000 7efd03400640 1 -- 172.21.15.7:0/807235093 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004b20 con 0x7efd0c1057d0 2024-06-13T07:08:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.600+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c06b7e0 con 0x7efd0c1057d0 2024-06-13T07:08:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.600+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c06bce0 con 0x7efd0c1057d0 2024-06-13T07:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.605+0000 7efd03400640 1 -- 172.21.15.7:0/807235093 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efd0401d020 con 0x7efd0c1057d0 2024-06-13T07:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.607+0000 7efd14b6a740 1 Processor -- start 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.607+0000 7efd14b6a740 1 -- start start 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae34bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55678ad90420 con 0x55678ae2d5f0 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55678ad37c50 con 0x55678ae2cc00 2024-06-13T07:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd14b6a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55678ad8c3e0 con 0x55678ae32760 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52606/0 (socket says 172.21.15.7:52606) 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 -- 172.21.15.7:0/3073574464 learned_addr learned my addr 172.21.15.7:0/3073574464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae34bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 -- 172.21.15.7:0/3073574464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 msgr2=0x55678ae34bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae34bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0a000640 1 -- 172.21.15.7:0/3073574464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 msgr2=0x55678ae320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.608+0000 7efd0aa00640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd0a000640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd0a000640 1 -- 172.21.15.7:0/3073574464 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55678aaa4210 con 0x55678ae2cc00 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd0aa00640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd03400640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 0x7efcf00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd0a000640 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009f30 tx=0x7efcf8009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57ef965c0533eb64 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efd03400640 1 -- 172.21.15.7:0/807235093 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efd040cef70 con 0x7efd0c1057d0 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efcf7e00640 1 -- 172.21.15.7:0/3073574464 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8018030 con 0x55678ae2cc00 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efcf7e00640 1 -- 172.21.15.7:0/3073574464 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf8005500 con 0x55678ae2cc00 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.609+0000 7efcf7e00640 1 -- 172.21.15.7:0/3073574464 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8012450 con 0x55678ae2cc00 2024-06-13T07:08:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.610+0000 7efd0aa00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 0x7efcf00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.610+0000 7efd14b6a740 1 -- 172.21.15.7:0/3073574464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 msgr2=0x55678ae2d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.610+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009f30 tx=0x7efcf8009760 comp rx=0 tx=0).stop 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 -- 172.21.15.7:0/3073574464 shutdown_connections 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae34bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3073574464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae2d020 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 -- 172.21.15.7:0/3073574464 >> 172.21.15.7:0/3073574464 conn(0x55678adf4fb0 msgr2=0x55678ae28680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd0aa00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 0x7efcf00a5380 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efcfc035040 tx=0x7efcfc030340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 -- 172.21.15.7:0/3073574464 shutdown_connections 2024-06-13T07:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 -- 172.21.15.7:0/3073574464 wait complete. 2024-06-13T07:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.611+0000 7efd14b6a740 1 -- start start 2024-06-13T07:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.612+0000 7efd14b6a740 1 Processor -- start 2024-06-13T07:08:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2d5f0 0x55678ae37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55678ae35710 con 0x55678ae32760 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55678ad4b0b0 con 0x55678ae2d5f0 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55678ae35890 con 0x55678ae2cc00 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.613+0000 7efd14b6a740 10 client.0 _ll_register_callbacks cb 0x55678ad36f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.614+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.614+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:53902/0 (socket says 172.21.15.7:53902) 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.614+0000 7efd0aa00640 1 -- 172.21.15.7:0/2300945848 learned_addr learned my addr 172.21.15.7:0/2300945848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.615+0000 7efd0b400640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.615+0000 7efd0a000640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2d5f0 0x55678ae37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.616+0000 7efd0aa00640 1 -- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2d5f0 msgr2=0x55678ae37450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.616+0000 7efd0aa00640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2d5f0 0x55678ae37450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.616+0000 7efd0aa00640 1 -- 172.21.15.7:0/2300945848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 msgr2=0x55678ae3e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.616+0000 7efd0aa00640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae3e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.616+0000 7efd0aa00640 1 -- 172.21.15.7:0/2300945848 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55678ab70620 con 0x55678ae2cc00 2024-06-13T07:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.617+0000 7efd0b400640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae3e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.619+0000 7efd0aa00640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd0c104c70 tx=0x7efcfc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd06de7467d55e62 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.619+0000 7efcf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.622+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc04e060 con 0x55678ae2cc00 2024-06-13T07:08:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.622+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efcfc0046c0 con 0x55678ae2cc00 2024-06-13T07:08:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.623+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf8005500 con 0x55678ae2cc00 2024-06-13T07:08:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.623+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efcfc0048c0 con 0x55678ae2cc00 2024-06-13T07:08:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.623+0000 7efd14b6a740 10 client.27127 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.623+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55678ae41830 con 0x55678ae2cc00 2024-06-13T07:08:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf8005500 con 0x55678ae2cc00 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7efcfc003c70 con 0x55678ae2cc00 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 10 client.27127 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 10 client.27127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 20 client.27127 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678ae42f20 0x55678ae45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 4 client.27127 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.626+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55678ad9db50 con 0x55678ae42f20 2024-06-13T07:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.630+0000 7efd0a000640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678ae42f20 0x55678ae45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.634+0000 7efd0a000640 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678ae42f20 0x55678ae45350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c17db6cf461283 server_cookie=f85ca4aa330ea9ae in_seq=0 out_seq=0 2024-06-13T07:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.634+0000 7efcf4c00640 10 client.27127 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.635+0000 7efcf4c00640 1 -- 172.21.15.7:0/2300945848 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55678ad9db50 con 0x55678ae42f20 2024-06-13T07:08:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.635+0000 7efcf4c00640 10 client.27127 handle_command_reply: tid=0 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.635+0000 7efd14b6a740 1 client.27127 shutdown 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 msgr2=0x55678ae36f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efd0c104c70 tx=0x7efcfc000bc0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 shutdown_connections 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678ae42f20 0x55678ae45350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae3e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2d5f0 0x55678ae37450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 --2- 172.21.15.7:0/2300945848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2cc00 0x55678ae36f10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efcf4c00640 0 client.27127 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 >> 172.21.15.7:0/2300945848 conn(0x55678adf4fb0 msgr2=0x55678ae27840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 shutdown_connections 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.636+0000 7efd14b6a740 1 -- 172.21.15.7:0/2300945848 wait complete. 2024-06-13T07:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.637+0000 7efd14b6a740 20 client.27127 trim_cache size 0 max 16384 2024-06-13T07:08:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.684+0000 7efd14b6a740 1 Processor -- start 2024-06-13T07:08:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.685+0000 7efd14b6a740 1 -- start start 2024-06-13T07:08:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.685+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2d5f0 0x55678ae3e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd14b6a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1bbac0 con 0x55678ae32760 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd14b6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1b9330 con 0x55678ae2cc00 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd14b6a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c0736e0 con 0x55678ae2d5f0 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52624/0 (socket says 172.21.15.7:52624) 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.686+0000 7efd0b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60942/0 (socket says 172.21.15.7:60942) 2024-06-13T07:08:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0b400640 1 -- 172.21.15.7:0/543075946 learned_addr learned my addr 172.21.15.7:0/543075946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0aa00640 1 -- 172.21.15.7:0/543075946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2d5f0 msgr2=0x55678ae3e450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0aa00640 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2d5f0 0x55678ae3e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0aa00640 1 -- 172.21.15.7:0/543075946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 msgr2=0x55678ae2b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0aa00640 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0aa00640 1 -- 172.21.15.7:0/543075946 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55678ae23d40 con 0x55678ae2cc00 2024-06-13T07:08:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.687+0000 7efd0b400640 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.688+0000 7efd0aa00640 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efcfc049610 tx=0x7efcfc005510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345e9f118ec0a893 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.688+0000 7efcf7400640 1 -- 172.21.15.7:0/543075946 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc058080 con 0x55678ae2cc00 2024-06-13T07:08:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.688+0000 7efcf7400640 1 -- 172.21.15.7:0/543075946 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcfc038040 con 0x55678ae2cc00 2024-06-13T07:08:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.688+0000 7efcf7400640 1 -- 172.21.15.7:0/543075946 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0048c0 con 0x55678ae2cc00 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.689+0000 7efd14b6a740 1 -- 172.21.15.7:0/543075946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 msgr2=0x55678ae37230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.689+0000 7efd14b6a740 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efcfc049610 tx=0x7efcfc005510 comp rx=0 tx=0).stop 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.690+0000 7efd14b6a740 1 -- 172.21.15.7:0/543075946 shutdown_connections 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.690+0000 7efd14b6a740 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae32760 0x55678ae2b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.690+0000 7efd14b6a740 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae2d5f0 0x55678ae3e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.690+0000 7efd14b6a740 1 --2- 172.21.15.7:0/543075946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae37230 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.690+0000 7efd14b6a740 1 -- 172.21.15.7:0/543075946 >> 172.21.15.7:0/543075946 conn(0x55678adf4fb0 msgr2=0x55678ae36b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.692+0000 7efd14b6a740 1 -- 172.21.15.7:0/543075946 shutdown_connections 2024-06-13T07:08:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.692+0000 7efd14b6a740 1 -- 172.21.15.7:0/543075946 wait complete. 2024-06-13T07:08:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.694+0000 7efd14b6a740 1 -- start start 2024-06-13T07:08:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.694+0000 7efd14b6a740 1 Processor -- start 2024-06-13T07:08:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae3ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1b77f0 con 0x55678ae2d5f0 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1b8c60 con 0x55678ae2cc00 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c1b7ec0 con 0x55678ae32760 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.695+0000 7efd14b6a740 10 client.0 _ll_register_callbacks cb 0x7efcfc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.696+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.696+0000 7efd0aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae3b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:52634/0 (socket says 172.21.15.7:52634) 2024-06-13T07:08:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.697+0000 7efd0aa00640 1 -- 172.21.15.7:0/3430925642 learned_addr learned my addr 172.21.15.7:0/3430925642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.697+0000 7efd0b400640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.698+0000 7efd0a000640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae3ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.698+0000 7efd0b400640 1 -- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 msgr2=0x55678ae3b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.699+0000 7efd0b400640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae3b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.699+0000 7efd0b400640 1 -- 172.21.15.7:0/3430925642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 msgr2=0x55678ae3ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.699+0000 7efd0b400640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae3ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.699+0000 7efd0b400640 1 -- 172.21.15.7:0/3430925642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55678ae3d000 con 0x55678ae32760 2024-06-13T07:08:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.700+0000 7efd0a000640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae3ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.701+0000 7efd0b400640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae3ca10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd0409b190 tx=0x7efd0409adf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53108e497b9a198c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.701+0000 7efcf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.705+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0401a040 con 0x55678ae32760 2024-06-13T07:08:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.705+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efd04004240 con 0x55678ae32760 2024-06-13T07:08:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.705+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcfc038040 con 0x55678ae32760 2024-06-13T07:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.706+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efd0409f840 con 0x55678ae32760 2024-06-13T07:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.706+0000 7efd14b6a740 10 client.27130 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.706+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55678ae3e380 con 0x55678ae32760 2024-06-13T07:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcfc038040 con 0x55678ae32760 2024-06-13T07:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7efd0400b040 con 0x55678ae32760 2024-06-13T07:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efd14b6a740 10 client.27130 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efd14b6a740 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efd14b6a740 20 client.27130 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.709+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678aeadff0 0x55678aeb03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.710+0000 7efd14b6a740 4 client.27130 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.710+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55678ad97ea0 con 0x55678aeadff0 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.710+0000 7efd0aa00640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678aeadff0 0x55678aeb03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.711+0000 7efd0aa00640 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678aeadff0 0x55678aeb03e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3f356aec9ef76c server_cookie=7aa7b886c7ea3215 in_seq=0 out_seq=0 2024-06-13T07:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.711+0000 7efcf4c00640 10 client.27130 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efcf4c00640 1 -- 172.21.15.7:0/3430925642 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55678ad97ea0 con 0x55678aeadff0 2024-06-13T07:08:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efcf4c00640 10 client.27130 handle_command_reply: tid=0 2024-06-13T07:08:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 client.27130 shutdown 2024-06-13T07:08:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 msgr2=0x55678ae3ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae3ca10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd0409b190 tx=0x7efd0409adf0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 shutdown_connections 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55678aeadff0 0x55678aeb03e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55678ae32760 0x55678ae3ca10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55678ae2d5f0 0x55678ae3ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 --2- 172.21.15.7:0/3430925642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55678ae2cc00 0x55678ae3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.855+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 >> 172.21.15.7:0/3430925642 conn(0x55678adf4fb0 msgr2=0x55678ae26fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.856+0000 7efcf4c00640 0 client.27130 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.856+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 shutdown_connections 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.856+0000 7efd14b6a740 1 -- 172.21.15.7:0/3430925642 wait complete. 2024-06-13T07:08:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.856+0000 7efd14b6a740 20 client.27130 trim_cache size 0 max 16384 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.858+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 msgr2=0x7efcf00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.858+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 0x7efcf00a5380 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efcfc035040 tx=0x7efcfc030340 comp rx=0 tx=0).stop 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.858+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 msgr2=0x7efd0c1a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.858+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd04009fd0 tx=0x7efd0400c800 comp rx=0 tx=0).stop 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 shutdown_connections 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efcf00a2ec0 0x7efcf00a5380 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd0c1057d0 0x7efd0c1a5820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd0c104e10 0x7efd0c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 --2- 172.21.15.7:0/807235093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0c103c20 0x7efd0c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 >> 172.21.15.7:0/807235093 conn(0x7efd0c0ff190 msgr2=0x7efd0c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 shutdown_connections 2024-06-13T07:08:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:40.859+0000 7efd11c00640 1 -- 172.21.15.7:0/807235093 wait complete. 2024-06-13T07:08:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:41 smithi175 ceph-mon[36886]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.3 MiB/s wr, 551 op/s 2024-06-13T07:08:41.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:41 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:41 smithi007 ceph-mon[32077]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.3 MiB/s wr, 551 op/s 2024-06-13T07:08:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:41 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:42.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:41 smithi189 ceph-mon[36616]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.3 MiB/s wr, 551 op/s 2024-06-13T07:08:42.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:41 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:08:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 760.0 s, 453.2 tps, lat 219.809 ms stddev 259.691 2024-06-13T07:08:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:43 smithi175 ceph-mon[36886]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-06-13T07:08:43.868 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:43 smithi007 ceph-mon[32077]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-06-13T07:08:44.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:43 smithi189 ceph-mon[36616]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-06-13T07:08:45.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.005+0000 7fb324c00700 1 -- 172.21.15.7:0/1764754256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102fd0 msgr2=0x7fb320107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.005+0000 7fb324c00700 1 --2- 172.21.15.7:0/1764754256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102fd0 0x7fb320107020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00dc80 tx=0x7fb30c00df90 comp rx=0 tx=0).stop 2024-06-13T07:08:45.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.007+0000 7fb324c00700 1 -- 172.21.15.7:0/1764754256 shutdown_connections 2024-06-13T07:08:45.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.007+0000 7fb324c00700 1 --2- 172.21.15.7:0/1764754256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.007+0000 7fb324c00700 1 --2- 172.21.15.7:0/1764754256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102620 0x7fb320102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.007+0000 7fb324c00700 1 --2- 172.21.15.7:0/1764754256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102fd0 0x7fb320107020 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.007+0000 7fb324c00700 1 -- 172.21.15.7:0/1764754256 >> 172.21.15.7:0/1764754256 conn(0x7fb3200fdf10 msgr2=0x7fb320100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.008+0000 7fb324c00700 1 -- 172.21.15.7:0/1764754256 shutdown_connections 2024-06-13T07:08:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.008+0000 7fb324c00700 1 -- 172.21.15.7:0/1764754256 wait complete. 2024-06-13T07:08:45.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.009+0000 7fb324c00700 1 Processor -- start 2024-06-13T07:08:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.012+0000 7fb324c00700 1 -- start start 2024-06-13T07:08:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 0x7fb320073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 0x7fb320074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201105f0 con 0x7fb320102fd0 2024-06-13T07:08:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb320110470 con 0x7fb320102620 2024-06-13T07:08:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.013+0000 7fb324c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb320110770 con 0x7fb320107700 2024-06-13T07:08:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.014+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.014+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:32808/0 (socket says 172.21.15.7:32808) 2024-06-13T07:08:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.014+0000 7fb31f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 0x7fb320073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.014+0000 7fb31fe00700 1 -- 172.21.15.7:0/809745144 learned_addr learned my addr 172.21.15.7:0/809745144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:45.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.014+0000 7fb31ea00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 0x7fb320074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.015+0000 7fb31f400700 1 -- 172.21.15.7:0/809745144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 msgr2=0x7fb32006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31f400700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31f400700 1 -- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 msgr2=0x7fb320074100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31f400700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 0x7fb320074100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31f400700 1 -- 172.21.15.7:0/809745144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32006d7a0 con 0x7fb320102620 2024-06-13T07:08:45.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31ea00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 0x7fb320074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:45.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.016+0000 7fb31fe00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:45.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb31f400700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 0x7fb320073bc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb31800ef10 tx=0x7fb31800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb317e00700 1 -- 172.21.15.7:0/809745144 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800cad0 con 0x7fb320102620 2024-06-13T07:08:45.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb317e00700 1 -- 172.21.15.7:0/809745144 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb318019070 con 0x7fb320102620 2024-06-13T07:08:45.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb317e00700 1 -- 172.21.15.7:0/809745144 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180146e0 con 0x7fb320102620 2024-06-13T07:08:45.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32006da30 con 0x7fb320102620 2024-06-13T07:08:45.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.018+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb320071970 con 0x7fb320102620 2024-06-13T07:08:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.024+0000 7fb317e00700 1 -- 172.21.15.7:0/809745144 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb318012050 con 0x7fb320102620 2024-06-13T07:08:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.025+0000 7fb330a84b80 1 Processor -- start 2024-06-13T07:08:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.025+0000 7fb330a84b80 1 -- start start 2024-06-13T07:08:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.026+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.026+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 0x55664b1a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.026+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.026+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:32818/0 (socket says 172.21.15.7:32818) 2024-06-13T07:08:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.027+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1aa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.027+0000 7fb330a84b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55664b0aca10 con 0x55664b1a8530 2024-06-13T07:08:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.027+0000 7fb330a84b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55664b0efb00 con 0x55664b19d680 2024-06-13T07:08:45.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.027+0000 7fb330a84b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55664b0aa8f0 con 0x55664b19cd60 2024-06-13T07:08:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.027+0000 7fb317e00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 0x7fb3040a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.028+0000 7fb317e00700 1 -- 172.21.15.7:0/809745144 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb3180cf340 con 0x7fb320102620 2024-06-13T07:08:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.028+0000 7fb31f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 0x55664b1a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.028+0000 7fb31f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 0x55664b1a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42290/0 (socket says 172.21.15.7:42290) 2024-06-13T07:08:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.028+0000 7fb31f400700 1 -- 172.21.15.7:0/2164359446 learned_addr learned my addr 172.21.15.7:0/2164359446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.028+0000 7fb31ea00700 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.029+0000 7fb31fe00700 1 -- 172.21.15.7:0/2164359446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 msgr2=0x55664b1a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.029+0000 7fb31fe00700 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 0x55664b1a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.029+0000 7fb31fe00700 1 -- 172.21.15.7:0/2164359446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 msgr2=0x55664b1aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.029+0000 7fb31fe00700 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.029+0000 7fb31fe00700 1 -- 172.21.15.7:0/2164359446 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664b097f90 con 0x55664b19cd60 2024-06-13T07:08:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.030+0000 7fb31fe00700 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb310007ad0 tx=0x7fb310007de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5903f4ef417cbda3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.030+0000 7fb31fe00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 0x7fb3040a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.030+0000 7fb314c00700 1 -- 172.21.15.7:0/2164359446 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310018020 con 0x55664b19cd60 2024-06-13T07:08:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.030+0000 7fb314c00700 1 -- 172.21.15.7:0/2164359446 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310010040 con 0x55664b19cd60 2024-06-13T07:08:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.030+0000 7fb314c00700 1 -- 172.21.15.7:0/2164359446 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31001c610 con 0x55664b19cd60 2024-06-13T07:08:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 -- 172.21.15.7:0/2164359446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 msgr2=0x55664b19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb310007ad0 tx=0x7fb310007de0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 -- 172.21.15.7:0/2164359446 shutdown_connections 2024-06-13T07:08:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b19cd60 0x55664b19d140 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2164359446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19d680 0x55664b1a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.031+0000 7fb330a84b80 1 -- 172.21.15.7:0/2164359446 >> 172.21.15.7:0/2164359446 conn(0x55664b0ca170 msgr2=0x55664b199280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.032+0000 7fb31fe00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 0x7fb3040a5260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb310007ad0 tx=0x7fb310007de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.032+0000 7fb330a84b80 1 -- 172.21.15.7:0/2164359446 shutdown_connections 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.032+0000 7fb330a84b80 1 -- 172.21.15.7:0/2164359446 wait complete. 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.033+0000 7fb330a84b80 1 -- start start 2024-06-13T07:08:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.033+0000 7fb330a84b80 1 Processor -- start 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b1a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 0x55664b1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55664b08bf60 con 0x55664b1a8530 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55664b0ebac0 con 0x55664b19cd60 2024-06-13T07:08:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55664b1aafc0 con 0x55664b1a5610 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb330a84b80 10 client.0 _ll_register_callbacks cb 0x55664b085380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 0x55664b1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 0x55664b1a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:32822/0 (socket says 172.21.15.7:32822) 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.034+0000 7fb31fe00700 1 -- 172.21.15.7:0/2162648509 learned_addr learned my addr 172.21.15.7:0/2162648509 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.035+0000 7fb31f400700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b1a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.035+0000 7fb31ea00700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.036+0000 7fb31f400700 1 -- 172.21.15.7:0/2162648509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 msgr2=0x55664b1a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.036+0000 7fb31f400700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 0x55664b1a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.036+0000 7fb31f400700 1 -- 172.21.15.7:0/2162648509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 msgr2=0x55664b1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.036+0000 7fb31f400700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb31f400700 1 -- 172.21.15.7:0/2162648509 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664b03e8d0 con 0x55664b19cd60 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb31f400700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b1a7dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb3180a3fa0 tx=0x7fb318012620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b521aaf6963a55bd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb309600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180b4060 con 0x55664b19cd60 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb3180a4040 con 0x55664b19cd60 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.037+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb310010040 con 0x55664b19cd60 2024-06-13T07:08:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.038+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb3180b9050 con 0x55664b19cd60 2024-06-13T07:08:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.038+0000 7fb330a84b80 10 client.27239 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.038+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55664b1c6c30 con 0x55664b19cd60 2024-06-13T07:08:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.038+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310010040 con 0x55664b19cd60 2024-06-13T07:08:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.039+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fb31809cb90 con 0x55664b19cd60 2024-06-13T07:08:45.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.039+0000 7fb330a84b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.040+0000 7fb330a84b80 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.040+0000 7fb330a84b80 20 client.27239 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.040+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b1c9270 0x55664b1cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.040+0000 7fb330a84b80 4 client.27239 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.040+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55664b0fd000 con 0x55664b1c9270 2024-06-13T07:08:45.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.041+0000 7fb31ea00700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b1c9270 0x55664b1cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.042+0000 7fb31ea00700 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b1c9270 0x55664b1cb660 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7c68bd1b91aa50 server_cookie=bf201a5aacc7008b in_seq=0 out_seq=0 2024-06-13T07:08:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.042+0000 7fb309600700 10 client.27239 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.043+0000 7fb309600700 1 -- 172.21.15.7:0/2162648509 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55664b0fd000 con 0x55664b1c9270 2024-06-13T07:08:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.043+0000 7fb309600700 10 client.27239 handle_command_reply: tid=0 2024-06-13T07:08:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.043+0000 7fb330a84b80 1 client.27239 shutdown 2024-06-13T07:08:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 msgr2=0x55664b1a7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b1a7dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb3180a3fa0 tx=0x7fb318012620 comp rx=0 tx=0).stop 2024-06-13T07:08:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 shutdown_connections 2024-06-13T07:08:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b1c9270 0x55664b1cb660 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a5610 0x55664b1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b1a7dd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 --2- 172.21.15.7:0/2162648509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a8530 0x55664b1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 >> 172.21.15.7:0/2162648509 conn(0x55664b0ca170 msgr2=0x55664b198e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb309600700 0 client.27239 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 shutdown_connections 2024-06-13T07:08:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.045+0000 7fb330a84b80 1 -- 172.21.15.7:0/2162648509 wait complete. 2024-06-13T07:08:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.046+0000 7fb330a84b80 20 client.27239 trim_cache size 0 max 16384 2024-06-13T07:08:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.085+0000 7fb330a84b80 1 Processor -- start 2024-06-13T07:08:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.086+0000 7fb330a84b80 1 -- start start 2024-06-13T07:08:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.087+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b19cd60 0x55664b1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.087+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.087+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb330a84b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b70d0 con 0x55664b19cd60 2024-06-13T07:08:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb330a84b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b4900 con 0x55664b1a5610 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb330a84b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b2750 con 0x55664b1a8530 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42312/0 (socket says 172.21.15.7:42312) 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 -- 172.21.15.7:0/4088639835 learned_addr learned my addr 172.21.15.7:0/4088639835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 -- 172.21.15.7:0/4088639835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 msgr2=0x55664b1d0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:08:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1d0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 -- 172.21.15.7:0/4088639835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b19cd60 msgr2=0x55664b1a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b19cd60 0x55664b1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.088+0000 7fb31ea00700 1 -- 172.21.15.7:0/4088639835 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664b19ebb0 con 0x55664b1a5610 2024-06-13T07:08:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.089+0000 7fb31ea00700 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb32006ada0 tx=0x7fb30c00d400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf7bb4924731f1d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.089+0000 7fb30be00700 1 -- 172.21.15.7:0/4088639835 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c01c040 con 0x55664b1a5610 2024-06-13T07:08:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.089+0000 7fb30be00700 1 -- 172.21.15.7:0/4088639835 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30c005040 con 0x55664b1a5610 2024-06-13T07:08:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.089+0000 7fb30be00700 1 -- 172.21.15.7:0/4088639835 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00f940 con 0x55664b1a5610 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.090+0000 7fb330a84b80 1 -- 172.21.15.7:0/4088639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 msgr2=0x55664b1a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.090+0000 7fb330a84b80 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb32006ada0 tx=0x7fb30c00d400 comp rx=0 tx=0).stop 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 -- 172.21.15.7:0/4088639835 shutdown_connections 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b19cd60 0x55664b1a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 --2- 172.21.15.7:0/4088639835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b1a5610 0x55664b1a4ec0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 -- 172.21.15.7:0/4088639835 >> 172.21.15.7:0/4088639835 conn(0x55664b0ca170 msgr2=0x55664b1cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 -- 172.21.15.7:0/4088639835 shutdown_connections 2024-06-13T07:08:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.091+0000 7fb330a84b80 1 -- 172.21.15.7:0/4088639835 wait complete. 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.092+0000 7fb330a84b80 1 -- start start 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.092+0000 7fb330a84b80 1 Processor -- start 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b22c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a5610 0x55664b22c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b2dc0 con 0x55664b1a5610 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b4230 con 0x55664b19cd60 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201b3490 con 0x55664b1a8530 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb330a84b80 10 client.0 _ll_register_callbacks cb 0x55664b085380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.093+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:32834/0 (socket says 172.21.15.7:32834) 2024-06-13T07:08:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 -- 172.21.15.7:0/3286815342 learned_addr learned my addr 172.21.15.7:0/3286815342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 -- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 msgr2=0x55664b22c260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b22c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 -- 172.21.15.7:0/3286815342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a5610 msgr2=0x55664b22c7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a5610 0x55664b22c7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.094+0000 7fb31fe00700 1 -- 172.21.15.7:0/3286815342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664b1bf030 con 0x55664b1a8530 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.095+0000 7fb31fe00700 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb310004730 tx=0x7fb31001f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2d0df2332a84bf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.095+0000 7fb309600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.096+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310036060 con 0x55664b1a8530 2024-06-13T07:08:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.097+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb310014610 con 0x55664b1a8530 2024-06-13T07:08:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.097+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30c005040 con 0x55664b1a8530 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.097+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb310018040 con 0x55664b1a8530 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.097+0000 7fb330a84b80 10 client.27142 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.097+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55664b233a50 con 0x55664b1a8530 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.098+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30c005040 con 0x55664b1a8530 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.099+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fb310030030 con 0x55664b1a8530 2024-06-13T07:08:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 10 client.27142 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:45.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 10 client.27142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:45.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 20 client.27142 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:45.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b233ee0 0x55664b236320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:45.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 4 client.27142 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.100+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55664b0f7580 con 0x55664b233ee0 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.101+0000 7fb31f400700 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b233ee0 0x55664b236320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.102+0000 7fb31f400700 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b233ee0 0x55664b236320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a58e74b1c0fe199 server_cookie=1e41844b45a34103 in_seq=0 out_seq=0 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.102+0000 7fb309600700 10 client.27142 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.102+0000 7fb309600700 1 -- 172.21.15.7:0/3286815342 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55664b0f7580 con 0x55664b233ee0 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.102+0000 7fb309600700 10 client.27142 handle_command_reply: tid=0 2024-06-13T07:08:45.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.102+0000 7fb330a84b80 1 client.27142 shutdown 2024-06-13T07:08:45.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 msgr2=0x55664b1be930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb310004730 tx=0x7fb31001f040 comp rx=0 tx=0).stop 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 shutdown_connections 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55664b233ee0 0x55664b236320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55664b1a8530 0x55664b1be930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55664b19cd60 0x55664b22c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.103+0000 7fb330a84b80 1 --2- 172.21.15.7:0/3286815342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55664b1a5610 0x55664b22c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.104+0000 7fb309600700 0 client.27142 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:45.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.104+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 >> 172.21.15.7:0/3286815342 conn(0x55664b0ca170 msgr2=0x55664b1a6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.104+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 shutdown_connections 2024-06-13T07:08:45.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.105+0000 7fb330a84b80 1 -- 172.21.15.7:0/3286815342 wait complete. 2024-06-13T07:08:45.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.105+0000 7fb330a84b80 20 client.27142 trim_cache size 0 max 16384 2024-06-13T07:08:45.107 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:08:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.106+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 msgr2=0x7fb3040a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.106+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 0x7fb3040a5260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb310007ad0 tx=0x7fb310007de0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.106+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 msgr2=0x7fb320073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.106+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 0x7fb320073bc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb31800ef10 tx=0x7fb31800c3a0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 shutdown_connections 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb320107700 0x7fb32006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb3040a2da0 0x7fb3040a5260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb320102620 0x7fb320073bc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 --2- 172.21.15.7:0/809745144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb320102fd0 0x7fb320074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 >> 172.21.15.7:0/809745144 conn(0x7fb3200fdf10 msgr2=0x7fb3200fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 shutdown_connections 2024-06-13T07:08:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:45.107+0000 7fb324c00700 1 -- 172.21.15.7:0/809745144 wait complete. 2024-06-13T07:08:45.888 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:45 smithi189 ceph-mon[36616]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 504 op/s 2024-06-13T07:08:45.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:45 smithi007 ceph-mon[32077]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 504 op/s 2024-06-13T07:08:46.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:45 smithi175 ceph-mon[36886]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 504 op/s 2024-06-13T07:08:47.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:08:47.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:08:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 765.0 s, 452.8 tps, lat 221.288 ms stddev 228.942 2024-06-13T07:08:47.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:47 smithi007 ceph-mon[32077]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:08:48.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:47 smithi189 ceph-mon[36616]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:08:48.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:47 smithi175 ceph-mon[36886]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-06-13T07:08:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:49.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:49 smithi007 ceph-mon[32077]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-06-13T07:08:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:49 smithi189 ceph-mon[36616]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-06-13T07:08:50.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:49 smithi175 ceph-mon[36886]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 446 op/s 2024-06-13T07:08:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:51 smithi189 ceph-mon[36616]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-06-13T07:08:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:51 smithi175 ceph-mon[36886]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-06-13T07:08:52.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:51 smithi007 ceph-mon[32077]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 486 op/s 2024-06-13T07:08:52.429 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:08:52.430 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:08:52.430 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "69bea289-0696-40a9-8206-af591ef8e416", 2024-06-13T07:08:52.430 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:08:52.430 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 -- 172.21.15.7:0/3404095759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec1155e0 msgr2=0x7fbbec1179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/3404095759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec1155e0 0x7fbbec1179d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400b0a0 tx=0x7fbbe400b3b0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 -- 172.21.15.7:0/3404095759 shutdown_connections 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/3404095759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec1155e0 0x7fbbec1179d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/3404095759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109490 0x7fbbec109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.197+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/3404095759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec109db0 0x7fbbec1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.198+0000 7fbbf1c00700 1 -- 172.21.15.7:0/3404095759 >> 172.21.15.7:0/3404095759 conn(0x7fbbec06d600 msgr2=0x7fbbec06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.198+0000 7fbbf1c00700 1 -- 172.21.15.7:0/3404095759 shutdown_connections 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.198+0000 7fbbf1c00700 1 -- 172.21.15.7:0/3404095759 wait complete. 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.199+0000 7fbbf1c00700 1 Processor -- start 2024-06-13T07:08:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.200+0000 7fbbf1c00700 1 -- start start 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.200+0000 7fbbf1c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 0x7fbbec1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.200+0000 7fbbf1c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.200+0000 7fbbf1c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.201+0000 7fbbea000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.201+0000 7fbbea000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40472/0 (socket says 172.21.15.7:40472) 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.201+0000 7fbbeb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.202+0000 7fbbf1c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec11b0c0 con 0x7fbbec109db0 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.202+0000 7fbbf1c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec11af40 con 0x7fbbec1155e0 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.202+0000 7fbbf1c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec11b240 con 0x7fbbec109490 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.202+0000 7fbbeb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42952/0 (socket says 172.21.15.7:42952) 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.202+0000 7fbbeb400700 1 -- 172.21.15.7:0/1015123953 learned_addr learned my addr 172.21.15.7:0/1015123953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.204+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 0x7fbbec1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.204+0000 7fbbeb400700 1 -- 172.21.15.7:0/1015123953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 msgr2=0x7fbbec1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.204+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 0x7fbbec1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.205+0000 7fbbeb400700 1 -- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 msgr2=0x7fbbec1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.205+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.205+0000 7fbbeb400700 1 -- 172.21.15.7:0/1015123953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec1a6320 con 0x7fbbec1155e0 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.205+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 0x7fbbec1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.205+0000 7fbbea000700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.206+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009fd0 tx=0x7fbbe4007770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.207+0000 7fbbe3400700 1 -- 172.21.15.7:0/1015123953 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4021070 con 0x7fbbec1155e0 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.207+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec1a65b0 con 0x7fbbec1155e0 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.207+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec1ade90 con 0x7fbbec1155e0 2024-06-13T07:08:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.207+0000 7fbbe3400700 1 -- 172.21.15.7:0/1015123953 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe4009c80 con 0x7fbbec1155e0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.208+0000 7fbbe3400700 1 -- 172.21.15.7:0/1015123953 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4021070 con 0x7fbbec1155e0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.212+0000 7fbbe3400700 1 -- 172.21.15.7:0/1015123953 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbbe4004560 con 0x7fbbec1155e0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.215+0000 7fbbe3400700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 0x7fbbd00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.216+0000 7fbbe3400700 1 -- 172.21.15.7:0/1015123953 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbbe40d0170 con 0x7fbbec1155e0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.216+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 0x7fbbd00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.222+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 0x7fbbd00a52e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009be0 tx=0x7fbbdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.230+0000 7fbbfc5d0b80 1 Processor -- start 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.231+0000 7fbbfc5d0b80 1 -- start start 2024-06-13T07:08:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.231+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.232+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c924f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.232+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c9250ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.232+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.232+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42966/0 (socket says 172.21.15.7:42966) 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.232+0000 7fbbeaa00700 1 -- 172.21.15.7:0/458329288 learned_addr learned my addr 172.21.15.7:0/458329288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c92408a90 con 0x559c924f9b70 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c9244bb80 con 0x559c92475ec0 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c92406970 con 0x559c924facc0 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbeb400700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c9250ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbea000700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c924f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbeaa00700 1 -- 172.21.15.7:0/458329288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 msgr2=0x559c924f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c924f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.233+0000 7fbbeaa00700 1 -- 172.21.15.7:0/458329288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 msgr2=0x559c9250ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.234+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c9250ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.234+0000 7fbbeaa00700 1 -- 172.21.15.7:0/458329288 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c923f4010 con 0x559c92475ec0 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.234+0000 7fbbeb400700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c9250ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.236+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc010d00 tx=0x7fbbdc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c41758b0a15bee server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.237+0000 7fbbd7e00700 1 -- 172.21.15.7:0/458329288 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc023070 con 0x559c92475ec0 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.237+0000 7fbbd7e00700 1 -- 172.21.15.7:0/458329288 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc01f450 con 0x559c92475ec0 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.237+0000 7fbbd7e00700 1 -- 172.21.15.7:0/458329288 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc01e5f0 con 0x559c92475ec0 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 msgr2=0x559c924f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc010d00 tx=0x7fbbdc013000 comp rx=0 tx=0).stop 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 shutdown_connections 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c9250ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c924f90f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/458329288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c924f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.238+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 >> 172.21.15.7:0/458329288 conn(0x559c924261f0 msgr2=0x559c924f5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.239+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 shutdown_connections 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.239+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/458329288 wait complete. 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.240+0000 7fbbfc5d0b80 1 -- start start 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.240+0000 7fbbfc5d0b80 1 Processor -- start 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.242+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 0x559c9250ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.242+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 0x559c9250bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c9250c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbfc5d0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559c923e7fe0 con 0x559c92475ec0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbea000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 0x559c9250bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbea000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 0x559c9250bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42970/0 (socket says 172.21.15.7:42970) 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbea000700 1 -- 172.21.15.7:0/3861049647 learned_addr learned my addr 172.21.15.7:0/3861049647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 0x559c9250ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.243+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559c92447b40 con 0x559c924f9b70 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.244+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c9250f870 con 0x559c924facc0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.244+0000 7fbbfc5d0b80 10 client.0 _ll_register_callbacks cb 0x559c923e57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.244+0000 7fbbeb400700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c9250c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.245+0000 7fbbeaa00700 1 -- 172.21.15.7:0/3861049647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 msgr2=0x559c9250c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.245+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c9250c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.245+0000 7fbbeaa00700 1 -- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 msgr2=0x559c9250bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.245+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 0x559c9250bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.245+0000 7fbbeaa00700 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9239a950 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.246+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 0x559c9250ec00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc011010 tx=0x7fbbdc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5d279e884af31 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.246+0000 7fbbd4c00700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.247+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc023030 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.247+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbbdc0128b0 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.247+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc01f450 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.248+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc012ab0 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.248+0000 7fbbfc5d0b80 10 client.16491 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.248+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559c92519fe0 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc01f450 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fbbdc00f040 con 0x559c92475ec0 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 20 client.16491 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9251c4b0 0x559c9251e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 4 client.16491 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.251+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c92459080 con 0x559c9251c4b0 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.252+0000 7fbbea000700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9251c4b0 0x559c9251e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.253+0000 7fbbea000700 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9251c4b0 0x559c9251e8a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8653282563de46 server_cookie=d39cc224e2ac773f in_seq=0 out_seq=0 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.253+0000 7fbbd4c00700 10 client.16491 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.253+0000 7fbbd4c00700 1 -- 172.21.15.7:0/3861049647 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c92459080 con 0x559c9251c4b0 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.253+0000 7fbbd4c00700 10 client.16491 handle_command_reply: tid=0 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 client.16491 shutdown 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 msgr2=0x559c9250ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 0x559c9250ec00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc011010 tx=0x7fbbdc0099e0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 shutdown_connections 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c9251c4b0 0x559c9251e8a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924facc0 0x559c9250c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbd4c00700 0 client.16491 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c92475ec0 0x559c9250ec00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/3861049647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924f9b70 0x559c9250bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.254+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 >> 172.21.15.7:0/3861049647 conn(0x559c924261f0 msgr2=0x559c924f47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.258+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 shutdown_connections 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.258+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/3861049647 wait complete. 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.258+0000 7fbbfc5d0b80 20 client.16491 trim_cache size 0 max 16384 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.300+0000 7fbbfc5d0b80 1 Processor -- start 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.300+0000 7fbbfc5d0b80 1 -- start start 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c9250c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 0x559c9250e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 0x559c924f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0112b0 con 0x559c924facc0 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc00eae0 con 0x559c92475ec0 2024-06-13T07:08:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbfc5d0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc00c930 con 0x559c924f9b70 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c9250c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbea000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 0x559c9250e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c9250c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42972/0 (socket says 172.21.15.7:42972) 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.301+0000 7fbbeaa00700 1 -- 172.21.15.7:0/2621638944 learned_addr learned my addr 172.21.15.7:0/2621638944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbeb400700 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 0x559c924f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 -- 172.21.15.7:0/2621638944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 msgr2=0x559c9250c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c9250c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 -- 172.21.15.7:0/2621638944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 msgr2=0x559c924f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 0x559c924f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 -- 172.21.15.7:0/2621638944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9251ee30 con 0x559c924f9b70 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbeb400700 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 0x559c924f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.302+0000 7fbbea000700 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 0x559c9250e740 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x559c924fc290 tx=0x7fbbd8009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473009e5a8f49c75 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.303+0000 7fbbd7400700 1 -- 172.21.15.7:0/2621638944 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800f040 con 0x559c924f9b70 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.304+0000 7fbbd7400700 1 -- 172.21.15.7:0/2621638944 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd801f030 con 0x559c924f9b70 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.304+0000 7fbbd7400700 1 -- 172.21.15.7:0/2621638944 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8023ac0 con 0x559c924f9b70 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.305+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/2621638944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 msgr2=0x559c9250e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.305+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 0x559c9250e740 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x559c924fc290 tx=0x7fbbd8009d20 comp rx=0 tx=0).stop 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.306+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/2621638944 shutdown_connections 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.306+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924facc0 0x559c924f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.306+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c92475ec0 0x559c9250c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.306+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/2621638944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c924f9b70 0x559c9250e740 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.306+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/2621638944 >> 172.21.15.7:0/2621638944 conn(0x559c924261f0 msgr2=0x559c925097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.307+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/2621638944 shutdown_connections 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.307+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/2621638944 wait complete. 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.307+0000 7fbbfc5d0b80 1 -- start start 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.308+0000 7fbbfc5d0b80 1 Processor -- start 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.309+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.309+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c92524360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbfc5d0b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 0x559c925248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbea000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c92524360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbfc5d0b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc00cfa0 con 0x559c924f9b70 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbea000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c92524360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40492/0 (socket says 172.21.15.7:40492) 2024-06-13T07:08:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbeb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 0x559c925248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbea000700 1 -- 172.21.15.7:0/1440809959 learned_addr learned my addr 172.21.15.7:0/1440809959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbeaa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50020/0 (socket says 172.21.15.7:50020) 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc00e410 con 0x559c924facc0 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc00d670 con 0x559c92475ec0 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.310+0000 7fbbfc5d0b80 10 client.0 _ll_register_callbacks cb 0x559c923e57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 -- 172.21.15.7:0/1440809959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 msgr2=0x559c92523e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 -- 172.21.15.7:0/1440809959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 msgr2=0x559c92524360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c92524360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c92509ee0 con 0x559c924facc0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbeb400700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 0x559c925248a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40cd8d0 tx=0x7fbbe409c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae685dcfd22f4e37 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.311+0000 7fbbd4c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40b4020 con 0x559c924facc0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbbe40870b0 con 0x559c924facc0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd801f030 con 0x559c924facc0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbbe40ba050 con 0x559c924facc0 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbfc5d0b80 10 client.27257 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.312+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559c92587300 con 0x559c924facc0 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd801f030 con 0x559c924facc0 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fbbe4011070 con 0x559c924facc0 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 10 client.27257 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 20 client.27257 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c92587960 0x559c92589d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 4 client.27257 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.313+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c92453600 con 0x559c92587960 2024-06-13T07:08:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.314+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c92587960 0x559c92589d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.314+0000 7fbbeaa00700 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c92587960 0x559c92589d80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b36228af15470ed server_cookie=1e400719022438e8 in_seq=0 out_seq=0 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.315+0000 7fbbd4c00700 10 client.27257 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.315+0000 7fbbd4c00700 1 -- 172.21.15.7:0/1440809959 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c92453600 con 0x559c92587960 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbd4c00700 10 client.27257 handle_command_reply: tid=0 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 client.27257 shutdown 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 msgr2=0x559c925248a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 0x559c925248a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40cd8d0 tx=0x7fbbe409c0a0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 shutdown_connections 2024-06-13T07:08:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559c92587960 0x559c92589d80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559c924facc0 0x559c925248a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbd4c00700 0 client.27257 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c92475ec0 0x559c92523e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 --2- 172.21.15.7:0/1440809959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c924f9b70 0x559c92524360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 >> 172.21.15.7:0/1440809959 conn(0x559c924261f0 msgr2=0x559c924fb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.316+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 shutdown_connections 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.317+0000 7fbbfc5d0b80 1 -- 172.21.15.7:0/1440809959 wait complete. 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.317+0000 7fbbfc5d0b80 20 client.27257 trim_cache size 0 max 16384 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.318+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 msgr2=0x7fbbd00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.318+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 0x7fbbd00a52e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009be0 tx=0x7fbbdc009260 comp rx=0 tx=0).stop 2024-06-13T07:08:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.318+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 msgr2=0x7fbbec1a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.318+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009fd0 tx=0x7fbbe4007770 comp rx=0 tx=0).stop 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 shutdown_connections 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbbd00a2e20 0x7fbbd00a52e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbec1155e0 0x7fbbec1a5be0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbec109490 0x7fbbec1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 --2- 172.21.15.7:0/1015123953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbec109db0 0x7fbbec1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 >> 172.21.15.7:0/1015123953 conn(0x7fbbec06d600 msgr2=0x7fbbec071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 shutdown_connections 2024-06-13T07:08:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:52.319+0000 7fbbf1c00700 1 -- 172.21.15.7:0/1015123953 wait complete. 2024-06-13T07:08:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 770.0 s, 447.4 tps, lat 225.100 ms stddev 336.018 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 -- 172.21.15.7:0/990067733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658101a10 msgr2=0x7fb658101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 --2- 172.21.15.7:0/990067733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658101a10 0x7fb658101e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb64802a400 comp rx=0 tx=0).stop 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 -- 172.21.15.7:0/990067733 shutdown_connections 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 --2- 172.21.15.7:0/990067733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb658109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 --2- 172.21.15.7:0/990067733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658102c00 0x7fb658103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 --2- 172.21.15.7:0/990067733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658101a10 0x7fb658101e30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 -- 172.21.15.7:0/990067733 >> 172.21.15.7:0/990067733 conn(0x7fb6580fcf60 msgr2=0x7fb6580ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 -- 172.21.15.7:0/990067733 shutdown_connections 2024-06-13T07:08:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.053+0000 7fb65e600640 1 -- 172.21.15.7:0/990067733 wait complete. 2024-06-13T07:08:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.054+0000 7fb65e600640 1 Processor -- start 2024-06-13T07:08:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 -- start start 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 0x7fb658107740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658102c00 0x7fb658107c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810d6e0 con 0x7fb658101a10 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810d560 con 0x7fb6581035c0 2024-06-13T07:08:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.055+0000 7fb65e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb65810d860 con 0x7fb658102c00 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42998/0 (socket says 172.21.15.7:42998) 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 -- 172.21.15.7:0/1739239759 learned_addr learned my addr 172.21.15.7:0/1739239759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 -- 172.21.15.7:0/1739239759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658102c00 msgr2=0x7fb658107c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658102c00 0x7fb658107c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.056+0000 7fb657e00640 1 -- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 msgr2=0x7fb658107740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.057+0000 7fb657400640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 0x7fb658107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.057+0000 7fb657e00640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 0x7fb658107740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.057+0000 7fb657e00640 1 -- 172.21.15.7:0/1739239759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb658104b70 con 0x7fb6581035c0 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.057+0000 7fb657400640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 0x7fb658107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.057+0000 7fb657e00640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb65000eb10 tx=0x7fb65000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.058+0000 7fb64fe00640 1 -- 172.21.15.7:0/1739239759 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650004b20 con 0x7fb6581035c0 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.058+0000 7fb64fe00640 1 -- 172.21.15.7:0/1739239759 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650004cc0 con 0x7fb6581035c0 2024-06-13T07:08:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.058+0000 7fb64fe00640 1 -- 172.21.15.7:0/1739239759 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650010760 con 0x7fb6581035c0 2024-06-13T07:08:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.058+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581a6350 con 0x7fb6581035c0 2024-06-13T07:08:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.058+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a6700 con 0x7fb6581035c0 2024-06-13T07:08:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.061+0000 7fb64fe00640 1 -- 172.21.15.7:0/1739239759 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb650005020 con 0x7fb6581035c0 2024-06-13T07:08:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.062+0000 7fb66140e740 1 Processor -- start 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 -- start start 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076890d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb64fe00640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 0x7fb63c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890d5f0 0x5607689120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560768870420 con 0x56076890cc00 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560768817c50 con 0x56076890d5f0 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb66140e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56076886c3e0 con 0x560768912760 2024-06-13T07:08:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb64fe00640 1 -- 172.21.15.7:0/1739239759 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb6500cfb80 con 0x7fb6581035c0 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb656a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076890d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.063+0000 7fb657400640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 0x7fb63c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb656a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076890d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40522/0 (socket says 172.21.15.7:40522) 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb656a00640 1 -- 172.21.15.7:0/2498235802 learned_addr learned my addr 172.21.15.7:0/2498235802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50042/0 (socket says 172.21.15.7:50042) 2024-06-13T07:08:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb656a00640 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890d5f0 0x5607689120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 -- 172.21.15.7:0/2498235802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890d5f0 msgr2=0x5607689120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890d5f0 0x5607689120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 -- 172.21.15.7:0/2498235802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 msgr2=0x56076890d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076890d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 -- 172.21.15.7:0/2498235802 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560768584210 con 0x560768912760 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.064+0000 7fb657e00640 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb65009bfd0 tx=0x7fb65009bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d64a724a0a1b681 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.065+0000 7fb64cc00640 1 -- 172.21.15.7:0/2498235802 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65009c040 con 0x560768912760 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.065+0000 7fb64cc00640 1 -- 172.21.15.7:0/2498235802 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6500a03f0 con 0x560768912760 2024-06-13T07:08:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.065+0000 7fb64cc00640 1 -- 172.21.15.7:0/2498235802 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500986b0 con 0x560768912760 2024-06-13T07:08:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.066+0000 7fb66140e740 1 -- 172.21.15.7:0/2498235802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 msgr2=0x560768914bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.066+0000 7fb66140e740 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb65009bfd0 tx=0x7fb65009bf00 comp rx=0 tx=0).stop 2024-06-13T07:08:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.066+0000 7fb657400640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 0x7fb63c0a51f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb64802a920 tx=0x7fb648000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.067+0000 7fb66140e740 1 -- 172.21.15.7:0/2498235802 shutdown_connections 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.067+0000 7fb66140e740 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560768912760 0x560768914bf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.067+0000 7fb66140e740 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890d5f0 0x5607689120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.067+0000 7fb66140e740 1 --2- 172.21.15.7:0/2498235802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076890d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.067+0000 7fb66140e740 1 -- 172.21.15.7:0/2498235802 >> 172.21.15.7:0/2498235802 conn(0x5607688d4fb0 msgr2=0x560768908680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.068+0000 7fb66140e740 1 -- 172.21.15.7:0/2498235802 shutdown_connections 2024-06-13T07:08:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.068+0000 7fb66140e740 1 -- 172.21.15.7:0/2498235802 wait complete. 2024-06-13T07:08:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.069+0000 7fb66140e740 1 -- start start 2024-06-13T07:08:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.069+0000 7fb66140e740 1 Processor -- start 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 0x560768916f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x5607689174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560768915710 con 0x56076890d5f0 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56076882b0b0 con 0x560768912760 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560768915890 con 0x56076890cc00 2024-06-13T07:08:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb66140e740 10 client.0 _ll_register_callbacks cb 0x560768816f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb657e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb657e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43026/0 (socket says 172.21.15.7:43026) 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb657e00640 1 -- 172.21.15.7:0/3389086961 learned_addr learned my addr 172.21.15.7:0/3389086961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.070+0000 7fb657400640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 0x560768916f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb656a00640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x5607689174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 -- 172.21.15.7:0/3389086961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 msgr2=0x560768916f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 0x560768916f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 -- 172.21.15.7:0/3389086961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 msgr2=0x5607689174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x5607689174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 -- 172.21.15.7:0/3389086961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560768650620 con 0x560768912760 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657400640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 0x560768916f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb657e00640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb650000f80 tx=0x7fb650098960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d7acb759bb929d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb641600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.071+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650096db0 con 0x560768912760 2024-06-13T07:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb650005310 con 0x560768912760 2024-06-13T07:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6500a03f0 con 0x560768912760 2024-06-13T07:08:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb6500b5020 con 0x560768912760 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb66140e740 10 client.27269 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5607689217f0 con 0x560768912760 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.072+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6500a03f0 con 0x560768912760 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fb65009b490 con 0x560768912760 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 10 client.27269 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 10 client.27269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 20 client.27269 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768922b40 0x560768924f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 4 client.27269 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56076887db50 con 0x560768922b40 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.073+0000 7fb657400640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768922b40 0x560768924f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.074+0000 7fb641600640 10 client.27269 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.074+0000 7fb657400640 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768922b40 0x560768924f30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a61787000d08a46 server_cookie=2321145314f70053 in_seq=0 out_seq=0 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb641600640 1 -- 172.21.15.7:0/3389086961 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56076887db50 con 0x560768922b40 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb641600640 10 client.27269 handle_command_reply: tid=0 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 client.27269 shutdown 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 msgr2=0x56076891e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb650000f80 tx=0x7fb650098960 comp rx=0 tx=0).stop 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 shutdown_connections 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768922b40 0x560768924f30 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560768912760 0x56076891e480 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb641600640 0 client.27269 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x5607689174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 --2- 172.21.15.7:0/3389086961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890cc00 0x560768916f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.075+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 >> 172.21.15.7:0/3389086961 conn(0x5607688d4fb0 msgr2=0x560768907840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.076+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 shutdown_connections 2024-06-13T07:08:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.076+0000 7fb66140e740 1 -- 172.21.15.7:0/3389086961 wait complete. 2024-06-13T07:08:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.076+0000 7fb66140e740 20 client.27269 trim_cache size 0 max 16384 2024-06-13T07:08:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-06-13T07:08:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:08:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:53 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 Processor -- start 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 -- start start 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076891e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b3450 con 0x56076890cc00 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b0cc0 con 0x5607688d4fb0 2024-06-13T07:08:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.101+0000 7fb66140e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aeb10 con 0x56076890d5f0 2024-06-13T07:08:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb656a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076891e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43038/0 (socket says 172.21.15.7:43038) 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 -- 172.21.15.7:0/2198391587 learned_addr learned my addr 172.21.15.7:0/2198391587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50056/0 (socket says 172.21.15.7:50056) 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 -- 172.21.15.7:0/2198391587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 msgr2=0x5607689296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 -- 172.21.15.7:0/2198391587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 msgr2=0x56076891e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076891e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 -- 172.21.15.7:0/2198391587 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076890de80 con 0x5607688d4fb0 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657e00640 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.102+0000 7fb657400640 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb658102a60 tx=0x7fb64803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33b0c010638c053 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.103+0000 7fb643e00640 1 -- 172.21.15.7:0/2198391587 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648055070 con 0x5607688d4fb0 2024-06-13T07:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.103+0000 7fb643e00640 1 -- 172.21.15.7:0/2198391587 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb648041070 con 0x5607688d4fb0 2024-06-13T07:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.103+0000 7fb643e00640 1 -- 172.21.15.7:0/2198391587 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648035050 con 0x5607688d4fb0 2024-06-13T07:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.103+0000 7fb66140e740 1 -- 172.21.15.7:0/2198391587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 msgr2=0x56076890bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb658102a60 tx=0x7fb64803b000 comp rx=0 tx=0).stop 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 -- 172.21.15.7:0/2198391587 shutdown_connections 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076890d5f0 0x5607689296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890cc00 0x56076891e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 --2- 172.21.15.7:0/2198391587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5607688d4fb0 0x56076890bc80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 -- 172.21.15.7:0/2198391587 >> 172.21.15.7:0/2198391587 conn(0x5607689252a0 msgr2=0x56076890b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 -- 172.21.15.7:0/2198391587 shutdown_connections 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 -- 172.21.15.7:0/2198391587 wait complete. 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.104+0000 7fb66140e740 1 -- start start 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 Processor -- start 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607688d4fb0 0x56076891bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890cc00 0x56076891c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581af180 con 0x56076890d5f0 2024-06-13T07:08:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b05f0 con 0x56076890cc00 2024-06-13T07:08:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581af850 con 0x5607688d4fb0 2024-06-13T07:08:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.105+0000 7fb66140e740 10 client.0 _ll_register_callbacks cb 0x7fb6500a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.106+0000 7fb657e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.106+0000 7fb657e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40544/0 (socket says 172.21.15.7:40544) 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.106+0000 7fb657e00640 1 -- 172.21.15.7:0/1873364854 learned_addr learned my addr 172.21.15.7:0/1873364854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.107+0000 7fb657e00640 1 -- 172.21.15.7:0/1873364854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607688d4fb0 msgr2=0x56076891bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.107+0000 7fb657e00640 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607688d4fb0 0x56076891bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.107+0000 7fb657e00640 1 -- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890cc00 msgr2=0x56076891c140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.107+0000 7fb657e00640 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890cc00 0x56076891c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.107+0000 7fb657e00640 1 -- 172.21.15.7:0/1873364854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076891d6f0 con 0x56076890d5f0 2024-06-13T07:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.108+0000 7fb641600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.108+0000 7fb657e00640 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6500984a0 tx=0x7fb65009b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350601b9669f25cf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.108+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65009be10 con 0x56076890d5f0 2024-06-13T07:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.109+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb650096dc0 con 0x56076890d5f0 2024-06-13T07:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.109+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb648041070 con 0x56076890d5f0 2024-06-13T07:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.109+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb65009be10 con 0x56076890d5f0 2024-06-13T07:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.110+0000 7fb66140e740 10 client.16509 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.110+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56076898d770 con 0x56076890d5f0 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.110+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650082020 con 0x56076890d5f0 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fb6500a1580 con 0x56076890d5f0 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 10 client.16509 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 20 client.16509 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768912760 0x560768990000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 4 client.16509 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb648041070 con 0x560768912760 2024-06-13T07:08:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.111+0000 7fb657400640 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768912760 0x560768990000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.112+0000 7fb657400640 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768912760 0x560768990000 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582105b15f76dace server_cookie=6349e32b029c5985 in_seq=0 out_seq=0 2024-06-13T07:08:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.112+0000 7fb641600640 10 client.16509 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-06-13T07:08:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:53 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb641600640 1 -- 172.21.15.7:0/1873364854 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb648041070 con 0x560768912760 2024-06-13T07:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb641600640 10 client.16509 handle_command_reply: tid=0 2024-06-13T07:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 client.16509 shutdown 2024-06-13T07:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 msgr2=0x56076891d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6500984a0 tx=0x7fb65009b6a0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 shutdown_connections 2024-06-13T07:08:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x560768912760 0x560768990000 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56076890d5f0 0x56076891d100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56076890cc00 0x56076891c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 --2- 172.21.15.7:0/1873364854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607688d4fb0 0x56076891bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 >> 172.21.15.7:0/1873364854 conn(0x5607689252a0 msgr2=0x56076890b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.122+0000 7fb641600640 0 client.16509 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.123+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 shutdown_connections 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.123+0000 7fb66140e740 1 -- 172.21.15.7:0/1873364854 wait complete. 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.123+0000 7fb66140e740 20 client.16509 trim_cache size 0 max 16384 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 msgr2=0x7fb63c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 0x7fb63c0a51f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb64802a920 tx=0x7fb648000bc0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 msgr2=0x7fb6581042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb65000eb10 tx=0x7fb65000c2d0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 shutdown_connections 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb63c0a2d30 0x7fb63c0a51f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6581035c0 0x7fb6581042b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb658102c00 0x7fb658107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 --2- 172.21.15.7:0/1739239759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb658101a10 0x7fb658107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 >> 172.21.15.7:0/1739239759 conn(0x7fb6580fcf60 msgr2=0x7fb658073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 shutdown_connections 2024-06-13T07:08:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:08:54.124+0000 7fb65e600640 1 -- 172.21.15.7:0/1739239759 wait complete. 2024-06-13T07:08:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 441 op/s 2024-06-13T07:08:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:08:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:53 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:54.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:08:55.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:54 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:08:55.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:54 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:55.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:54 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:08:55.988 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:55 smithi007 ceph-mon[32077]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:08:56.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:55 smithi175 ceph-mon[36886]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:08:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:55 smithi189 ceph-mon[36616]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-06-13T07:08:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 775.0 s, 425.0 tps, lat 238.846 ms stddev 286.700 2024-06-13T07:08:57.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:57 smithi007 ceph-mon[32077]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-06-13T07:08:58.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:57 smithi189 ceph-mon[36616]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-06-13T07:08:58.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:57 smithi175 ceph-mon[36886]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-06-13T07:08:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 -- 172.21.15.7:0/378399303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e50110b80 msgr2=0x7f7e50109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/378399303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e50110b80 0x7f7e50109490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800b0a0 tx=0x7f7e4800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 -- 172.21.15.7:0/378399303 shutdown_connections 2024-06-13T07:08:59.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/378399303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e5011bc70 0x7f7e5011e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/378399303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e50110b80 0x7f7e50109490 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/378399303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110ff0 0x7f7e501099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.063+0000 7f7e4fe00700 1 -- 172.21.15.7:0/378399303 >> 172.21.15.7:0/378399303 conn(0x7f7e5006d470 msgr2=0x7f7e5006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.064+0000 7f7e4fe00700 1 -- 172.21.15.7:0/378399303 shutdown_connections 2024-06-13T07:08:59.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.064+0000 7f7e4fe00700 1 -- 172.21.15.7:0/378399303 wait complete. 2024-06-13T07:08:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.068+0000 7f7e4fe00700 1 Processor -- start 2024-06-13T07:08:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.069+0000 7f7e4fe00700 1 -- start start 2024-06-13T07:08:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.069+0000 7f7e4fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110b80 0x7f7e501ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.069+0000 7f7e4fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 0x7f7e501adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.069+0000 7f7e4fe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.070+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 0x7f7e501adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.070+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 0x7f7e501adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50080/0 (socket says 172.21.15.7:50080) 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.071+0000 7f7e4fe00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501216a0 con 0x7f7e50110b80 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.071+0000 7f7e4fe00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50121520 con 0x7f7e5011bc70 2024-06-13T07:08:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.071+0000 7f7e4fe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50121820 con 0x7f7e50110ff0 2024-06-13T07:08:59.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.071+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110b80 0x7f7e501ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.073+0000 7f7e4f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.073+0000 7f7e4f400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43062/0 (socket says 172.21.15.7:43062) 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.073+0000 7f7e4f400700 1 -- 172.21.15.7:0/4187662832 learned_addr learned my addr 172.21.15.7:0/4187662832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 -- 172.21.15.7:0/4187662832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 msgr2=0x7f7e501adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 0x7f7e501adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 -- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110b80 msgr2=0x7f7e501ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110b80 0x7f7e501ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 -- 172.21.15.7:0/4187662832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e501a8000 con 0x7f7e5011bc70 2024-06-13T07:08:59.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.074+0000 7f7e4f400700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b480 tx=0x7f7e4000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.078+0000 7f7e54a00700 1 -- 172.21.15.7:0/4187662832 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40015020 con 0x7f7e5011bc70 2024-06-13T07:08:59.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.078+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e501a82f0 con 0x7f7e5011bc70 2024-06-13T07:08:59.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.078+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501b45b0 con 0x7f7e5011bc70 2024-06-13T07:08:59.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.080+0000 7f7e54a00700 1 -- 172.21.15.7:0/4187662832 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e40011dd0 con 0x7f7e5011bc70 2024-06-13T07:08:59.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.080+0000 7f7e54a00700 1 -- 172.21.15.7:0/4187662832 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000f2e0 con 0x7f7e5011bc70 2024-06-13T07:08:59.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.089+0000 7f7e606a8b80 1 Processor -- start 2024-06-13T07:08:59.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.091+0000 7f7e54a00700 1 -- 172.21.15.7:0/4187662832 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7e40018020 con 0x7f7e5011bc70 2024-06-13T07:08:59.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.093+0000 7f7e606a8b80 1 -- start start 2024-06-13T07:08:59.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.095+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a56750b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 0x55a567515ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a56741aa10 con 0x55a56750ad60 2024-06-13T07:08:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a56745db00 con 0x55a567516530 2024-06-13T07:08:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5674188f0 con 0x55a56750b680 2024-06-13T07:08:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.098+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 0x55a567515ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.099+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.099+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43074/0 (socket says 172.21.15.7:43074) 2024-06-13T07:08:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.099+0000 7f7e4e000700 1 -- 172.21.15.7:0/3327810582 learned_addr learned my addr 172.21.15.7:0/3327810582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.099+0000 7f7e4e000700 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a56750b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.100+0000 7f7e54a00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 0x7f7e340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.101+0000 7f7e4e000700 1 -- 172.21.15.7:0/3327810582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 msgr2=0x55a567515ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.102+0000 7f7e4e000700 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 0x55a567515ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.102+0000 7f7e54a00700 1 -- 172.21.15.7:0/4187662832 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7e400ceb00 con 0x7f7e5011bc70 2024-06-13T07:08:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.102+0000 7f7e4e000700 1 -- 172.21.15.7:0/3327810582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 msgr2=0x55a56750b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.102+0000 7f7e4e000700 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a56750b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.103+0000 7f7e4e000700 1 -- 172.21.15.7:0/3327810582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a567405f90 con 0x55a567516530 2024-06-13T07:08:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.103+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 0x55a567515ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:08:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.104+0000 7f7e4e000700 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800aed0 tx=0x7f7e3800a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93666b318477b38c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.104+0000 7f7e44c00700 1 -- 172.21.15.7:0/3327810582 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3801c070 con 0x55a567516530 2024-06-13T07:08:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.105+0000 7f7e4f400700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 0x7f7e340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.106+0000 7f7e44c00700 1 -- 172.21.15.7:0/3327810582 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e38017460 con 0x55a567516530 2024-06-13T07:08:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.106+0000 7f7e44c00700 1 -- 172.21.15.7:0/3327810582 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3801c070 con 0x55a567516530 2024-06-13T07:08:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.107+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3327810582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 msgr2=0x55a567518920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.107+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800aed0 tx=0x7f7e3800a360 comp rx=0 tx=0).stop 2024-06-13T07:08:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.107+0000 7f7e4f400700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 0x7f7e340a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e4009afb0 tx=0x7f7e40004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3327810582 shutdown_connections 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a567516530 0x55a567518920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a56750b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3327810582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750b680 0x55a567515ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3327810582 >> 172.21.15.7:0/3327810582 conn(0x55a567438170 msgr2=0x55a567507280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3327810582 shutdown_connections 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.109+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3327810582 wait complete. 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.110+0000 7f7e606a8b80 1 -- start start 2024-06-13T07:08:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.110+0000 7f7e606a8b80 1 Processor -- start 2024-06-13T07:08:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.112+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567515cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.112+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567512fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.113+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a567513510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.113+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a5673f9f60 con 0x55a56750ad60 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.113+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a567459ac0 con 0x55a56750b680 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.113+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a567518fc0 con 0x55a567516530 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.113+0000 7f7e606a8b80 10 client.0 _ll_register_callbacks cb 0x55a5673f3380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.114+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567515cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.114+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567512fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.114+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567512fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43076/0 (socket says 172.21.15.7:43076) 2024-06-13T07:08:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.114+0000 7f7e4ea00700 1 -- 172.21.15.7:0/1561035478 learned_addr learned my addr 172.21.15.7:0/1561035478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.115+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a567513510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.116+0000 7f7e4e000700 1 -- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 msgr2=0x55a567512fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.116+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567512fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.116+0000 7f7e4e000700 1 -- 172.21.15.7:0/1561035478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 msgr2=0x55a567515cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.116+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567515cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.116+0000 7f7e4e000700 1 -- 172.21.15.7:0/1561035478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5673ac8d0 con 0x55a567516530 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.117+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a567513510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800aea0 tx=0x7f7e3800a9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=288c59d48a8a61d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.117+0000 7f7e3d600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.117+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3801c070 con 0x55a567516530 2024-06-13T07:08:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.117+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7e38017820 con 0x55a567516530 2024-06-13T07:08:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.118+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e38017460 con 0x55a567516530 2024-06-13T07:08:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.120+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e38017a20 con 0x55a567516530 2024-06-13T07:08:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.120+0000 7f7e606a8b80 10 client.27175 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.120+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a567523760 con 0x55a567516530 2024-06-13T07:08:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.120+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e38017460 con 0x55a567516530 2024-06-13T07:08:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f7e38012050 con 0x55a567516530 2024-06-13T07:08:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 10 client.27175 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 20 client.27175 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a567525c30 0x55a567528020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 4 client.27175 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:08:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.122+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a56746b000 con 0x55a567525c30 2024-06-13T07:08:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.123+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a567525c30 0x55a567528020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.125+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a567525c30 0x55a567528020 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f2c9d15377a806 server_cookie=ed087a9fdb385fd in_seq=0 out_seq=0 2024-06-13T07:08:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.125+0000 7f7e3d600700 10 client.27175 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.126+0000 7f7e3d600700 1 -- 172.21.15.7:0/1561035478 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a56746b000 con 0x55a567525c30 2024-06-13T07:08:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.126+0000 7f7e3d600700 10 client.27175 handle_command_reply: tid=0 2024-06-13T07:08:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.126+0000 7f7e606a8b80 1 client.27175 shutdown 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 msgr2=0x55a567513510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a567513510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800aea0 tx=0x7f7e3800a9d0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 shutdown_connections 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a567525c30 0x55a567528020 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a567513510 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567515cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1561035478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567512fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 >> 172.21.15.7:0/1561035478 conn(0x55a567438170 msgr2=0x55a567506e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.127+0000 7f7e3d600700 0 client.27175 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.128+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 shutdown_connections 2024-06-13T07:08:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.128+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1561035478 wait complete. 2024-06-13T07:08:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.128+0000 7f7e606a8b80 20 client.27175 trim_cache size 0 max 16384 2024-06-13T07:08:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 Processor -- start 2024-06-13T07:08:59.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 -- start start 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567513560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a56752ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c011340 con 0x55a56750ad60 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c00eb70 con 0x55a56750b680 2024-06-13T07:08:59.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.176+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c00c9c0 con 0x55a567516530 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.177+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.177+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43084/0 (socket says 172.21.15.7:43084) 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.177+0000 7f7e4ea00700 1 -- 172.21.15.7:0/3771012389 learned_addr learned my addr 172.21.15.7:0/3771012389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.177+0000 7f7e4e000700 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a56752ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 -- 172.21.15.7:0/3771012389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 msgr2=0x55a56752ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a56752ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 -- 172.21.15.7:0/3771012389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 msgr2=0x55a567513560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567513560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 -- 172.21.15.7:0/3771012389 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a567516040 con 0x55a56750b680 2024-06-13T07:08:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.178+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e48000f80 tx=0x7f7e4800b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1942cfaa8197ee server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.180+0000 7f7e3fe00700 1 -- 172.21.15.7:0/3771012389 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48004a50 con 0x55a56750b680 2024-06-13T07:08:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.180+0000 7f7e3fe00700 1 -- 172.21.15.7:0/3771012389 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e48004bf0 con 0x55a56750b680 2024-06-13T07:08:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.180+0000 7f7e3fe00700 1 -- 172.21.15.7:0/3771012389 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480074e0 con 0x55a56750b680 2024-06-13T07:08:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3771012389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 msgr2=0x55a567511f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e48000f80 tx=0x7f7e4800b7f0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3771012389 shutdown_connections 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a567516530 0x55a56752ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a56750ad60 0x55a567513560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/3771012389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a567511f50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.181+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3771012389 >> 172.21.15.7:0/3771012389 conn(0x55a567438170 msgr2=0x55a56750e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.182+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3771012389 shutdown_connections 2024-06-13T07:08:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.182+0000 7f7e606a8b80 1 -- 172.21.15.7:0/3771012389 wait complete. 2024-06-13T07:08:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.183+0000 7f7e606a8b80 1 -- start start 2024-06-13T07:08:59.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.183+0000 7f7e606a8b80 1 Processor -- start 2024-06-13T07:08:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.184+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 0x55a5675291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a5675296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e606a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567516530 0x55a56758f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e606a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c00d030 con 0x55a567516530 2024-06-13T07:08:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a5675296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e4ea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a5675296e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43086/0 (socket says 172.21.15.7:43086) 2024-06-13T07:08:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.185+0000 7f7e4ea00700 1 -- 172.21.15.7:0/1938253717 learned_addr learned my addr 172.21.15.7:0/1938253717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:08:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.186+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 0x55a5675291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.186+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c00e4a0 con 0x55a56750b680 2024-06-13T07:08:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.186+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c00d700 con 0x55a56750ad60 2024-06-13T07:08:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.186+0000 7f7e606a8b80 10 client.0 _ll_register_callbacks cb 0x55a5673f3380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:08:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.188+0000 7f7e4e000700 1 -- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 msgr2=0x55a5675296e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.188+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a5675296e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.188+0000 7f7e4e000700 1 -- 172.21.15.7:0/1938253717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567516530 msgr2=0x55a56758f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:08:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.188+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567516530 0x55a56758f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.188+0000 7f7e4e000700 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a56758fb00 con 0x55a56750ad60 2024-06-13T07:08:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.189+0000 7f7e3d600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:08:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.189+0000 7f7e4e000700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 0x55a5675291a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e38016c20 tx=0x7f7e38012fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb6d8b21bbdf6b46 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:08:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.190+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3801c040 con 0x55a56750ad60 2024-06-13T07:08:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.190+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7e3800f040 con 0x55a56750ad60 2024-06-13T07:08:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.191+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e48004bf0 con 0x55a56750ad60 2024-06-13T07:08:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.191+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7e38012370 con 0x55a56750ad60 2024-06-13T07:08:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.191+0000 7f7e606a8b80 10 client.27181 fetch_fsmap learned FSMap version 40 2024-06-13T07:08:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.191+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a567590e10 con 0x55a56750ad60 2024-06-13T07:08:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.194+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e48004bf0 con 0x55a56750ad60 2024-06-13T07:08:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.194+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f7e38012050 con 0x55a56750ad60 2024-06-13T07:08:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.195+0000 7f7e606a8b80 10 client.27181 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:08:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.195+0000 7f7e606a8b80 10 client.27181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:08:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.195+0000 7f7e606a8b80 20 client.27181 populate_metadata read hostname 'smithi007' 2024-06-13T07:08:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.195+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a5675912d0 0x55a5675936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:08:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.197+0000 7f7e606a8b80 4 client.27181 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:08:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.197+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a567465580 con 0x55a5675912d0 2024-06-13T07:08:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.197+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a5675912d0 0x55a5675936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:08:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.198+0000 7f7e4ea00700 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a5675912d0 0x55a5675936c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad92c154455c9d83 server_cookie=1ffcb5688fe18dd1 in_seq=0 out_seq=0 2024-06-13T07:08:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.198+0000 7f7e3d600700 10 client.27181 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.198+0000 7f7e3d600700 1 -- 172.21.15.7:0/1938253717 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a567465580 con 0x55a5675912d0 2024-06-13T07:08:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.198+0000 7f7e3d600700 10 client.27181 handle_command_reply: tid=0 2024-06-13T07:08:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 client.27181 shutdown 2024-06-13T07:08:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 msgr2=0x55a5675291a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 0x55a5675291a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e38016c20 tx=0x7f7e38012fd0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 shutdown_connections 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a5675912d0 0x55a5675936c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a567516530 0x55a56758f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e3d600700 0 client.27181 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a56750ad60 0x55a5675291a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 --2- 172.21.15.7:0/1938253717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56750b680 0x55a5675296e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.199+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 >> 172.21.15.7:0/1938253717 conn(0x55a567438170 msgr2=0x55a567514800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.200+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 shutdown_connections 2024-06-13T07:08:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.200+0000 7f7e606a8b80 1 -- 172.21.15.7:0/1938253717 wait complete. 2024-06-13T07:08:59.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.200+0000 7f7e606a8b80 20 client.27181 trim_cache size 0 max 16384 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "69bea289-0696-40a9-8206-af591ef8e416": { 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "69bea289-0696-40a9-8206-af591ef8e416", 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:08:59.203 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 msgr2=0x7f7e340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 0x7f7e340a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7e4009afb0 tx=0x7f7e40004750 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 msgr2=0x7f7e501a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b480 tx=0x7f7e4000b840 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 shutdown_connections 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7e340a2cf0 0x7f7e340a51b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5011bc70 0x7f7e501a78a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e50110b80 0x7f7e501ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 --2- 172.21.15.7:0/4187662832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e50110ff0 0x7f7e501adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 >> 172.21.15.7:0/4187662832 conn(0x7f7e5006d470 msgr2=0x7f7e5010e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:08:59.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.202+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 shutdown_connections 2024-06-13T07:08:59.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:08:59.203+0000 7f7e4fe00700 1 -- 172.21.15.7:0/4187662832 wait complete. 2024-06-13T07:09:00.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:08:59 smithi189 ceph-mon[36616]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 525 op/s 2024-06-13T07:09:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:08:59 smithi175 ceph-mon[36886]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 525 op/s 2024-06-13T07:09:00.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:08:59 smithi007 ceph-mon[32077]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 525 op/s 2024-06-13T07:09:00.857 INFO:tasks.cephfs.filesystem:scrub status for tag:69bea289-0696-40a9-8206-af591ef8e416 - {'path': '/', 'tag': '69bea289-0696-40a9-8206-af591ef8e416', 'options': 'recursive,force'} 2024-06-13T07:09:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:01 smithi189 ceph-mon[36616]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.8 MiB/s wr, 584 op/s 2024-06-13T07:09:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:01 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:01 smithi175 ceph-mon[36886]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.8 MiB/s wr, 584 op/s 2024-06-13T07:09:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:01 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:02.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:01 smithi007 ceph-mon[32077]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.8 MiB/s wr, 584 op/s 2024-06-13T07:09:02.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:01 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:09:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 780.0 s, 571.4 tps, lat 165.730 ms stddev 183.254 2024-06-13T07:09:03.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:02 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:02 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:03.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:02 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.2 MiB/s wr, 538 op/s 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:04.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:09:04.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.2 MiB/s wr, 538 op/s 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:04.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.2 MiB/s wr, 538 op/s 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:09:04.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:09:05.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:05 smithi007 ceph-mon[32077]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 4.4 MiB/s wr, 577 op/s 2024-06-13T07:09:06.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:05 smithi189 ceph-mon[36616]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 4.4 MiB/s wr, 577 op/s 2024-06-13T07:09:06.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:05 smithi175 ceph-mon[36886]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 4.4 MiB/s wr, 577 op/s 2024-06-13T07:09:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.338+0000 7f5b07200640 1 -- 172.21.15.7:0/1579942274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 msgr2=0x7f5b00107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.338+0000 7f5b07200640 1 --2- 172.21.15.7:0/1579942274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00107020 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009900 tx=0x7f5af002a410 comp rx=0 tx=0).stop 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 -- 172.21.15.7:0/1579942274 shutdown_connections 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 --2- 172.21.15.7:0/1579942274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b00109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 --2- 172.21.15.7:0/1579942274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00107020 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 --2- 172.21.15.7:0/1579942274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 0x7f5b001046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 -- 172.21.15.7:0/1579942274 >> 172.21.15.7:0/1579942274 conn(0x7f5b000fbc90 msgr2=0x7f5b000fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 -- 172.21.15.7:0/1579942274 shutdown_connections 2024-06-13T07:09:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.340+0000 7f5b07200640 1 -- 172.21.15.7:0/1579942274 wait complete. 2024-06-13T07:09:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.341+0000 7f5b07200640 1 Processor -- start 2024-06-13T07:09:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 -- start start 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 0x7f5b00195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010d590 con 0x7f5b00102300 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010d410 con 0x7f5b00107560 2024-06-13T07:09:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.345+0000 7f5b07200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0010d710 con 0x7f5b00104c30 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b05400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33738/0 (socket says 172.21.15.7:33738) 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 -- 172.21.15.7:0/562658208 learned_addr learned my addr 172.21.15.7:0/562658208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b05e00640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 0x7f5b00195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 -- 172.21.15.7:0/562658208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 msgr2=0x7f5b00196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 -- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 msgr2=0x7f5b00195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 0x7f5b00195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.346+0000 7f5b06800640 1 -- 172.21.15.7:0/562658208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b00197180 con 0x7f5b00107560 2024-06-13T07:09:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.347+0000 7f5b06800640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b9c0 tx=0x7f5afc00be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.347+0000 7f5afaa00640 1 -- 172.21.15.7:0/562658208 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc019070 con 0x7f5b00107560 2024-06-13T07:09:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.347+0000 7f5afaa00640 1 -- 172.21.15.7:0/562658208 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc00cba0 con 0x7f5b00107560 2024-06-13T07:09:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.347+0000 7f5afaa00640 1 -- 172.21.15.7:0/562658208 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc007650 con 0x7f5b00107560 2024-06-13T07:09:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.347+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b00197470 con 0x7f5b00107560 2024-06-13T07:09:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.348+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b001979f0 con 0x7f5b00107560 2024-06-13T07:09:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.354+0000 7f5afaa00640 1 -- 172.21.15.7:0/562658208 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5afc01c070 con 0x7f5b00107560 2024-06-13T07:09:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.355+0000 7f5b0b5fd740 1 Processor -- start 2024-06-13T07:09:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.355+0000 7f5b0b5fd740 1 -- start start 2024-06-13T07:09:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.355+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x5590676011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.356+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x5590676062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.356+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.356+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559067564420 con 0x559067600dc0 2024-06-13T07:09:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.356+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55906750bc50 con 0x559067606920 2024-06-13T07:09:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.356+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590675603e0 con 0x5590676017b0 2024-06-13T07:09:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.357+0000 7f5b05400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.357+0000 7f5b05400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33754/0 (socket says 172.21.15.7:33754) 2024-06-13T07:09:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.357+0000 7f5b05400640 1 -- 172.21.15.7:0/4173463819 learned_addr learned my addr 172.21.15.7:0/4173463819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.357+0000 7f5b05400640 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x5590676011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.357+0000 7f5b05e00640 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x5590676062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 -- 172.21.15.7:0/4173463819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 msgr2=0x5590676062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x5590676062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 -- 172.21.15.7:0/4173463819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 msgr2=0x5590676011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x5590676011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 -- 172.21.15.7:0/4173463819 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559067278210 con 0x559067606920 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5b05400640 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005c20 tx=0x7f5af0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b5e45b6e717dd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.358+0000 7f5afaa00640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 0x7f5ae80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.359+0000 7f5afaa00640 1 -- 172.21.15.7:0/562658208 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5afc0ce9e0 con 0x7f5b00107560 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.359+0000 7f5aef000640 1 -- 172.21.15.7:0/4173463819 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0035040 con 0x559067606920 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.359+0000 7f5aef000640 1 -- 172.21.15.7:0/4173463819 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af0031a20 con 0x559067606920 2024-06-13T07:09:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.359+0000 7f5aef000640 1 -- 172.21.15.7:0/4173463819 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af003c670 con 0x559067606920 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.360+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/4173463819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 msgr2=0x559067608db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.360+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005c20 tx=0x7f5af0031400 comp rx=0 tx=0).stop 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b05e00640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 0x7f5ae80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/4173463819 shutdown_connections 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067608db0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x5590676062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/4173463819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x5590676011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/4173463819 >> 172.21.15.7:0/4173463819 conn(0x5590675c8fb0 msgr2=0x5590675fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.361+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/4173463819 shutdown_connections 2024-06-13T07:09:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.362+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/4173463819 wait complete. 2024-06-13T07:09:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.362+0000 7f5b0b5fd740 1 -- start start 2024-06-13T07:09:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.364+0000 7f5b0b5fd740 1 Processor -- start 2024-06-13T07:09:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.365+0000 7f5b05e00640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 0x7f5ae80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5590675fd6f0 tx=0x7f5af4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 0x55906760b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x5590676126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559067609870 con 0x5590676017b0 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55906751f0b0 con 0x559067606920 2024-06-13T07:09:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590676099f0 con 0x559067600dc0 2024-06-13T07:09:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.366+0000 7f5b0b5fd740 10 client.0 _ll_register_callbacks cb 0x55906750af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.368+0000 7f5b05400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.368+0000 7f5b05400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51918/0 (socket says 172.21.15.7:51918) 2024-06-13T07:09:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.368+0000 7f5b05400640 1 -- 172.21.15.7:0/2916267488 learned_addr learned my addr 172.21.15.7:0/2916267488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.368+0000 7f5b05e00640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 0x55906760b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.368+0000 7f5b06800640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x5590676126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 -- 172.21.15.7:0/2916267488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 msgr2=0x55906760b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 0x55906760b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 -- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 msgr2=0x5590676126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x5590676126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 -- 172.21.15.7:0/2916267488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559067344620 con 0x5590676017b0 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b06800640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x5590676126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05e00640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 0x55906760b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5b05400640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001eb0 tx=0x7f5af0003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5808a70be029f08 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.369+0000 7f5ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:09:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.370+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af004c020 con 0x5590676017b0 2024-06-13T07:09:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.370+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5af0004310 con 0x5590676017b0 2024-06-13T07:09:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.370+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af0031a20 con 0x5590676017b0 2024-06-13T07:09:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.372+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5af0035040 con 0x5590676017b0 2024-06-13T07:09:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.373+0000 7f5b0b5fd740 10 client.16521 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.373+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559067615a10 con 0x5590676017b0 2024-06-13T07:09:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af0031a20 con 0x5590676017b0 2024-06-13T07:09:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f5af0047030 con 0x5590676017b0 2024-06-13T07:09:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 10 client.16521 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 20 client.16521 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067616d60 0x559067619150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 4 client.16521 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.375+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559067571b50 con 0x559067616d60 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.377+0000 7f5b05e00640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067616d60 0x559067619150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.377+0000 7f5b05e00640 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067616d60 0x559067619150 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ada099456268a5 server_cookie=bdae96f42bb98fef in_seq=0 out_seq=0 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.377+0000 7f5ae3400640 10 client.16521 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.378+0000 7f5ae3400640 1 -- 172.21.15.7:0/2916267488 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559067571b50 con 0x559067616d60 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.378+0000 7f5ae3400640 10 client.16521 handle_command_reply: tid=0 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 client.16521 shutdown 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 msgr2=0x55906760b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001eb0 tx=0x7f5af0003ad0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 shutdown_connections 2024-06-13T07:09:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067616d60 0x559067619150 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x5590676126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5590676017b0 0x55906760b660 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/2916267488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067600dc0 0x55906760b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5ae3400640 0 client.16521 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 >> 172.21.15.7:0/2916267488 conn(0x5590675c8fb0 msgr2=0x5590675fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 shutdown_connections 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.379+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/2916267488 wait complete. 2024-06-13T07:09:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.380+0000 7f5b0b5fd740 20 client.16521 trim_cache size 0 max 16384 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.410+0000 7f5b0b5fd740 1 Processor -- start 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.410+0000 7f5b0b5fd740 1 -- start start 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x559067602ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x559067603010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001bba50 con 0x559067600dc0 2024-06-13T07:09:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001b92c0 con 0x559067606920 2024-06-13T07:09:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.411+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001b7110 con 0x5590676017b0 2024-06-13T07:09:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b05e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x559067602ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33778/0 (socket says 172.21.15.7:33778) 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 -- 172.21.15.7:0/689529038 learned_addr learned my addr 172.21.15.7:0/689529038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b05400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x559067603010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 -- 172.21.15.7:0/689529038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 msgr2=0x559067603010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x559067603010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 -- 172.21.15.7:0/689529038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 msgr2=0x559067602ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x559067602ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b06800640 1 -- 172.21.15.7:0/689529038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590676025c0 con 0x559067606920 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.412+0000 7f5b05400640 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x559067603010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.413+0000 7f5b06800640 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5afc097360 tx=0x7f5afc09b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5ef2a6499ebcfa server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.413+0000 7f5aee600640 1 -- 172.21.15.7:0/689529038 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0a3b70 con 0x559067606920 2024-06-13T07:09:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.414+0000 7f5aee600640 1 -- 172.21.15.7:0/689529038 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc00cd50 con 0x559067606920 2024-06-13T07:09:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.414+0000 7f5aee600640 1 -- 172.21.15.7:0/689529038 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0b9b40 con 0x559067606920 2024-06-13T07:09:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/689529038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 msgr2=0x559067612660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5afc097360 tx=0x7f5afc09b740 comp rx=0 tx=0).stop 2024-06-13T07:09:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/689529038 shutdown_connections 2024-06-13T07:09:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559067606920 0x559067612660 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590676017b0 0x559067603010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/689529038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x559067602ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.415+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/689529038 >> 172.21.15.7:0/689529038 conn(0x5590675c8fb0 msgr2=0x55906760b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.416+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/689529038 shutdown_connections 2024-06-13T07:09:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.416+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/689529038 wait complete. 2024-06-13T07:09:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.417+0000 7f5b0b5fd740 1 -- start start 2024-06-13T07:09:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.417+0000 7f5b0b5fd740 1 Processor -- start 2024-06-13T07:09:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.418+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x55906760f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001b7780 con 0x559067600dc0 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001b8bf0 con 0x5590676017b0 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001b7e50 con 0x559067606920 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b06800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b05400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b05400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:33786/0 (socket says 172.21.15.7:33786) 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b06800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:41916/0 (socket says 172.21.15.7:41916) 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b05400640 1 -- 172.21.15.7:0/210350591 learned_addr learned my addr 172.21.15.7:0/210350591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.419+0000 7f5b0b5fd740 10 client.0 _ll_register_callbacks cb 0x7f5af0032d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.420+0000 7f5b06800640 1 -- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 msgr2=0x55906760fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.420+0000 7f5b06800640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.421+0000 7f5b06800640 1 -- 172.21.15.7:0/210350591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 msgr2=0x55906760f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.421+0000 7f5b06800640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x55906760f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.421+0000 7f5b05400640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.421+0000 7f5b06800640 1 -- 172.21.15.7:0/210350591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559067611330 con 0x559067606920 2024-06-13T07:09:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.422+0000 7f5b06800640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b990 tx=0x7f5afc0098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fdc806c2bb3683 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.422+0000 7f5ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.422+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0a37e0 con 0x559067606920 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.422+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5afc00cd50 con 0x559067606920 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.422+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5afc0b9b40 con 0x559067606920 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.423+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5afc09a350 con 0x559067606920 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.423+0000 7f5b0b5fd740 10 client.27193 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.423+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559067612690 con 0x559067606920 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.424+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc00cd50 con 0x559067606920 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.424+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f5afc099dc0 con 0x559067606920 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.424+0000 7f5b0b5fd740 10 client.27193 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.425+0000 7f5b0b5fd740 10 client.27193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.425+0000 7f5b0b5fd740 20 client.27193 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.425+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067681ba0 0x559067683fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.425+0000 7f5b0b5fd740 4 client.27193 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.425+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55906756bea0 con 0x559067681ba0 2024-06-13T07:09:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.427+0000 7f5b05e00640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067681ba0 0x559067683fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.427+0000 7f5b05e00640 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067681ba0 0x559067683fc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957999496e99509d server_cookie=ed333aa4c7d4e67e in_seq=0 out_seq=0 2024-06-13T07:09:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.427+0000 7f5ae3400640 10 client.27193 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.732+0000 7f5ae3400640 1 -- 172.21.15.7:0/210350591 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55906756bea0 con 0x559067681ba0 2024-06-13T07:09:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.732+0000 7f5ae3400640 10 client.27193 handle_command_reply: tid=0 2024-06-13T07:09:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.732+0000 7f5b0b5fd740 1 client.27193 shutdown 2024-06-13T07:09:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 msgr2=0x559067610d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b990 tx=0x7f5afc0098d0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 shutdown_connections 2024-06-13T07:09:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x559067681ba0 0x559067683fc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559067606920 0x559067610d40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.734+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5590676017b0 0x55906760fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.734+0000 7f5b0b5fd740 1 --2- 172.21.15.7:0/210350591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559067600dc0 0x55906760f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.734+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 >> 172.21.15.7:0/210350591 conn(0x5590675c8fb0 msgr2=0x55906760b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.733+0000 7f5ae3400640 0 client.27193 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.734+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 shutdown_connections 2024-06-13T07:09:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.736+0000 7f5b0b5fd740 1 -- 172.21.15.7:0/210350591 wait complete. 2024-06-13T07:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.736+0000 7f5b0b5fd740 20 client.27193 trim_cache size 0 max 16384 2024-06-13T07:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:09:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:09:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.737+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 msgr2=0x7f5ae80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 0x7f5ae80a50c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5590675fd6f0 tx=0x7f5af4008040 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 msgr2=0x7f5b001a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00b9c0 tx=0x7f5afc00be80 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 shutdown_connections 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5ae80a2c00 0x7f5ae80a50c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b00107560 0x7f5b001a58c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b00104c30 0x7f5b00196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 --2- 172.21.15.7:0/562658208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b00102300 0x7f5b00195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 >> 172.21.15.7:0/562658208 conn(0x7f5b000fbc90 msgr2=0x7f5b00101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 shutdown_connections 2024-06-13T07:09:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:07.738+0000 7f5b07200640 1 -- 172.21.15.7:0/562658208 wait complete. 2024-06-13T07:09:07.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 785.0 s, 549.6 tps, lat 193.216 ms stddev 269.182 2024-06-13T07:09:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:07 smithi189 ceph-mon[36616]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 524 op/s 2024-06-13T07:09:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:07 smithi175 ceph-mon[36886]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 524 op/s 2024-06-13T07:09:08.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:07 smithi007 ceph-mon[32077]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 524 op/s 2024-06-13T07:09:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:09 smithi189 ceph-mon[36616]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-06-13T07:09:10.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:09 smithi175 ceph-mon[36886]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-06-13T07:09:10.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:09 smithi007 ceph-mon[32077]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-06-13T07:09:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:11 smithi189 ceph-mon[36616]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-06-13T07:09:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:11 smithi175 ceph-mon[36886]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-06-13T07:09:12.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:11 smithi007 ceph-mon[32077]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-06-13T07:09:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 790.0 s, 644.2 tps, lat 144.812 ms stddev 169.045 2024-06-13T07:09:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:13 smithi189 ceph-mon[36616]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.1 MiB/s wr, 530 op/s 2024-06-13T07:09:14.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:13 smithi175 ceph-mon[36886]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.1 MiB/s wr, 530 op/s 2024-06-13T07:09:14.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:13 smithi007 ceph-mon[32077]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 5.1 MiB/s wr, 530 op/s 2024-06-13T07:09:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:15 smithi189 ceph-mon[36616]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-06-13T07:09:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:15 smithi175 ceph-mon[36886]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-06-13T07:09:16.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:15 smithi007 ceph-mon[32077]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-06-13T07:09:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 795.0 s, 443.2 tps, lat 235.440 ms stddev 267.842 2024-06-13T07:09:18.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:17 smithi189 ceph-mon[36616]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-13T07:09:18.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:17 smithi175 ceph-mon[36886]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-13T07:09:18.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:17 smithi007 ceph-mon[32077]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.1 MiB/s wr, 538 op/s 2024-06-13T07:09:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:19 smithi189 ceph-mon[36616]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-06-13T07:09:20.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:19 smithi175 ceph-mon[36886]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-06-13T07:09:20.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:19 smithi007 ceph-mon[32077]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 4.8 MiB/s wr, 537 op/s 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 -- 172.21.15.7:0/4116809367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160109300 msgr2=0x7f416010fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 --2- 172.21.15.7:0/4116809367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160109300 0x7f416010fce0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f414c009860 tx=0x7f414c02a340 comp rx=0 tx=0).stop 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 -- 172.21.15.7:0/4116809367 shutdown_connections 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 --2- 172.21.15.7:0/4116809367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4160110220 0x7f4160112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 --2- 172.21.15.7:0/4116809367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160109300 0x7f416010fce0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 --2- 172.21.15.7:0/4116809367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41601089a0 0x7f4160108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 -- 172.21.15.7:0/4116809367 >> 172.21.15.7:0/4116809367 conn(0x7f416006ea10 msgr2=0x7f4160070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 -- 172.21.15.7:0/4116809367 shutdown_connections 2024-06-13T07:09:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.946+0000 7f4166e00640 1 -- 172.21.15.7:0/4116809367 wait complete. 2024-06-13T07:09:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 Processor -- start 2024-06-13T07:09:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 -- start start 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 0x7f416010b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160110220 0x7f416010a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4160115ee0 con 0x7f4160110220 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4160115d60 con 0x7f4160109300 2024-06-13T07:09:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.948+0000 7f4166e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4160116060 con 0x7f41601089a0 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 0x7f416010b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43534/0 (socket says 172.21.15.7:43534) 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 0x7f416010b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:47380/0 (socket says 172.21.15.7:47380) 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 -- 172.21.15.7:0/1934345519 learned_addr learned my addr 172.21.15.7:0/1934345519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 -- 172.21.15.7:0/1934345519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 msgr2=0x7f416010b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 0x7f416010b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 -- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160110220 msgr2=0x7f416010a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160110220 0x7f416010a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.949+0000 7f415f400640 1 -- 172.21.15.7:0/1934345519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416010aa40 con 0x7f4160109300 2024-06-13T07:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.950+0000 7f415f400640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f414c031000 tx=0x7f414c033da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.950+0000 7f415cc00640 1 -- 172.21.15.7:0/1934345519 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c037040 con 0x7f4160109300 2024-06-13T07:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.950+0000 7f415cc00640 1 -- 172.21.15.7:0/1934345519 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f414c030910 con 0x7f4160109300 2024-06-13T07:09:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.950+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601aa530 con 0x7f4160109300 2024-06-13T07:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.950+0000 7f415cc00640 1 -- 172.21.15.7:0/1934345519 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c004920 con 0x7f4160109300 2024-06-13T07:09:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.951+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601aaa80 con 0x7f4160109300 2024-06-13T07:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.957+0000 7f4169b87740 1 Processor -- start 2024-06-13T07:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 -- start start 2024-06-13T07:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8123c00 0x5637b8124020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 0x5637b812bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637b8087420 con 0x5637b8129760 2024-06-13T07:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637b802ec50 con 0x5637b8123c00 2024-06-13T07:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.958+0000 7f4169b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5637b80833e0 con 0x5637b81245f0 2024-06-13T07:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:47390/0 (socket says 172.21.15.7:47390) 2024-06-13T07:09:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f415fe00640 1 -- 172.21.15.7:0/1676744547 learned_addr learned my addr 172.21.15.7:0/1676744547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f415cc00640 1 -- 172.21.15.7:0/1934345519 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f414c005030 con 0x7f4160109300 2024-06-13T07:09:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f415f400640 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 0x5637b812bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.959+0000 7f4164e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8123c00 0x5637b8124020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415f400640 1 -- 172.21.15.7:0/1676744547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 msgr2=0x5637b81290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415f400640 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415f400640 1 -- 172.21.15.7:0/1676744547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8123c00 msgr2=0x5637b8124020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415f400640 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8123c00 0x5637b8124020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415f400640 1 -- 172.21.15.7:0/1676744547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b7d9b210 con 0x5637b8129760 2024-06-13T07:09:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.961+0000 7f415fe00640 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.962+0000 7f415f400640 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 0x5637b812bbf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f414c0ec8e0 tx=0x7f414c0f7220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0074679fb4f584 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.962+0000 7f4155600640 1 -- 172.21.15.7:0/1676744547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0f7a20 con 0x5637b8129760 2024-06-13T07:09:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.962+0000 7f4155600640 1 -- 172.21.15.7:0/1676744547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f414c0fc030 con 0x5637b8129760 2024-06-13T07:09:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.962+0000 7f4155600640 1 -- 172.21.15.7:0/1676744547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c100630 con 0x5637b8129760 2024-06-13T07:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 -- 172.21.15.7:0/1676744547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 msgr2=0x5637b812bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 0x5637b812bbf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f414c0ec8e0 tx=0x7f414c0f7220 comp rx=0 tx=0).stop 2024-06-13T07:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 -- 172.21.15.7:0/1676744547 shutdown_connections 2024-06-13T07:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8129760 0x5637b812bbf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b81290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 --2- 172.21.15.7:0/1676744547 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8123c00 0x5637b8124020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.964+0000 7f4169b87740 1 -- 172.21.15.7:0/1676744547 >> 172.21.15.7:0/1676744547 conn(0x5637b80ebfb0 msgr2=0x5637b811f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.965+0000 7f415cc00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 0x7f41440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.965+0000 7f4169b87740 1 -- 172.21.15.7:0/1676744547 shutdown_connections 2024-06-13T07:09:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.966+0000 7f4169b87740 1 -- 172.21.15.7:0/1676744547 wait complete. 2024-06-13T07:09:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.966+0000 7f415cc00640 1 -- 172.21.15.7:0/1934345519 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f414c0ef1e0 con 0x7f4160109300 2024-06-13T07:09:20.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.966+0000 7f4169b87740 1 -- start start 2024-06-13T07:09:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 Processor -- start 2024-06-13T07:09:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 0x5637b8136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637b812c4d0 con 0x5637b81245f0 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.967+0000 7f4169b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5637b80420b0 con 0x5637b8129760 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.968+0000 7f4169b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5637b812c650 con 0x5637b8123c00 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.968+0000 7f4169b87740 10 client.0 _ll_register_callbacks cb 0x5637b802df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.968+0000 7f415fe00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 0x7f41440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:47402/0 (socket says 172.21.15.7:47402) 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 -- 172.21.15.7:0/4024925821 learned_addr learned my addr 172.21.15.7:0/4024925821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f4164e00640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415f400640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 0x5637b8136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 -- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 msgr2=0x5637b813d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.969+0000 7f415fe00640 1 -- 172.21.15.7:0/4024925821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 msgr2=0x5637b8136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.970+0000 7f415fe00640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 0x5637b8136980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.970+0000 7f415fe00640 1 -- 172.21.15.7:0/4024925821 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b7e67620 con 0x5637b8123c00 2024-06-13T07:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.970+0000 7f415f400640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 0x5637b8136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.970+0000 7f415fe00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 0x7f41440a5240 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f415000ba40 tx=0x7f415000be40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.970+0000 7f415fe00640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41500039c0 tx=0x7f415001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61301450324c7b0f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.975+0000 7f414a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.975+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415001b040 con 0x5637b8123c00 2024-06-13T07:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.975+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4150005220 con 0x5637b8123c00 2024-06-13T07:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.975+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414c0fc030 con 0x5637b8123c00 2024-06-13T07:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.976+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4150005420 con 0x5637b8123c00 2024-06-13T07:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.976+0000 7f4169b87740 10 client.27202 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.976+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5637b81307a0 con 0x5637b8123c00 2024-06-13T07:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.976+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f414c0fc030 con 0x5637b8123c00 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f415001e030 con 0x5637b8123c00 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 10 client.27202 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 10 client.27202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 20 client.27202 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b8131ee0 0x5637b8134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 4 client.27202 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.978+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b8094b50 con 0x5637b8131ee0 2024-06-13T07:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.979+0000 7f415f400640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b8131ee0 0x5637b8134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.980+0000 7f415f400640 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b8131ee0 0x5637b8134370 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495df102c77d4413 server_cookie=de97bda0fde9a85a in_seq=0 out_seq=0 2024-06-13T07:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.980+0000 7f414a000640 10 client.27202 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.981+0000 7f414a000640 1 -- 172.21.15.7:0/4024925821 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5637b8094b50 con 0x5637b8131ee0 2024-06-13T07:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.981+0000 7f414a000640 10 client.27202 handle_command_reply: tid=0 2024-06-13T07:09:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.981+0000 7f4169b87740 1 client.27202 shutdown 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 msgr2=0x5637b8136440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41500039c0 tx=0x7f415001c000 comp rx=0 tx=0).stop 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 shutdown_connections 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b8131ee0 0x5637b8134370 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b81245f0 0x5637b8136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 --2- 172.21.15.7:0/4024925821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8123c00 0x5637b8136440 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41500039c0 tx=0x7f415001c000 comp rx=0 tx=0).stop 2024-06-13T07:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 >> 172.21.15.7:0/4024925821 conn(0x5637b80ebfb0 msgr2=0x5637b811e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f414a000640 0 client.27202 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 shutdown_connections 2024-06-13T07:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.982+0000 7f4169b87740 1 -- 172.21.15.7:0/4024925821 wait complete. 2024-06-13T07:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:20.983+0000 7f4169b87740 20 client.27202 trim_cache size 0 max 16384 2024-06-13T07:09:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.023+0000 7f4169b87740 1 Processor -- start 2024-06-13T07:09:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.023+0000 7f4169b87740 1 -- start start 2024-06-13T07:09:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 0x5637b813d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b8153f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b3410 con 0x5637b8122af0 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b0c80 con 0x5637b81245f0 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.024+0000 7f4169b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41601aead0 con 0x5637b8129760 2024-06-13T07:09:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 0x5637b813d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43552/0 (socket says 172.21.15.7:43552) 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 -- 172.21.15.7:0/1547368850 learned_addr learned my addr 172.21.15.7:0/1547368850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 -- 172.21.15.7:0/1547368850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 msgr2=0x5637b813d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 0x5637b813d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 -- 172.21.15.7:0/1547368850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 msgr2=0x5637b8153f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f4164e00640 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b8153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b8153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415fe00640 1 -- 172.21.15.7:0/1547368850 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b81249d0 con 0x5637b81245f0 2024-06-13T07:09:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.025+0000 7f415f400640 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 0x5637b813d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.027+0000 7f415fe00640 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4150004f80 tx=0x7f4150003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531c59cf882bd5d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.027+0000 7f4154c00640 1 -- 172.21.15.7:0/1547368850 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150038070 con 0x5637b81245f0 2024-06-13T07:09:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.027+0000 7f4154c00640 1 -- 172.21.15.7:0/1547368850 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150025030 con 0x5637b81245f0 2024-06-13T07:09:21.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.027+0000 7f4154c00640 1 -- 172.21.15.7:0/1547368850 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415001b040 con 0x5637b81245f0 2024-06-13T07:09:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.029+0000 7f4169b87740 1 -- 172.21.15.7:0/1547368850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 msgr2=0x5637b81225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.029+0000 7f4169b87740 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4150004f80 tx=0x7f4150003b20 comp rx=0 tx=0).stop 2024-06-13T07:09:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 -- 172.21.15.7:0/1547368850 shutdown_connections 2024-06-13T07:09:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b8153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b8129760 0x5637b813d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 --2- 172.21.15.7:0/1547368850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b81245f0 0x5637b81225b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 -- 172.21.15.7:0/1547368850 >> 172.21.15.7:0/1547368850 conn(0x5637b80ebfb0 msgr2=0x5637b811f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 -- 172.21.15.7:0/1547368850 shutdown_connections 2024-06-13T07:09:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.030+0000 7f4169b87740 1 -- 172.21.15.7:0/1547368850 wait complete. 2024-06-13T07:09:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.031+0000 7f4169b87740 1 -- start start 2024-06-13T07:09:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.031+0000 7f4169b87740 1 Processor -- start 2024-06-13T07:09:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.032+0000 7f4169b87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b813d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41601af140 con 0x5637b8122af0 2024-06-13T07:09:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b05b0 con 0x5637b8129760 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41601af810 con 0x5637b81245f0 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4169b87740 10 client.0 _ll_register_callbacks cb 0x7f4150005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f415fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f415fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55018/0 (socket says 172.21.15.7:55018) 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f415fe00640 1 -- 172.21.15.7:0/1595096312 learned_addr learned my addr 172.21.15.7:0/1595096312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.033+0000 7f4164e00640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415f400640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b813d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 -- 172.21.15.7:0/1595096312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 msgr2=0x5637b813d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b813d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 -- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 msgr2=0x5637b813b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 -- 172.21.15.7:0/1595096312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b813bef0 con 0x5637b8122af0 2024-06-13T07:09:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.034+0000 7f415fe00640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41500053f0 tx=0x7f4150004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bed7fd19b183d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.035+0000 7f414a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:09:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.035+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150038040 con 0x5637b8122af0 2024-06-13T07:09:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.036+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f4150021470 con 0x5637b8122af0 2024-06-13T07:09:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.036+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4150025030 con 0x5637b8122af0 2024-06-13T07:09:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.036+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4150047050 con 0x5637b8122af0 2024-06-13T07:09:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.037+0000 7f4169b87740 10 client.16533 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.037+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5637b81b58d0 con 0x5637b8122af0 2024-06-13T07:09:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.038+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150025030 con 0x5637b8122af0 2024-06-13T07:09:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.038+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f415001e050 con 0x5637b8122af0 2024-06-13T07:09:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 10 client.16533 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 20 client.16533 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b81b6190 0x5637b81b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 4 client.16533 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.039+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637b808eea0 con 0x5637b81b6190 2024-06-13T07:09:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.040+0000 7f415f400640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b81b6190 0x5637b81b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.040+0000 7f415f400640 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b81b6190 0x5637b81b85b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80378ef17e36f3b9 server_cookie=ddf0df4269e304de in_seq=0 out_seq=0 2024-06-13T07:09:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.040+0000 7f414a000640 10 client.16533 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.087+0000 7f414a000640 1 -- 172.21.15.7:0/1595096312 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5637b808eea0 con 0x5637b81b6190 2024-06-13T07:09:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.087+0000 7f414a000640 10 client.16533 handle_command_reply: tid=0 2024-06-13T07:09:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.087+0000 7f4169b87740 1 client.16533 shutdown 2024-06-13T07:09:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.088+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 msgr2=0x5637b813d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.088+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41500053f0 tx=0x7f4150004ad0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 shutdown_connections 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5637b81b6190 0x5637b81b85b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5637b8129760 0x5637b813b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5637b81245f0 0x5637b813d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 --2- 172.21.15.7:0/1595096312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637b8122af0 0x5637b813d1f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 >> 172.21.15.7:0/1595096312 conn(0x5637b80ebfb0 msgr2=0x5637b811dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f414a000640 0 client.16533 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.089+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 shutdown_connections 2024-06-13T07:09:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.092+0000 7f4169b87740 1 -- 172.21.15.7:0/1595096312 wait complete. 2024-06-13T07:09:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.092+0000 7f4169b87740 20 client.16533 trim_cache size 0 max 16384 2024-06-13T07:09:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:09:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:09:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.093+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 msgr2=0x7f41440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.093+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 0x7f41440a5240 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f415000ba40 tx=0x7f415000be40 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.093+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 msgr2=0x7f4160109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.093+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f414c031000 tx=0x7f414c033da0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 shutdown_connections 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f41440a2d80 0x7f41440a5240 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4160110220 0x7f416010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4160109300 0x7f4160109c70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 --2- 172.21.15.7:0/1934345519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41601089a0 0x7f416010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 >> 172.21.15.7:0/1934345519 conn(0x7f416006ea10 msgr2=0x7f4160070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 shutdown_connections 2024-06-13T07:09:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:21.094+0000 7f4166e00640 1 -- 172.21.15.7:0/1934345519 wait complete. 2024-06-13T07:09:22.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:21 smithi189 ceph-mon[36616]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-06-13T07:09:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:21 smithi175 ceph-mon[36886]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-06-13T07:09:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:21 smithi007 ceph-mon[32077]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-06-13T07:09:22.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 800.0 s, 470.4 tps, lat 216.106 ms stddev 254.786 2024-06-13T07:09:24.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:23 smithi175 ceph-mon[36886]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-06-13T07:09:24.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:23 smithi007 ceph-mon[32077]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-06-13T07:09:24.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:23 smithi189 ceph-mon[36616]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.5 MiB/s wr, 479 op/s 2024-06-13T07:09:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:25 smithi175 ceph-mon[36886]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-06-13T07:09:26.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:25 smithi007 ceph-mon[32077]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-06-13T07:09:26.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:25 smithi189 ceph-mon[36616]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-06-13T07:09:27.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 805.0 s, 509.8 tps, lat 196.107 ms stddev 204.904 2024-06-13T07:09:28.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:27 smithi175 ceph-mon[36886]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-06-13T07:09:28.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:27 smithi007 ceph-mon[32077]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-06-13T07:09:28.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:27 smithi189 ceph-mon[36616]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.0 MiB/s wr, 441 op/s 2024-06-13T07:09:30.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:29 smithi175 ceph-mon[36886]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-13T07:09:30.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:29 smithi007 ceph-mon[32077]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-13T07:09:30.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:29 smithi189 ceph-mon[36616]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-06-13T07:09:30.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:09:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:31 smithi007 ceph-mon[32077]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-06-13T07:09:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:31 smithi189 ceph-mon[36616]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-06-13T07:09:32.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:31 smithi175 ceph-mon[36886]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-06-13T07:09:32.489 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:09:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 810.0 s, 418.0 tps, lat 231.288 ms stddev 280.165 2024-06-13T07:09:34.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:33 smithi007 ceph-mon[32077]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-06-13T07:09:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.321+0000 7fa4ff200640 1 -- 172.21.15.7:0/1606935606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 msgr2=0x7fa500109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.321+0000 7fa4ff200640 1 --2- 172.21.15.7:0/1606935606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500109950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009860 tx=0x7fa4ec02a360 comp rx=0 tx=0).stop 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 -- 172.21.15.7:0/1606935606 shutdown_connections 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 --2- 172.21.15.7:0/1606935606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500109950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 --2- 172.21.15.7:0/1606935606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa500104bd0 0x7fa500106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 --2- 172.21.15.7:0/1606935606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5001022a0 0x7fa500104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 -- 172.21.15.7:0/1606935606 >> 172.21.15.7:0/1606935606 conn(0x7fa5000fbc90 msgr2=0x7fa5000fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.322+0000 7fa4ff200640 1 -- 172.21.15.7:0/1606935606 shutdown_connections 2024-06-13T07:09:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.323+0000 7fa4ff200640 1 -- 172.21.15.7:0/1606935606 wait complete. 2024-06-13T07:09:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 Processor -- start 2024-06-13T07:09:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 -- start start 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5001022a0 0x7fa500101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa50006be10 con 0x7fa500107500 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa50006bc90 con 0x7fa500104bd0 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.325+0000 7fa4ff200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa50006bf90 con 0x7fa5001022a0 2024-06-13T07:09:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.326+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.326+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50280/0 (socket says 172.21.15.7:50280) 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.326+0000 7fa4fd400640 1 -- 172.21.15.7:0/3097392007 learned_addr learned my addr 172.21.15.7:0/3097392007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.326+0000 7fa4fe800640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fd400640 1 -- 172.21.15.7:0/3097392007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5001022a0 msgr2=0x7fa500101c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5001022a0 0x7fa500101c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fd400640 1 -- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 msgr2=0x7fa500100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fd400640 1 -- 172.21.15.7:0/3097392007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001007c0 con 0x7fa500104bd0 2024-06-13T07:09:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.327+0000 7fa4fe800640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000d770 tx=0x7fa4f000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4f6a00640 1 -- 172.21.15.7:0/3097392007 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0017070 con 0x7fa500104bd0 2024-06-13T07:09:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa500100ab0 con 0x7fa500104bd0 2024-06-13T07:09:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4f6a00640 1 -- 172.21.15.7:0/3097392007 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0009bd0 con 0x7fa500104bd0 2024-06-13T07:09:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4f6a00640 1 -- 172.21.15.7:0/3097392007 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0021060 con 0x7fa500104bd0 2024-06-13T07:09:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.329+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa500101050 con 0x7fa500104bd0 2024-06-13T07:09:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.334+0000 7fa5079e1740 1 Processor -- start 2024-06-13T07:09:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 -- start start 2024-06-13T07:09:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd453b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd4542db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dcd449e420 con 0x55dcd453adc0 2024-06-13T07:09:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dcd4445c50 con 0x55dcd4540920 2024-06-13T07:09:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.335+0000 7fa5079e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcd449a3e0 con 0x55dcd453b7b0 2024-06-13T07:09:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.336+0000 7fa4fde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.336+0000 7fa4fde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51096/0 (socket says 172.21.15.7:51096) 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.336+0000 7fa4fde00640 1 -- 172.21.15.7:0/1472587446 learned_addr learned my addr 172.21.15.7:0/1472587446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4fde00640 1 -- 172.21.15.7:0/1472587446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 msgr2=0x55dcd4542db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4fde00640 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd4542db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4fde00640 1 -- 172.21.15.7:0/1472587446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 msgr2=0x55dcd453b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4fde00640 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd453b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4fde00640 1 -- 172.21.15.7:0/1472587446 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd41b2210 con 0x55dcd453b7b0 2024-06-13T07:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.337+0000 7fa4f6a00640 1 -- 172.21.15.7:0/3097392007 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa4f001a070 con 0x7fa500104bd0 2024-06-13T07:09:34.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:33 smithi189 ceph-mon[36616]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-06-13T07:09:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.338+0000 7fa4fde00640 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e3f0 tx=0x7fa4f800e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd03361e87dd6d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.338+0000 7fa4eb000640 1 -- 172.21.15.7:0/1472587446 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004b20 con 0x55dcd453b7b0 2024-06-13T07:09:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.339+0000 7fa4f6a00640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 0x7fa4e40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.340+0000 7fa4eb000640 1 -- 172.21.15.7:0/1472587446 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8003e50 con 0x55dcd453b7b0 2024-06-13T07:09:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.340+0000 7fa4eb000640 1 -- 172.21.15.7:0/1472587446 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004b20 con 0x55dcd453b7b0 2024-06-13T07:09:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.340+0000 7fa4f6a00640 1 -- 172.21.15.7:0/3097392007 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa4f00cef00 con 0x7fa500104bd0 2024-06-13T07:09:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.340+0000 7fa4fe800640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 0x7fa4e40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 -- 172.21.15.7:0/1472587446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 msgr2=0x55dcd45402b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800e3f0 tx=0x7fa4f800e8b0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 -- 172.21.15.7:0/1472587446 shutdown_connections 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd4542db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd45402b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 --2- 172.21.15.7:0/1472587446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd453b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.341+0000 7fa5079e1740 1 -- 172.21.15.7:0/1472587446 >> 172.21.15.7:0/1472587446 conn(0x55dcd4502fb0 msgr2=0x55dcd4536840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.342+0000 7fa4fe800640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 0x7fa4e40a5140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec02a950 tx=0x7fa4ec005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.342+0000 7fa5079e1740 1 -- 172.21.15.7:0/1472587446 shutdown_connections 2024-06-13T07:09:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.342+0000 7fa5079e1740 1 -- 172.21.15.7:0/1472587446 wait complete. 2024-06-13T07:09:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.343+0000 7fa5079e1740 1 -- start start 2024-06-13T07:09:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.343+0000 7fa5079e1740 1 Processor -- start 2024-06-13T07:09:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.344+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd4545120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.344+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd454c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa5079e1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dcd45438d0 con 0x55dcd453adc0 2024-06-13T07:09:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa4fe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd454c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51112/0 (socket says 172.21.15.7:51112) 2024-06-13T07:09:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.345+0000 7fa4fd400640 1 -- 172.21.15.7:0/3139091441 learned_addr learned my addr 172.21.15.7:0/3139091441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.347+0000 7fa4fde00640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd4545120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.347+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55dcd44590b0 con 0x55dcd4540920 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.347+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcd4543a50 con 0x55dcd453b7b0 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.347+0000 7fa5079e1740 10 client.0 _ll_register_callbacks cb 0x55dcd4444f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.347+0000 7fa4fde00640 1 -- 172.21.15.7:0/3139091441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 msgr2=0x55dcd4545660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fde00640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fde00640 1 -- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 msgr2=0x55dcd454c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fde00640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd454c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fde00640 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd427e620 con 0x55dcd453adc0 2024-06-13T07:09:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fe800640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd454c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.348+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.351+0000 7fa4fde00640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd4545120 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55dcd45376f0 tx=0x7fa4f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546e6eadea906580 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.351+0000 7fa4df400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:09:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.351+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8003c70 con 0x55dcd453adc0 2024-06-13T07:09:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.351+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa4f8021980 con 0x55dcd453adc0 2024-06-13T07:09:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.352+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8003e50 con 0x55dcd453adc0 2024-06-13T07:09:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.352+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4f8021b80 con 0x55dcd453adc0 2024-06-13T07:09:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.352+0000 7fa5079e1740 10 client.16542 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.352+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dcd454f800 con 0x55dcd453adc0 2024-06-13T07:09:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.353+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8003e50 con 0x55dcd453adc0 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.353+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fa4f800b040 con 0x55dcd453adc0 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 10 client.16542 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 20 client.16542 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 4 client.16542 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd44abb50 con 0x55dcd4550cf0 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.354+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.356+0000 7fa4fd400640 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45530e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c68fae6aebd7d0a server_cookie=d4db29cc2dc12571 in_seq=0 out_seq=0 2024-06-13T07:09:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.356+0000 7fa4df400640 10 client.16542 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.357+0000 7fa4df400640 1 -- 172.21.15.7:0/3139091441 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dcd44abb50 con 0x55dcd4550cf0 2024-06-13T07:09:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.357+0000 7fa4df400640 10 client.16542 handle_command_reply: tid=0 2024-06-13T07:09:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.357+0000 7fa5079e1740 1 client.16542 shutdown 2024-06-13T07:09:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 msgr2=0x55dcd4545120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd4545120 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55dcd45376f0 tx=0x7fa4f8005fd0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 shutdown_connections 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45530e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa4df400640 0 client.16542 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd4540920 0x55dcd454c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd4545660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 --2- 172.21.15.7:0/3139091441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd453adc0 0x55dcd4545120 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 >> 172.21.15.7:0/3139091441 conn(0x55dcd4502fb0 msgr2=0x55dcd45359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 shutdown_connections 2024-06-13T07:09:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.358+0000 7fa5079e1740 1 -- 172.21.15.7:0/3139091441 wait complete. 2024-06-13T07:09:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.359+0000 7fa5079e1740 20 client.16542 trim_cache size 0 max 16384 2024-06-13T07:09:34.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:33 smithi175 ceph-mon[36886]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-06-13T07:09:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 Processor -- start 2024-06-13T07:09:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 -- start start 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453adc0 0x55dcd4539e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd4544740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b3250 con 0x55dcd4540920 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b0ac0 con 0x55dcd453adc0 2024-06-13T07:09:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.409+0000 7fa5079e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001ae910 con 0x55dcd453b7b0 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51122/0 (socket says 172.21.15.7:51122) 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 -- 172.21.15.7:0/879738520 learned_addr learned my addr 172.21.15.7:0/879738520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fe800640 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd4544740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 -- 172.21.15.7:0/879738520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453adc0 msgr2=0x55dcd4539e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453adc0 0x55dcd4539e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 -- 172.21.15.7:0/879738520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 msgr2=0x55dcd4544740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd4544740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.410+0000 7fa4fd400640 1 -- 172.21.15.7:0/879738520 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd4553450 con 0x55dcd453b7b0 2024-06-13T07:09:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.411+0000 7fa4fd400640 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa4f009b300 tx=0x7fa4f00bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b85bb49038034a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.411+0000 7fa4ea600640 1 -- 172.21.15.7:0/879738520 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0009d80 con 0x55dcd453b7b0 2024-06-13T07:09:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.411+0000 7fa4ea600640 1 -- 172.21.15.7:0/879738520 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f00cc770 con 0x55dcd453b7b0 2024-06-13T07:09:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.411+0000 7fa4ea600640 1 -- 172.21.15.7:0/879738520 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f009aae0 con 0x55dcd453b7b0 2024-06-13T07:09:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 -- 172.21.15.7:0/879738520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 msgr2=0x55dcd454c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa4f009b300 tx=0x7fa4f00bb040 comp rx=0 tx=0).stop 2024-06-13T07:09:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 -- 172.21.15.7:0/879738520 shutdown_connections 2024-06-13T07:09:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd4544740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453b7b0 0x55dcd454c660 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 --2- 172.21.15.7:0/879738520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453adc0 0x55dcd4539e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 -- 172.21.15.7:0/879738520 >> 172.21.15.7:0/879738520 conn(0x55dcd4502fb0 msgr2=0x55dcd4539780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 -- 172.21.15.7:0/879738520 shutdown_connections 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.413+0000 7fa5079e1740 1 -- 172.21.15.7:0/879738520 wait complete. 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.414+0000 7fa5079e1740 1 -- start start 2024-06-13T07:09:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.414+0000 7fa5079e1740 1 Processor -- start 2024-06-13T07:09:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.415+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 0x55dcd45492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.415+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 0x55dcd45497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 0x55dcd45492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa5079e1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd45b9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 0x55dcd45492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51138/0 (socket says 172.21.15.7:51138) 2024-06-13T07:09:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 0x55dcd45497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fd400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 0x55dcd45497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50300/0 (socket says 172.21.15.7:50300) 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fde00640 1 -- 172.21.15.7:0/2306300957 learned_addr learned my addr 172.21.15.7:0/2306300957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001aef80 con 0x55dcd4540920 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b03f0 con 0x55dcd453b7b0 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001af650 con 0x55dcd453adc0 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa5079e1740 10 client.0 _ll_register_callbacks cb 0x7fa4f8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.416+0000 7fa4fe800640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd45b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 -- 172.21.15.7:0/2306300957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 msgr2=0x55dcd45492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 0x55dcd45492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 -- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 msgr2=0x55dcd45497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 0x55dcd45497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd454c8c0 con 0x55dcd4540920 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4fe800640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd45b9970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009780 tx=0x7fa4ec02a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0f97f3cd4425f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.417+0000 7fa4df400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:09:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.418+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec04e070 con 0x55dcd4540920 2024-06-13T07:09:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.418+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fa4ec0048c0 con 0x55dcd4540920 2024-06-13T07:09:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.418+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f00cc770 con 0x55dcd4540920 2024-06-13T07:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.420+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4ec058050 con 0x55dcd4540920 2024-06-13T07:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.420+0000 7fa5079e1740 10 client.16548 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.420+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55dcd45baeb0 con 0x55dcd4540920 2024-06-13T07:09:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.421+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f00cc770 con 0x55dcd4540920 2024-06-13T07:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fa4ec03d470 con 0x55dcd4540920 2024-06-13T07:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 10 client.16548 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 20 client.16548 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 4 client.16548 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcd44a5ea0 con 0x55dcd4550cf0 2024-06-13T07:09:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.422+0000 7fa4fde00640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.423+0000 7fa4fde00640 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45bd940 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb25072c6b639bb3 server_cookie=c37784b2a9770c16 in_seq=0 out_seq=0 2024-06-13T07:09:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.423+0000 7fa4df400640 10 client.16548 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.645+0000 7fa4df400640 1 -- 172.21.15.7:0/2306300957 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dcd44a5ea0 con 0x55dcd4550cf0 2024-06-13T07:09:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.645+0000 7fa4df400640 10 client.16548 handle_command_reply: tid=0 2024-06-13T07:09:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.645+0000 7fa5079e1740 1 client.16548 shutdown 2024-06-13T07:09:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 msgr2=0x55dcd45b9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd45b9970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009780 tx=0x7fa4ec02a360 comp rx=0 tx=0).stop 2024-06-13T07:09:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 shutdown_connections 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55dcd4550cf0 0x55dcd45bd940 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dcd4540920 0x55dcd45b9970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55dcd453b7b0 0x55dcd45497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 --2- 172.21.15.7:0/2306300957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcd453adc0 0x55dcd45492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 >> 172.21.15.7:0/2306300957 conn(0x55dcd4502fb0 msgr2=0x55dcd4539780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa4df400640 0 client.16548 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.646+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 shutdown_connections 2024-06-13T07:09:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.647+0000 7fa5079e1740 1 -- 172.21.15.7:0/2306300957 wait complete. 2024-06-13T07:09:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.647+0000 7fa5079e1740 20 client.16548 trim_cache size 0 max 16384 2024-06-13T07:09:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:09:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:09:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 msgr2=0x7fa4e40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 0x7fa4e40a5140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec02a950 tx=0x7fa4ec005d20 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 msgr2=0x7fa500073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000d770 tx=0x7fa4f000dc30 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 shutdown_connections 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fa4e40a2c80 0x7fa4e40a5140 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa500107500 0x7fa500100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa500104bd0 0x7fa500073360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 --2- 172.21.15.7:0/3097392007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5001022a0 0x7fa500101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 >> 172.21.15.7:0/3097392007 conn(0x7fa5000fbc90 msgr2=0x7fa500102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 shutdown_connections 2024-06-13T07:09:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:34.649+0000 7fa4ff200640 1 -- 172.21.15.7:0/3097392007 wait complete. 2024-06-13T07:09:36.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:35 smithi007 ceph-mon[32077]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-06-13T07:09:36.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:35 smithi189 ceph-mon[36616]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-06-13T07:09:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:35 smithi175 ceph-mon[36886]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-06-13T07:09:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 815.0 s, 585.8 tps, lat 179.942 ms stddev 232.220 2024-06-13T07:09:37.930 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:09:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:09:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:09:37.930 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:09:37.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 -- 172.21.15.7:0/2653794395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 msgr2=0x7f8028074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 --2- 172.21.15.7:0/2653794395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f8028074aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8018009990 tx=0x7f8018009ca0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 -- 172.21.15.7:0/2653794395 shutdown_connections 2024-06-13T07:09:37.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 --2- 172.21.15.7:0/2653794395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f802810ee90 0x7f8028111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 --2- 172.21.15.7:0/2653794395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f8028074aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 --2- 172.21.15.7:0/2653794395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8028074fe0 0x7f802810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 -- 172.21.15.7:0/2653794395 >> 172.21.15.7:0/2653794395 conn(0x7f80280795f0 msgr2=0x7f8028079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 -- 172.21.15.7:0/2653794395 shutdown_connections 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.797+0000 7f802e400700 1 -- 172.21.15.7:0/2653794395 wait complete. 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 Processor -- start 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 -- start start 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8028074fe0 0x7f802819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f802810ee90 0x7f802819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80281149c0 con 0x7f80280746c0 2024-06-13T07:09:37.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114840 con 0x7f8028074fe0 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f802e400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8028114b40 con 0x7f802810ee90 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f8027400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f8027400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50982/0 (socket says 172.21.15.7:50982) 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.799+0000 7f8027400700 1 -- 172.21.15.7:0/3353703175 learned_addr learned my addr 172.21.15.7:0/3353703175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027e00700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f802810ee90 0x7f802819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8026a00700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8028074fe0 0x7f802819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 -- 172.21.15.7:0/3353703175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f802810ee90 msgr2=0x7f802819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f802810ee90 0x7f802819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 -- 172.21.15.7:0/3353703175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8028074fe0 msgr2=0x7f802819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8028074fe0 0x7f802819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 -- 172.21.15.7:0/3353703175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819fe30 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f8027400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f801800bae0 tx=0x7f801800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f801fe00700 1 -- 172.21.15.7:0/3353703175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018026070 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f801fe00700 1 -- 172.21.15.7:0/3353703175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8018003d70 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f801fe00700 1 -- 172.21.15.7:0/3353703175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018021410 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.800+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a00c0 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.801+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a79a0 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.804+0000 7f801fe00700 1 -- 172.21.15.7:0/3353703175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8018029070 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f801fe00700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 0x7f800c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 Processor -- start 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f801fe00700 1 -- 172.21.15.7:0/3353703175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80180d0170 con 0x7f80280746c0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 -- start start 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8026a00700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 0x7f800c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce7b5b0 0x55a18ce7ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce886f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a18cd8aa10 con 0x55a18ce7b5b0 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.805+0000 7f8038c90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a18cdcdb00 con 0x55a18ce86300 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8038c90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a18cd888f0 con 0x55a18cdf7e40 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:51156/0 (socket says 172.21.15.7:51156) 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 -- 172.21.15.7:0/1025740976 learned_addr learned my addr 172.21.15.7:0/1025740976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027e00700 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce886f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8026a00700 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce7b5b0 0x55a18ce7ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 -- 172.21.15.7:0/1025740976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 msgr2=0x55a18ce886f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce886f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 -- 172.21.15.7:0/1025740976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce7b5b0 msgr2=0x55a18ce7ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce7b5b0 0x55a18ce7ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 -- 172.21.15.7:0/1025740976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18cd75f90 con 0x55a18cdf7e40 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8026a00700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 0x7f800c0a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8028110a70 tx=0x7f8014011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.806+0000 7f8027400700 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8018098b20 tx=0x7f801800fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30daa5d3820fdf0c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.807+0000 7f801cc00700 1 -- 172.21.15.7:0/1025740976 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018087110 con 0x55a18cdf7e40 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.807+0000 7f801cc00700 1 -- 172.21.15.7:0/1025740976 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801809cdc0 con 0x55a18cdf7e40 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.807+0000 7f801cc00700 1 -- 172.21.15.7:0/1025740976 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801809cb20 con 0x55a18cdf7e40 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.807+0000 7f8038c90b80 1 -- 172.21.15.7:0/1025740976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 msgr2=0x55a18ce7b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.807+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8018098b20 tx=0x7f801800fa20 comp rx=0 tx=0).stop 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 -- 172.21.15.7:0/1025740976 shutdown_connections 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce886f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce7b070 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1025740976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce7b5b0 0x55a18ce7ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 -- 172.21.15.7:0/1025740976 >> 172.21.15.7:0/1025740976 conn(0x55a18cda8170 msgr2=0x55a18ce774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 -- 172.21.15.7:0/1025740976 shutdown_connections 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 -- 172.21.15.7:0/1025740976 wait complete. 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.808+0000 7f8038c90b80 1 -- start start 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 Processor -- start 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce85a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a18cd69f60 con 0x55a18ce86300 2024-06-13T07:09:37.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a18cdc9ac0 con 0x55a18ce7b5b0 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a18ce88d70 con 0x55a18cdf7e40 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8038c90b80 10 client.0 _ll_register_callbacks cb 0x55a18cd67720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.809+0000 7f8027e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce85a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51000/0 (socket says 172.21.15.7:51000) 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027e00700 1 -- 172.21.15.7:0/1940299844 learned_addr learned my addr 172.21.15.7:0/1940299844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8026a00700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 -- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 msgr2=0x55a18ce82d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce82d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 -- 172.21.15.7:0/1940299844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 msgr2=0x55a18ce832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 -- 172.21.15.7:0/1940299844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18cd1c8d0 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8026a00700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce82d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027e00700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8027400700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce85a90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55a18ce783b0 tx=0x7f801809ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da712da70f39f81 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8011600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018087110 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f80180a4e10 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.810+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801809cdc0 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f80180a0a70 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 10 client.27232 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a18ce93440 con 0x55a18cdf7e40 2024-06-13T07:09:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801809cdc0 con 0x55a18cdf7e40 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f801809bb10 con 0x55a18cdf7e40 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 10 client.27232 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 10 client.27232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 20 client.27232 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18ce95910 0x55a18ce97d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 4 client.27232 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.811+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18cddb000 con 0x55a18ce95910 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.812+0000 7f8026a00700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18ce95910 0x55a18ce97d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.812+0000 7f8026a00700 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18ce95910 0x55a18ce97d00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a8f13c8c4e818 server_cookie=805969e5db0b5c09 in_seq=0 out_seq=0 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.812+0000 7f8011600700 10 client.27232 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:37.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8011600700 1 -- 172.21.15.7:0/1940299844 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a18cddb000 con 0x55a18ce95910 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8011600700 10 client.27232 handle_command_reply: tid=0 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 client.27232 shutdown 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 msgr2=0x55a18ce85a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce85a90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55a18ce783b0 tx=0x7f801809ccb0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 shutdown_connections 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18ce95910 0x55a18ce97d00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18ce86300 0x55a18ce832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18cdf7e40 0x55a18ce85a90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8011600700 0 client.27232 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1940299844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce82d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 >> 172.21.15.7:0/1940299844 conn(0x55a18cda8170 msgr2=0x55a18ce76740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.813+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 shutdown_connections 2024-06-13T07:09:37.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.814+0000 7f8038c90b80 1 -- 172.21.15.7:0/1940299844 wait complete. 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.814+0000 7f8038c90b80 20 client.27232 trim_cache size 0 max 16384 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.835+0000 7f8038c90b80 1 Processor -- start 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.835+0000 7f8038c90b80 1 -- start start 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce7b5b0 0x55a18ce855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80281b2d40 con 0x55a18cdf7e40 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80281b0570 con 0x55a18ce86300 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8038c90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80281ae3c0 con 0x55a18ce7b5b0 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51008/0 (socket says 172.21.15.7:51008) 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8026a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce7b5b0 0x55a18ce855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50334/0 (socket says 172.21.15.7:50334) 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027400700 1 -- 172.21.15.7:0/2725977230 learned_addr learned my addr 172.21.15.7:0/2725977230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027e00700 1 -- 172.21.15.7:0/2725977230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce7b5b0 msgr2=0x55a18ce855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.836+0000 7f8027e00700 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce7b5b0 0x55a18ce855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8027e00700 1 -- 172.21.15.7:0/2725977230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 msgr2=0x55a18ce765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8027e00700 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8027e00700 1 -- 172.21.15.7:0/2725977230 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18ce7bfd0 con 0x55a18ce86300 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8027e00700 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8020009d60 tx=0x7f8020009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afdb170af2220bc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8013e00700 1 -- 172.21.15.7:0/2725977230 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020004db0 con 0x55a18ce86300 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8013e00700 1 -- 172.21.15.7:0/2725977230 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802001b070 con 0x55a18ce86300 2024-06-13T07:09:37.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.837+0000 7f8013e00700 1 -- 172.21.15.7:0/2725977230 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802001f680 con 0x55a18ce86300 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- 172.21.15.7:0/2725977230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 msgr2=0x55a18ce876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8020009d60 tx=0x7f8020009d90 comp rx=0 tx=0).stop 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- 172.21.15.7:0/2725977230 shutdown_connections 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce86300 0x55a18ce876f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 --2- 172.21.15.7:0/2725977230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce7b5b0 0x55a18ce855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- 172.21.15.7:0/2725977230 >> 172.21.15.7:0/2725977230 conn(0x55a18cda8170 msgr2=0x55a18ce75eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- 172.21.15.7:0/2725977230 shutdown_connections 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- 172.21.15.7:0/2725977230 wait complete. 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 -- start start 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.838+0000 7f8038c90b80 1 Processor -- start 2024-06-13T07:09:37.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce9d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce86300 0x55a18ce9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80281aea30 con 0x55a18cdf7e40 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80281afea0 con 0x55a18ce7b5b0 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80281af100 con 0x55a18ce86300 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8038c90b80 10 client.0 _ll_register_callbacks cb 0x55a18cd67720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8027400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce9d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50340/0 (socket says 172.21.15.7:50340) 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 -- 172.21.15.7:0/1367608348 learned_addr learned my addr 172.21.15.7:0/1367608348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8027e00700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce86300 0x55a18ce9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 -- 172.21.15.7:0/1367608348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce86300 msgr2=0x55a18ce9dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce86300 0x55a18ce9dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 -- 172.21.15.7:0/1367608348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 msgr2=0x55a18ce9d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.839+0000 7f8026a00700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce9d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8026a00700 1 -- 172.21.15.7:0/1367608348 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18ce8c980 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8026a00700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f801401b000 tx=0x7f8014016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=522852ee62195f50 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8011600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014017040 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f802001b070 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f801402b4e0 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8014004830 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8038c90b80 10 client.27347 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a18cf00810 con 0x55a18ce7b5b0 2024-06-13T07:09:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.840+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802001b070 con 0x55a18ce7b5b0 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f801402b6e0 con 0x55a18ce7b5b0 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 10 client.27347 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 10 client.27347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 20 client.27347 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18cf00f00 0x55a18cf03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 4 client.27347 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a18cdd5580 con 0x55a18cf00f00 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.841+0000 7f8027400700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18cf00f00 0x55a18cf03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8027400700 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18cf00f00 0x55a18cf03320 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cdfa9e2cc9f1a5 server_cookie=8617b1cee2d86647 in_seq=0 out_seq=0 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8011600700 10 client.27347 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8011600700 1 -- 172.21.15.7:0/1367608348 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a18cdd5580 con 0x55a18cf00f00 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8011600700 10 client.27347 handle_command_reply: tid=0 2024-06-13T07:09:37.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 client.27347 shutdown 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 msgr2=0x55a18ce9d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f801401b000 tx=0x7f8014016000 comp rx=0 tx=0).stop 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 shutdown_connections 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a18cf00f00 0x55a18cf03320 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a18ce86300 0x55a18ce9dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a18cdf7e40 0x55a18ce9d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8011600700 0 client.27347 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 --2- 172.21.15.7:0/1367608348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a18ce7b5b0 0x55a18ce9d680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 >> 172.21.15.7:0/1367608348 conn(0x55a18cda8170 msgr2=0x55a18ce7cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 shutdown_connections 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 1 -- 172.21.15.7:0/1367608348 wait complete. 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.842+0000 7f8038c90b80 20 client.27347 trim_cache size 0 max 16384 2024-06-13T07:09:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 msgr2=0x7f800c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 0x7f800c0a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8028110a70 tx=0x7f8014011720 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 msgr2=0x7f802819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f801800bae0 tx=0x7f801800bb10 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 shutdown_connections 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f802810ee90 0x7f802819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80280746c0 0x7f802819ae70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f800c0a2c20 0x7f800c0a50e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 --2- 172.21.15.7:0/3353703175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8028074fe0 0x7f802819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 >> 172.21.15.7:0/3353703175 conn(0x7f80280795f0 msgr2=0x7f80280ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 shutdown_connections 2024-06-13T07:09:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:35.843+0000 7f802e400700 1 -- 172.21.15.7:0/3353703175 wait complete. 2024-06-13T07:09:38.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:37 smithi007 ceph-mon[32077]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-06-13T07:09:38.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:37 smithi189 ceph-mon[36616]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-06-13T07:09:38.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:37 smithi175 ceph-mon[36886]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-06-13T07:09:40.015 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:09:40.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:09:40.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:39 smithi007 ceph-mon[32077]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-06-13T07:09:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:39 smithi189 ceph-mon[36616]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-06-13T07:09:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:39 smithi175 ceph-mon[36886]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-06-13T07:09:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:09:42.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:41 smithi007 ceph-mon[32077]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:09:42.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:41 smithi189 ceph-mon[36616]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:09:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:41 smithi175 ceph-mon[36886]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-06-13T07:09:42.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 820.0 s, 416.7 tps, lat 233.296 ms stddev 303.699 2024-06-13T07:09:44.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:43 smithi007 ceph-mon[32077]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-06-13T07:09:44.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:43 smithi189 ceph-mon[36616]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-06-13T07:09:44.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:43 smithi175 ceph-mon[36886]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.9 MiB/s wr, 533 op/s 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.421+0000 7efe92c00700 1 -- 172.21.15.7:0/2806081415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06e8f0 msgr2=0x7efe8c11b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.421+0000 7efe92c00700 1 --2- 172.21.15.7:0/2806081415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06e8f0 0x7efe8c11b600 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efe780098f0 tx=0x7efe78009c00 comp rx=0 tx=0).stop 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.423+0000 7efe92c00700 1 -- 172.21.15.7:0/2806081415 shutdown_connections 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.423+0000 7efe92c00700 1 --2- 172.21.15.7:0/2806081415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c11bb40 0x7efe8c11df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.423+0000 7efe92c00700 1 --2- 172.21.15.7:0/2806081415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06dfd0 0x7efe8c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.423+0000 7efe92c00700 1 --2- 172.21.15.7:0/2806081415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06e8f0 0x7efe8c11b600 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.423+0000 7efe92c00700 1 -- 172.21.15.7:0/2806081415 >> 172.21.15.7:0/2806081415 conn(0x7efe8c06d6b0 msgr2=0x7efe8c10b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:45.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.424+0000 7efe92c00700 1 -- 172.21.15.7:0/2806081415 shutdown_connections 2024-06-13T07:09:45.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.425+0000 7efe92c00700 1 -- 172.21.15.7:0/2806081415 wait complete. 2024-06-13T07:09:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.426+0000 7efe92c00700 1 Processor -- start 2024-06-13T07:09:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 -- start start 2024-06-13T07:09:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06dfd0 0x7efe8c1ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:45.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c11bb40 0x7efe8c1a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1216b0 con 0x7efe8c11bb40 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c121530 con 0x7efe8c06dfd0 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.427+0000 7efe92c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c121830 con 0x7efe8c06e8f0 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.428+0000 7efe8b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.428+0000 7efe8b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50250/0 (socket says 172.21.15.7:50250) 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.428+0000 7efe8b400700 1 -- 172.21.15.7:0/853265032 learned_addr learned my addr 172.21.15.7:0/853265032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.428+0000 7efe8be00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06dfd0 0x7efe8c1ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 -- 172.21.15.7:0/853265032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06dfd0 msgr2=0x7efe8c1ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06dfd0 0x7efe8c1ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 -- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c11bb40 msgr2=0x7efe8c1a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c11bb40 0x7efe8c1a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 -- 172.21.15.7:0/853265032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c1a80d0 con 0x7efe8c06e8f0 2024-06-13T07:09:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.429+0000 7efe8b400700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe78009230 tx=0x7efe780050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.430+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78005310 con 0x7efe8c06e8f0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.430+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7800ba40 con 0x7efe8c06e8f0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.430+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78003c20 con 0x7efe8c06e8f0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.430+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c1a8360 con 0x7efe8c06e8f0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.430+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1b4680 con 0x7efe8c06e8f0 2024-06-13T07:09:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.432+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c06f710 con 0x7efe8c06e8f0 2024-06-13T07:09:45.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.441+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efe780054b0 con 0x7efe8c06e8f0 2024-06-13T07:09:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.444+0000 7efe88c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 0x7efe700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:45.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.445+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7efe780d0480 con 0x7efe8c06e8f0 2024-06-13T07:09:45.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.445+0000 7efe8be00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 0x7efe700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:45.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.446+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe78013030 con 0x7efe8c06e8f0 2024-06-13T07:09:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.446+0000 7efe8be00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 0x7efe700a5260 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efe8c1a8ab0 tx=0x7efe7c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.910+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8c1a8ea0 con 0x7efe8c06e8f0 2024-06-13T07:09:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.923+0000 7efe88c00700 1 -- 172.21.15.7:0/853265032 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11871 (secure 0 0 0) 0x7efe7809d1b0 con 0x7efe8c06e8f0 2024-06-13T07:09:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:09:45.923 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:08:39.686612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24527,24539,24521]},"id":1}]} 2024-06-13T07:09:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 msgr2=0x7efe700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 0x7efe700a5260 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efe8c1a8ab0 tx=0x7efe7c009310 comp rx=0 tx=0).stop 2024-06-13T07:09:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 msgr2=0x7efe8c1adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe78009230 tx=0x7efe780050f0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 shutdown_connections 2024-06-13T07:09:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c11bb40 0x7efe8c1a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7efe700a2da0 0x7efe700a5260 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe8c06dfd0 0x7efe8c1ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 --2- 172.21.15.7:0/853265032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe8c06e8f0 0x7efe8c1adcf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.927+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 >> 172.21.15.7:0/853265032 conn(0x7efe8c06d6b0 msgr2=0x7efe8c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.928+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 shutdown_connections 2024-06-13T07:09:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:45.928+0000 7efe92c00700 1 -- 172.21.15.7:0/853265032 wait complete. 2024-06-13T07:09:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2024-06-13T07:09:46.309 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:45 smithi007 ceph-mon[32077]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-06-13T07:09:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:45 smithi189 ceph-mon[36616]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-06-13T07:09:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:45 smithi175 ceph-mon[36886]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-06-13T07:09:47.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:46 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/853265032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:09:47.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:46 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/853265032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:09:47.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:46 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/853265032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 -- 172.21.15.7:0/142785523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc103c20 msgr2=0x7fc5fc104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 --2- 172.21.15.7:0/142785523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc103c20 0x7fc5fc104040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009900 tx=0x7fc5e802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 -- 172.21.15.7:0/142785523 shutdown_connections 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 --2- 172.21.15.7:0/142785523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc1057d0 0x7fc5fc10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 --2- 172.21.15.7:0/142785523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc104e10 0x7fc5fc105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 --2- 172.21.15.7:0/142785523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc103c20 0x7fc5fc104040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 -- 172.21.15.7:0/142785523 >> 172.21.15.7:0/142785523 conn(0x7fc5fc0ff190 msgr2=0x7fc5fc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 -- 172.21.15.7:0/142785523 shutdown_connections 2024-06-13T07:09:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.879+0000 7fc604400640 1 -- 172.21.15.7:0/142785523 wait complete. 2024-06-13T07:09:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 825.0 s, 462.3 tps, lat 205.238 ms stddev 247.706 2024-06-13T07:09:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.881+0000 7fc604400640 1 Processor -- start 2024-06-13T07:09:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.881+0000 7fc604400640 1 -- start start 2024-06-13T07:09:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.882+0000 7fc604400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.882+0000 7fc604400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 0x7fc5fc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.882+0000 7fc601a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.882+0000 7fc601a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50742/0 (socket says 172.21.15.7:50742) 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.882+0000 7fc604400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc1057d0 0x7fc5fc1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc601000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 0x7fc5fc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc601000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 0x7fc5fc19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50268/0 (socket says 172.21.15.7:50268) 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc601a00640 1 -- 172.21.15.7:0/3080690557 learned_addr learned my addr 172.21.15.7:0/3080690557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc602400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc1057d0 0x7fc5fc1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc076e90 con 0x7fc5fc1057d0 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc076d10 con 0x7fc5fc103c20 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.883+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc077010 con 0x7fc5fc104e10 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.884+0000 7fc601a00640 1 -- 172.21.15.7:0/3080690557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 msgr2=0x7fc5fc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.884+0000 7fc601a00640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 0x7fc5fc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.884+0000 7fc601a00640 1 -- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc1057d0 msgr2=0x7fc5fc1a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.884+0000 7fc601a00640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc1057d0 0x7fc5fc1a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.884+0000 7fc601a00640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc1a5eb0 con 0x7fc5fc103c20 2024-06-13T07:09:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.886+0000 7fc601a00640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8031440 tx=0x7fc5e8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.886+0000 7fc5fa000640 1 -- 172.21.15.7:0/3080690557 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e803f030 con 0x7fc5fc103c20 2024-06-13T07:09:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.886+0000 7fc5fa000640 1 -- 172.21.15.7:0/3080690557 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e8003ce0 con 0x7fc5fc103c20 2024-06-13T07:09:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.887+0000 7fc5fa000640 1 -- 172.21.15.7:0/3080690557 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8035070 con 0x7fc5fc103c20 2024-06-13T07:09:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.887+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc1a6140 con 0x7fc5fc103c20 2024-06-13T07:09:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.889+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1a65b0 con 0x7fc5fc103c20 2024-06-13T07:09:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.893+0000 7fc6071e8740 1 Processor -- start 2024-06-13T07:09:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 -- start start 2024-06-13T07:09:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc791c00 0x55d5bc792020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc7970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc6f5420 con 0x55d5bc791c00 2024-06-13T07:09:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc69cc50 con 0x55d5bc797760 2024-06-13T07:09:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc6071e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc6f13e0 con 0x55d5bc7925f0 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.894+0000 7fc5fa000640 1 -- 172.21.15.7:0/3080690557 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc5e8042030 con 0x7fc5fc103c20 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50752/0 (socket says 172.21.15.7:50752) 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 -- 172.21.15.7:0/2856725478 learned_addr learned my addr 172.21.15.7:0/2856725478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc7970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 -- 172.21.15.7:0/2856725478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 msgr2=0x55d5bc7970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc7970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 -- 172.21.15.7:0/2856725478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc791c00 msgr2=0x55d5bc792020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc791c00 0x55d5bc792020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.895+0000 7fc601000640 1 -- 172.21.15.7:0/2856725478 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bc409210 con 0x55d5bc797760 2024-06-13T07:09:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.896+0000 7fc601a00640 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc7970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.896+0000 7fc601000640 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009c10 tx=0x7fc5f00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c41a6595b1a0375b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.896+0000 7fc5eea00640 1 -- 172.21.15.7:0/2856725478 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0012450 con 0x55d5bc797760 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.896+0000 7fc5eea00640 1 -- 172.21.15.7:0/2856725478 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0012ad0 con 0x55d5bc797760 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.896+0000 7fc5eea00640 1 -- 172.21.15.7:0/2856725478 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000fbf0 con 0x55d5bc797760 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 -- 172.21.15.7:0/2856725478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 msgr2=0x55d5bc799bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009c10 tx=0x7fc5f00117b0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 -- 172.21.15.7:0/2856725478 shutdown_connections 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc797760 0x55d5bc799bf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc7970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 --2- 172.21.15.7:0/2856725478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc791c00 0x55d5bc792020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.897+0000 7fc6071e8740 1 -- 172.21.15.7:0/2856725478 >> 172.21.15.7:0/2856725478 conn(0x55d5bc759fb0 msgr2=0x55d5bc78d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.898+0000 7fc6071e8740 1 -- 172.21.15.7:0/2856725478 shutdown_connections 2024-06-13T07:09:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.898+0000 7fc6071e8740 1 -- 172.21.15.7:0/2856725478 wait complete. 2024-06-13T07:09:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.898+0000 7fc6071e8740 1 -- start start 2024-06-13T07:09:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.899+0000 7fc6071e8740 1 Processor -- start 2024-06-13T07:09:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.899+0000 7fc5fa000640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 0x7fc5e40a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.900+0000 7fc5fa000640 1 -- 172.21.15.7:0/3080690557 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5e80eee00 con 0x7fc5fc103c20 2024-06-13T07:09:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc79c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc79a710 con 0x55d5bc797760 2024-06-13T07:09:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc6b00b0 con 0x55d5bc791c00 2024-06-13T07:09:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d5bc79a890 con 0x55d5bc7925f0 2024-06-13T07:09:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc6071e8740 10 client.0 _ll_register_callbacks cb 0x55d5bc69bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50768/0 (socket says 172.21.15.7:50768) 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 -- 172.21.15.7:0/4113057744 learned_addr learned my addr 172.21.15.7:0/4113057744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601000640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc79c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 -- 172.21.15.7:0/4113057744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 msgr2=0x55d5bc79c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc79c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc601a00640 1 -- 172.21.15.7:0/4113057744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 msgr2=0x55d5bc7a34d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.902+0000 7fc602400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 0x7fc5e40a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.903+0000 7fc601a00640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.903+0000 7fc601a00640 1 -- 172.21.15.7:0/4113057744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bc4d5620 con 0x55d5bc791c00 2024-06-13T07:09:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.904+0000 7fc601a00640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80092a0 tx=0x7fc5e80b6f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f02fd0f5e93053 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.904+0000 7fc5e3400640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:09:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.904+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80d4060 con 0x55d5bc791c00 2024-06-13T07:09:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.904+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f0012ad0 con 0x55d5bc791c00 2024-06-13T07:09:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.904+0000 7fc602400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 0x7fc5e40a4ff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55d5bc792e50 tx=0x7fc5f4009c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.905+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5e803b460 con 0x55d5bc791c00 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.905+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5e80bae00 con 0x55d5bc791c00 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.905+0000 7fc6071e8740 10 client.27365 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.905+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d5bc7a67b0 con 0x55d5bc791c00 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.906+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0012ad0 con 0x55d5bc791c00 2024-06-13T07:09:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.906+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fc5e803bc80 con 0x55d5bc791c00 2024-06-13T07:09:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 10 client.27365 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 20 client.27365 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc7a7e90 0x55d5bc7aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 4 client.27365 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5bc702b50 con 0x55d5bc7a7e90 2024-06-13T07:09:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.907+0000 7fc601000640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc7a7e90 0x55d5bc7aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.908+0000 7fc601000640 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc7a7e90 0x55d5bc7aa2e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b4385776ee7a9c server_cookie=d7ff034cee6ac94e in_seq=0 out_seq=0 2024-06-13T07:09:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.908+0000 7fc5e3400640 10 client.27365 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.910+0000 7fc5e3400640 1 -- 172.21.15.7:0/4113057744 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d5bc702b50 con 0x55d5bc7a7e90 2024-06-13T07:09:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.910+0000 7fc5e3400640 10 client.27365 handle_command_reply: tid=0 2024-06-13T07:09:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.910+0000 7fc6071e8740 1 client.27365 shutdown 2024-06-13T07:09:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.911+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 msgr2=0x55d5bc79bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.911+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80092a0 tx=0x7fc5e80b6f00 comp rx=0 tx=0).stop 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 shutdown_connections 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc7a7e90 0x55d5bc7aa2e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc7925f0 0x55d5bc79c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 --2- 172.21.15.7:0/4113057744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc79bf50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.913+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 >> 172.21.15.7:0/4113057744 conn(0x55d5bc759fb0 msgr2=0x55d5bc78c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.914+0000 7fc5e3400640 0 client.27365 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.914+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 shutdown_connections 2024-06-13T07:09:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.914+0000 7fc6071e8740 1 -- 172.21.15.7:0/4113057744 wait complete. 2024-06-13T07:09:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.915+0000 7fc6071e8740 20 client.27365 trim_cache size 0 max 16384 2024-06-13T07:09:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.981+0000 7fc6071e8740 1 Processor -- start 2024-06-13T07:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.981+0000 7fc6071e8740 1 -- start start 2024-06-13T07:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.982+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc790e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.982+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.982+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 0x55d5bc79c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.982+0000 7fc601000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.982+0000 7fc601000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38852/0 (socket says 172.21.15.7:38852) 2024-06-13T07:09:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.983+0000 7fc601000640 1 -- 172.21.15.7:0/698944158 learned_addr learned my addr 172.21.15.7:0/698944158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.983+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1b31d0 con 0x55d5bc7925f0 2024-06-13T07:09:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.983+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1b0a40 con 0x55d5bc791c00 2024-06-13T07:09:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.983+0000 7fc602400640 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 0x55d5bc79c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1ae890 con 0x55d5bc797760 2024-06-13T07:09:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.983+0000 7fc601a00640 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc790e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc602400640 1 -- 172.21.15.7:0/698944158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 msgr2=0x55d5bc790e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc602400640 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc790e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc602400640 1 -- 172.21.15.7:0/698944158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 msgr2=0x55d5bc7a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc602400640 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.984+0000 7fc602400640 1 -- 172.21.15.7:0/698944158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bc79a5f0 con 0x55d5bc797760 2024-06-13T07:09:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.986+0000 7fc602400640 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 0x55d5bc79c270 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5f401d5b0 tx=0x7fc5f4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30bc3f716ae22099 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.987+0000 7fc5ee000640 1 -- 172.21.15.7:0/698944158 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f401b410 con 0x55d5bc797760 2024-06-13T07:09:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.987+0000 7fc5ee000640 1 -- 172.21.15.7:0/698944158 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f401ba90 con 0x55d5bc797760 2024-06-13T07:09:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.987+0000 7fc5ee000640 1 -- 172.21.15.7:0/698944158 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4027b00 con 0x55d5bc797760 2024-06-13T07:09:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.988+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 msgr2=0x55d5bc79c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.988+0000 7fc6071e8740 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 0x55d5bc79c270 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5f401d5b0 tx=0x7fc5f4014000 comp rx=0 tx=0).stop 2024-06-13T07:09:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.989+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 shutdown_connections 2024-06-13T07:09:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.989+0000 7fc6071e8740 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc797760 0x55d5bc79c270 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.989+0000 7fc6071e8740 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.989+0000 7fc6071e8740 1 --2- 172.21.15.7:0/698944158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc791c00 0x55d5bc790e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.989+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 >> 172.21.15.7:0/698944158 conn(0x55d5bc759fb0 msgr2=0x55d5bc79bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.993+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 shutdown_connections 2024-06-13T07:09:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.994+0000 7fc6071e8740 1 -- 172.21.15.7:0/698944158 wait complete. 2024-06-13T07:09:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.994+0000 7fc6071e8740 1 -- start start 2024-06-13T07:09:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.995+0000 7fc6071e8740 1 Processor -- start 2024-06-13T07:09:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1aef00 con 0x55d5bc797760 2024-06-13T07:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1b0370 con 0x55d5bc7925f0 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1af5d0 con 0x55d5bc791c00 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:47.999+0000 7fc6071e8740 10 client.0 _ll_register_callbacks cb 0x7fc5e8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.000+0000 7fc601000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.000+0000 7fc601a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.000+0000 7fc601a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:50306/0 (socket says 172.21.15.7:50306) 2024-06-13T07:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.000+0000 7fc601a00640 1 -- 172.21.15.7:0/3033726441 learned_addr learned my addr 172.21.15.7:0/3033726441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.001+0000 7fc602400640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601a00640 1 -- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 msgr2=0x55d5bc7a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601a00640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601a00640 1 -- 172.21.15.7:0/3033726441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 msgr2=0x55d5bc7a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601a00640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601a00640 1 -- 172.21.15.7:0/3033726441 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bc7a2450 con 0x55d5bc791c00 2024-06-13T07:09:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.006+0000 7fc601000640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:09:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc601a00640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80c3ea0 tx=0x7fc5e80092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef11007b4e2d1d8f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc5e3400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:09:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80d4040 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc5e80bf170 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f401ba90 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.007+0000 7fc602400640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.008+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5e80bac20 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.008+0000 7fc6071e8740 10 client.27259 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.008+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d5bc7a3760 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.008+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f401ba90 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.008+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7fc5e80db0b0 con 0x55d5bc791c00 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 10 client.27259 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 10 client.27259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:09:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 20 client.27259 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc8130a0 0x55d5bc8154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 4 client.27259 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:09:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.009+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5bc7aa670 con 0x55d5bc8130a0 2024-06-13T07:09:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.013+0000 7fc601000640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc8130a0 0x55d5bc8154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.013+0000 7fc601000640 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc8130a0 0x55d5bc8154c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231fd62971cb8a2d server_cookie=b0381d3f27327328 in_seq=0 out_seq=0 2024-06-13T07:09:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.013+0000 7fc5e3400640 10 client.27259 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc5e3400640 1 -- 172.21.15.7:0/3033726441 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d5bc7aa670 con 0x55d5bc8130a0 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc5e3400640 10 client.27259 handle_command_reply: tid=0 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 client.27259 shutdown 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 msgr2=0x55d5bc7a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80c3ea0 tx=0x7fc5e80092d0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 shutdown_connections 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d5bc8130a0 0x55d5bc8154c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5bc797760 0x55d5bc7a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d5bc7925f0 0x55d5bc7a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc5e3400640 0 client.27259 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 --2- 172.21.15.7:0/3033726441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5bc791c00 0x55d5bc7a0910 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.043+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 >> 172.21.15.7:0/3033726441 conn(0x55d5bc759fb0 msgr2=0x55d5bc790c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.044+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 shutdown_connections 2024-06-13T07:09:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.045+0000 7fc6071e8740 1 -- 172.21.15.7:0/3033726441 wait complete. 2024-06-13T07:09:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.045+0000 7fc6071e8740 20 client.27259 trim_cache size 0 max 16384 2024-06-13T07:09:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:09:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:09:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:09:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 msgr2=0x7fc5e40a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 0x7fc5e40a4ff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55d5bc792e50 tx=0x7fc5f4009c50 comp rx=0 tx=0).stop 2024-06-13T07:09:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 msgr2=0x7fc5fc19e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8031440 tx=0x7fc5e8032ce0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 shutdown_connections 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc5e40a2b30 0x7fc5e40a4ff0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5fc1057d0 0x7fc5fc1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5fc104e10 0x7fc5fc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 --2- 172.21.15.7:0/3080690557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5fc103c20 0x7fc5fc19e290 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 >> 172.21.15.7:0/3080690557 conn(0x7fc5fc0ff190 msgr2=0x7fc5fc0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 shutdown_connections 2024-06-13T07:09:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:09:48.047+0000 7fc604400640 1 -- 172.21.15.7:0/3080690557 wait complete. 2024-06-13T07:09:48.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:09:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:47 smithi189 ceph-mon[36616]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-06-13T07:09:48.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:47 smithi175 ceph-mon[36886]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-06-13T07:09:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:47 smithi007 ceph-mon[32077]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-06-13T07:09:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:09:50.165 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:49 smithi007 ceph-mon[32077]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-06-13T07:09:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:49 smithi189 ceph-mon[36616]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-06-13T07:09:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:49 smithi175 ceph-mon[36886]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 522 op/s 2024-06-13T07:09:52.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:51 smithi189 ceph-mon[36616]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-06-13T07:09:52.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:51 smithi175 ceph-mon[36886]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-06-13T07:09:52.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:51 smithi007 ceph-mon[32077]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-06-13T07:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 830.0 s, 446.7 tps, lat 237.820 ms stddev 300.482 2024-06-13T07:09:53.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:53 smithi189 ceph-mon[36616]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 528 op/s 2024-06-13T07:09:53.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:53 smithi175 ceph-mon[36886]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 528 op/s 2024-06-13T07:09:53.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:53 smithi007 ceph-mon[32077]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 528 op/s 2024-06-13T07:09:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.774+0000 7f529e200700 1 -- 172.21.15.7:0/4076220595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5298075180 msgr2=0x7f529810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.774+0000 7f529e200700 1 --2- 172.21.15.7:0/4076220595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5298075180 0x7f529810e720 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f5284009ca0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 -- 172.21.15.7:0/4076220595 shutdown_connections 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 --2- 172.21.15.7:0/4076220595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f529810ec60 0x7f5298111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 --2- 172.21.15.7:0/4076220595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298074860 0x7f5298074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 --2- 172.21.15.7:0/4076220595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5298075180 0x7f529810e720 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 -- 172.21.15.7:0/4076220595 >> 172.21.15.7:0/4076220595 conn(0x7f52980fdf10 msgr2=0x7f5298100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 -- 172.21.15.7:0/4076220595 shutdown_connections 2024-06-13T07:09:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.775+0000 7f529e200700 1 -- 172.21.15.7:0/4076220595 wait complete. 2024-06-13T07:09:53.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.777+0000 7f529e200700 1 Processor -- start 2024-06-13T07:09:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.777+0000 7f529e200700 1 -- start start 2024-06-13T07:09:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.778+0000 7f529e200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 0x7f5298073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.779+0000 7f529e200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298075180 0x7f52980740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.779+0000 7f529e200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 0x7f529806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.779+0000 7f5297e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298075180 0x7f52980740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529e200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52981149c0 con 0x7f5298075180 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529d800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 0x7f529806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529d800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 0x7f529806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529d800700 1 -- 172.21.15.7:0/3771387336 learned_addr learned my addr 172.21.15.7:0/3771387336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5298114840 con 0x7f529810ec60 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.780+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5298114b40 con 0x7f5298074860 2024-06-13T07:09:53.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.781+0000 7f529ce00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 0x7f5298073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 -- 172.21.15.7:0/3771387336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 msgr2=0x7f529806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 0x7f529806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 -- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298075180 msgr2=0x7f52980740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298075180 0x7f52980740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 -- 172.21.15.7:0/3771387336 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529806d6e0 con 0x7f5298074860 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.782+0000 7f529ce00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 0x7f5298073b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f528c00efc0 tx=0x7f528c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.783+0000 7f5295600700 1 -- 172.21.15.7:0/3771387336 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0092e0 con 0x7f5298074860 2024-06-13T07:09:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.784+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529806d970 con 0x7f5298074860 2024-06-13T07:09:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.784+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52980718c0 con 0x7f5298074860 2024-06-13T07:09:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.787+0000 7f5295600700 1 -- 172.21.15.7:0/3771387336 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528c003a00 con 0x7f5298074860 2024-06-13T07:09:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.787+0000 7f5295600700 1 -- 172.21.15.7:0/3771387336 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c004740 con 0x7f5298074860 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.792+0000 7f52aa1acb80 1 Processor -- start 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.793+0000 7f52aa1acb80 1 -- start start 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.793+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.793+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac23ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.793+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f529d800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f529d800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40318/0 (socket says 172.21.15.7:40318) 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f529d800700 1 -- 172.21.15.7:0/793511171 learned_addr learned my addr 172.21.15.7:0/793511171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55914ab28a10 con 0x55914ac24530 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55914ab6bb00 con 0x55914ac18d60 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55914ab268f0 con 0x55914ac19680 2024-06-13T07:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.794+0000 7f5297e00700 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f529d800700 1 -- 172.21.15.7:0/793511171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 msgr2=0x55914ac23ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f529d800700 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac23ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f529d800700 1 -- 172.21.15.7:0/793511171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 msgr2=0x55914ac26920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f529d800700 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac26920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f529d800700 1 -- 172.21.15.7:0/793511171 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914ab13f90 con 0x55914ac18d60 2024-06-13T07:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.795+0000 7f5297e00700 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac26920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.799+0000 7f529d800700 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f529002ebf0 tx=0x7f529002ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c40316e23f093f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.802+0000 7f5295600700 1 -- 172.21.15.7:0/3771387336 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f528c01b030 con 0x7f5298074860 2024-06-13T07:09:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.803+0000 7f528a000700 1 -- 172.21.15.7:0/793511171 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290030dd0 con 0x55914ac18d60 2024-06-13T07:09:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.803+0000 7f528a000700 1 -- 172.21.15.7:0/793511171 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5290004500 con 0x55914ac18d60 2024-06-13T07:09:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.803+0000 7f528a000700 1 -- 172.21.15.7:0/793511171 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900326a0 con 0x55914ac18d60 2024-06-13T07:09:53.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.805+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 msgr2=0x55914ac19140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.805+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f529002ebf0 tx=0x7f529002ef00 comp rx=0 tx=0).stop 2024-06-13T07:09:53.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 shutdown_connections 2024-06-13T07:09:53.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac26920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac19140 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/793511171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac23ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 >> 172.21.15.7:0/793511171 conn(0x55914ab46170 msgr2=0x55914ac15280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 shutdown_connections 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.807+0000 7f52aa1acb80 1 -- 172.21.15.7:0/793511171 wait complete. 2024-06-13T07:09:53.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.808+0000 7f5295600700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 0x7f52800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.808+0000 7f52aa1acb80 1 -- start start 2024-06-13T07:09:53.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.808+0000 7f52aa1acb80 1 Processor -- start 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.809+0000 7f5295600700 1 -- 172.21.15.7:0/3771387336 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f528c0cecb0 con 0x7f5298074860 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac21020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55914ab07f60 con 0x55914ac24530 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55914ab67ac0 con 0x55914ac18d60 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.810+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55914ac26fc0 con 0x55914ac19680 2024-06-13T07:09:53.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.811+0000 7f52aa1acb80 10 client.0 _ll_register_callbacks cb 0x55914ab01380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:53.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.811+0000 7f5297e00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 0x7f52800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.812+0000 7f529ce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.812+0000 7f529ce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40326/0 (socket says 172.21.15.7:40326) 2024-06-13T07:09:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.812+0000 7f529ce00700 1 -- 172.21.15.7:0/781320981 learned_addr learned my addr 172.21.15.7:0/781320981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:53.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.812+0000 7f529d800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.813+0000 7f5297e00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac21020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 -- 172.21.15.7:0/781320981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 msgr2=0x55914ac21020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac21020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 -- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 msgr2=0x55914ac21560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac21560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 -- 172.21.15.7:0/781320981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914aaba8d0 con 0x55914ac18d60 2024-06-13T07:09:53.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f529ce00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f528c097bd0 tx=0x7f528c09ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb05a093d0d54ca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.814+0000 7f527ea00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:09:53.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.815+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0b4060 con 0x55914ac18d60 2024-06-13T07:09:53.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.815+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f528c09b460 con 0x55914ac18d60 2024-06-13T07:09:53.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.815+0000 7f5297e00700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 0x7f52800a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f529806e450 tx=0x7f528400b690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.815+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5290004500 con 0x55914ac18d60 2024-06-13T07:09:53.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.816+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f528c09f370 con 0x55914ac18d60 2024-06-13T07:09:53.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.816+0000 7f52aa1acb80 10 client.27383 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:53.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.816+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55914ac31820 con 0x55914ac18d60 2024-06-13T07:09:53.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.816+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5290004500 con 0x55914ac18d60 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f528c0a3950 con 0x55914ac18d60 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 10 client.27383 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 10 client.27383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 20 client.27383 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac33e60 0x55914ac36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 4 client.27383 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.818+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914ab79000 con 0x55914ac33e60 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.820+0000 7f5297e00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac33e60 0x55914ac36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.821+0000 7f5297e00700 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac33e60 0x55914ac36250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e84b38a7a3f198 server_cookie=dc2c1502b3820e33 in_seq=0 out_seq=0 2024-06-13T07:09:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.823+0000 7f527ea00700 10 client.27383 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:09:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.823+0000 7f527ea00700 1 -- 172.21.15.7:0/781320981 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55914ab79000 con 0x55914ac33e60 2024-06-13T07:09:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.823+0000 7f527ea00700 10 client.27383 handle_command_reply: tid=0 2024-06-13T07:09:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.823+0000 7f52aa1acb80 1 client.27383 shutdown 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 msgr2=0x55914ac23d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f528c097bd0 tx=0x7f528c09ac50 comp rx=0 tx=0).stop 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 shutdown_connections 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac24530 0x55914ac21560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac33e60 0x55914ac36250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac23d20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/781320981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac19680 0x55914ac21020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 >> 172.21.15.7:0/781320981 conn(0x55914ab46170 msgr2=0x55914ac14e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f527ea00700 0 client.27383 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 shutdown_connections 2024-06-13T07:09:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 1 -- 172.21.15.7:0/781320981 wait complete. 2024-06-13T07:09:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.824+0000 7f52aa1acb80 20 client.27383 trim_cache size 0 max 16384 2024-06-13T07:09:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.871+0000 7f52aa1acb80 1 Processor -- start 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 -- start start 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac18d60 0x55914ac30130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5278011270 con 0x55914ac18d60 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f527800eaa0 con 0x55914ac19680 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.872+0000 7f52aa1acb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f527800c8f0 con 0x55914ac24530 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40340/0 (socket says 172.21.15.7:40340) 2024-06-13T07:09:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 -- 172.21.15.7:0/2308617495 learned_addr learned my addr 172.21.15.7:0/2308617495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f529d800700 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 -- 172.21.15.7:0/2308617495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 msgr2=0x55914ac23860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac23860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 -- 172.21.15.7:0/2308617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac18d60 msgr2=0x55914ac30130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac18d60 0x55914ac30130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.873+0000 7f5297e00700 1 -- 172.21.15.7:0/2308617495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914ac13910 con 0x55914ac19680 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.874+0000 7f5297e00700 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f528402c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9a8b5e2e007585 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.874+0000 7f5289600700 1 -- 172.21.15.7:0/2308617495 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284039070 con 0x55914ac19680 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.874+0000 7f5289600700 1 -- 172.21.15.7:0/2308617495 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284005730 con 0x55914ac19680 2024-06-13T07:09:53.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.874+0000 7f5289600700 1 -- 172.21.15.7:0/2308617495 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284033c00 con 0x55914ac19680 2024-06-13T07:09:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.875+0000 7f52aa1acb80 1 -- 172.21.15.7:0/2308617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 msgr2=0x55914ac36250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.875+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5284009990 tx=0x7f528402c000 comp rx=0 tx=0).stop 2024-06-13T07:09:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.876+0000 7f52aa1acb80 1 -- 172.21.15.7:0/2308617495 shutdown_connections 2024-06-13T07:09:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.876+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac23860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.876+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac18d60 0x55914ac30130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.876+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/2308617495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac19680 0x55914ac36250 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.876+0000 7f52aa1acb80 1 -- 172.21.15.7:0/2308617495 >> 172.21.15.7:0/2308617495 conn(0x55914ab46170 msgr2=0x55914ac1c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.877+0000 7f52aa1acb80 1 -- 172.21.15.7:0/2308617495 shutdown_connections 2024-06-13T07:09:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.877+0000 7f52aa1acb80 1 -- 172.21.15.7:0/2308617495 wait complete. 2024-06-13T07:09:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.877+0000 7f52aa1acb80 1 -- start start 2024-06-13T07:09:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.881+0000 7f52aa1acb80 1 Processor -- start 2024-06-13T07:09:53.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.882+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.883+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 0x55914ac2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.883+0000 7f52aa1acb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.883+0000 7f529ce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.883+0000 7f529ce00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40348/0 (socket says 172.21.15.7:40348) 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.883+0000 7f529ce00700 1 -- 172.21.15.7:0/1767179263 learned_addr learned my addr 172.21.15.7:0/1767179263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:09:53.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.884+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f527800cf60 con 0x55914ac19680 2024-06-13T07:09:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.884+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f527800e3d0 con 0x55914ac18d60 2024-06-13T07:09:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.884+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f527800d630 con 0x55914ac24530 2024-06-13T07:09:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f52aa1acb80 10 client.0 _ll_register_callbacks cb 0x55914ab01380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:09:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f5297e00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 0x55914ac2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f529ce00700 1 -- 172.21.15.7:0/1767179263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 msgr2=0x55914ac1d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f529ce00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f529ce00700 1 -- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 msgr2=0x55914ac2caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f529ce00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 0x55914ac2caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f529ce00700 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914ac1e190 con 0x55914ac18d60 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.885+0000 7f5297e00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 0x55914ac2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.886+0000 7f529ce00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c7a0 tx=0x7f528c09b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e7ef92daa3e100 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.886+0000 7f527ea00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.886+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c07d020 con 0x55914ac18d60 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.886+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f528c09b460 con 0x55914ac18d60 2024-06-13T07:09:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.886+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5284005730 con 0x55914ac18d60 2024-06-13T07:09:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.887+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f528c09f5b0 con 0x55914ac18d60 2024-06-13T07:09:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.887+0000 7f52aa1acb80 10 client.27389 fetch_fsmap learned FSMap version 40 2024-06-13T07:09:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.887+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55914ac9e610 con 0x55914ac18d60 2024-06-13T07:09:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284005730 con 0x55914ac18d60 2024-06-13T07:09:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f528c00cad0 con 0x55914ac18d60 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 10 client.27389 fetch_fsmap finished waiting for FSMap version 40 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 10 client.27389 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 20 client.27389 populate_metadata read hostname 'smithi007' 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac9ed00 0x55914aca1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 4 client.27389 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55914ab73580 con 0x55914ac9ed00 2024-06-13T07:09:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.889+0000 7f5297e00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac9ed00 0x55914aca1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:09:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.892+0000 7f5297e00700 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac9ed00 0x55914aca1120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=690305b066caaa7b server_cookie=2d4055c2807c0370 in_seq=0 out_seq=0 2024-06-13T07:09:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.892+0000 7f527ea00700 10 client.27389 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:09:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.893+0000 7f527ea00700 1 -- 172.21.15.7:0/1767179263 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55914ab73580 con 0x55914ac9ed00 2024-06-13T07:09:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.893+0000 7f527ea00700 10 client.27389 handle_command_reply: tid=0 2024-06-13T07:09:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.893+0000 7f52aa1acb80 1 client.27389 shutdown 2024-06-13T07:09:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 msgr2=0x55914ac2c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c7a0 tx=0x7f528c09b2c0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 shutdown_connections 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914ac24530 0x55914ac1d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55914ac9ed00 0x55914aca1120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55914ac18d60 0x55914ac2c560 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 --2- 172.21.15.7:0/1767179263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55914ac19680 0x55914ac2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 >> 172.21.15.7:0/1767179263 conn(0x55914ab46170 msgr2=0x55914ac1b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f527ea00700 0 client.27389 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:09:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 shutdown_connections 2024-06-13T07:09:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.894+0000 7f52aa1acb80 1 -- 172.21.15.7:0/1767179263 wait complete. 2024-06-13T07:09:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.895+0000 7f52aa1acb80 20 client.27389 trim_cache size 0 max 16384 2024-06-13T07:09:53.895 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:09:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 msgr2=0x7f52800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 0x7f52800a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f529806e450 tx=0x7f528400b690 comp rx=0 tx=0).stop 2024-06-13T07:09:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 msgr2=0x7f5298073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:09:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 0x7f5298073b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f528c00efc0 tx=0x7f528c00c3a0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 shutdown_connections 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52800a2c30 0x7f52800a50f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f529810ec60 0x7f529806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5298074860 0x7f5298073b70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 --2- 172.21.15.7:0/3771387336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5298075180 0x7f52980740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 >> 172.21.15.7:0/3771387336 conn(0x7f52980fdf10 msgr2=0x7f5298110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.897+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 shutdown_connections 2024-06-13T07:09:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:09:53.898+0000 7f529e200700 1 -- 172.21.15.7:0/3771387336 wait complete. 2024-06-13T07:09:55.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:09:55.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:55 smithi189 ceph-mon[36616]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-06-13T07:09:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:55 smithi175 ceph-mon[36886]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-06-13T07:09:55.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:55 smithi007 ceph-mon[32077]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-06-13T07:09:56.133 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:09:57.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:57 smithi007 ceph-mon[32077]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-06-13T07:09:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:57 smithi189 ceph-mon[36616]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-06-13T07:09:57.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:57 smithi175 ceph-mon[36886]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-06-13T07:09:57.881 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 835.0 s, 361.6 tps, lat 272.991 ms stddev 298.395 2024-06-13T07:09:59.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:09:59 smithi189 ceph-mon[36616]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 7.5 MiB/s wr, 701 op/s 2024-06-13T07:09:59.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:09:59 smithi175 ceph-mon[36886]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 7.5 MiB/s wr, 701 op/s 2024-06-13T07:09:59.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:09:59 smithi007 ceph-mon[32077]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 7.5 MiB/s wr, 701 op/s 2024-06-13T07:10:00.539 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:00 smithi007 ceph-mon[32077]: overall HEALTH_OK 2024-06-13T07:10:00.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:00 smithi189 ceph-mon[36616]: overall HEALTH_OK 2024-06-13T07:10:00.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:00 smithi175 ceph-mon[36886]: overall HEALTH_OK 2024-06-13T07:10:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 -- 172.21.15.7:0/3633557879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80110220 msgr2=0x7fcc80112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 --2- 172.21.15.7:0/3633557879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80110220 0x7fcc80112610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009ca0 tx=0x7fcc7802a8e0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 -- 172.21.15.7:0/3633557879 shutdown_connections 2024-06-13T07:10:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 --2- 172.21.15.7:0/3633557879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80110220 0x7fcc80112610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 --2- 172.21.15.7:0/3633557879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc80109300 0x7fcc8010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 --2- 172.21.15.7:0/3633557879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc801089a0 0x7fcc80108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.260+0000 7fcc88200640 1 -- 172.21.15.7:0/3633557879 >> 172.21.15.7:0/3633557879 conn(0x7fcc8006ea10 msgr2=0x7fcc80070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.262+0000 7fcc88200640 1 -- 172.21.15.7:0/3633557879 shutdown_connections 2024-06-13T07:10:01.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.262+0000 7fcc88200640 1 -- 172.21.15.7:0/3633557879 wait complete. 2024-06-13T07:10:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.263+0000 7fcc88200640 1 Processor -- start 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 -- start start 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 0x7fcc8010f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 0x7fcc8010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc80110220 0x7fcc8010c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc80115f20 con 0x7fcc80110220 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc80115da0 con 0x7fcc80109300 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc88200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801160a0 con 0x7fcc801089a0 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.264+0000 7fcc84e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 0x7fcc8010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc84e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 0x7fcc8010fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43218/0 (socket says 172.21.15.7:43218) 2024-06-13T07:10:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc84e00640 1 -- 172.21.15.7:0/3359605053 learned_addr learned my addr 172.21.15.7:0/3359605053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 0x7fcc8010f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 -- 172.21.15.7:0/3359605053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 msgr2=0x7fcc8010fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 0x7fcc8010fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 -- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc80110220 msgr2=0x7fcc8010c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc80110220 0x7fcc8010c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.265+0000 7fcc85800640 1 -- 172.21.15.7:0/3359605053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8010ca60 con 0x7fcc801089a0 2024-06-13T07:10:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.266+0000 7fcc85800640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 0x7fcc8010f630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c970 tx=0x7fcc7400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.266+0000 7fcc7e000640 1 -- 172.21.15.7:0/3359605053 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74007bb0 con 0x7fcc801089a0 2024-06-13T07:10:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.266+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc801a2280 con 0x7fcc801089a0 2024-06-13T07:10:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.267+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc801a28a0 con 0x7fcc801089a0 2024-06-13T07:10:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.267+0000 7fcc7e000640 1 -- 172.21.15.7:0/3359605053 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc74007d50 con 0x7fcc801089a0 2024-06-13T07:10:01.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.267+0000 7fcc7e000640 1 -- 172.21.15.7:0/3359605053 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74007bb0 con 0x7fcc801089a0 2024-06-13T07:10:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.273+0000 7fcc8b062740 1 Processor -- start 2024-06-13T07:10:01.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.273+0000 7fcc8b062740 1 -- start start 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbf5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbbf55f0 0x556cdbbfa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556cdbb58420 con 0x556cdbbf4c00 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cdbaffc50 con 0x556cdbbf55f0 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.275+0000 7fcc8b062740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556cdbb543e0 con 0x556cdbbfa760 2024-06-13T07:10:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.276+0000 7fcc84e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.276+0000 7fcc84e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55302/0 (socket says 172.21.15.7:55302) 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.276+0000 7fcc84e00640 1 -- 172.21.15.7:0/1241344621 learned_addr learned my addr 172.21.15.7:0/1241344621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc86200640 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbf5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc84e00640 1 -- 172.21.15.7:0/1241344621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbbf55f0 msgr2=0x556cdbbfa0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc84e00640 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbbf55f0 0x556cdbbfa0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc84e00640 1 -- 172.21.15.7:0/1241344621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 msgr2=0x556cdbbf5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc84e00640 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbf5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.277+0000 7fcc84e00640 1 -- 172.21.15.7:0/1241344621 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdb86c210 con 0x556cdbbfa760 2024-06-13T07:10:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.278+0000 7fcc84e00640 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009ae0 tx=0x7fcc6c00e1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e304e7f2221f9cf4 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.278+0000 7fcc7e000640 1 -- 172.21.15.7:0/3359605053 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcc7400f450 con 0x7fcc801089a0 2024-06-13T07:10:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.278+0000 7fcc72a00640 1 -- 172.21.15.7:0/1241344621 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x556cdbbfa760 2024-06-13T07:10:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.278+0000 7fcc72a00640 1 -- 172.21.15.7:0/1241344621 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c00ebb0 con 0x556cdbbfa760 2024-06-13T07:10:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.278+0000 7fcc72a00640 1 -- 172.21.15.7:0/1241344621 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c012630 con 0x556cdbbfa760 2024-06-13T07:10:01.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 -- 172.21.15.7:0/1241344621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 msgr2=0x556cdbbfcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009ae0 tx=0x7fcc6c00e1d0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 -- 172.21.15.7:0/1241344621 shutdown_connections 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbfa760 0x556cdbbfcbf0 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009ae0 tx=0x7fcc6c00e1d0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbbf55f0 0x556cdbbfa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 --2- 172.21.15.7:0/1241344621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbf5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.279+0000 7fcc8b062740 1 -- 172.21.15.7:0/1241344621 >> 172.21.15.7:0/1241344621 conn(0x556cdbbbcfb0 msgr2=0x556cdbbf0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.283+0000 7fcc7e000640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 0x7fcc680a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.284+0000 7fcc8b062740 1 -- 172.21.15.7:0/1241344621 shutdown_connections 2024-06-13T07:10:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.285+0000 7fcc84e00640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 0x7fcc680a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.286+0000 7fcc8b062740 1 -- 172.21.15.7:0/1241344621 wait complete. 2024-06-13T07:10:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.286+0000 7fcc8b062740 1 -- start start 2024-06-13T07:10:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.286+0000 7fcc7e000640 1 -- 172.21.15.7:0/3359605053 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcc740cec20 con 0x7fcc801089a0 2024-06-13T07:10:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.287+0000 7fcc8b062740 1 Processor -- start 2024-06-13T07:10:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.288+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.288+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbbff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.288+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc06ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc85800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc85800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbff150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60230/0 (socket says 172.21.15.7:60230) 2024-06-13T07:10:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc85800640 1 -- 172.21.15.7:0/4183050973 learned_addr learned my addr 172.21.15.7:0/4183050973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556cdbbfd710 con 0x556cdbbf4c00 2024-06-13T07:10:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556cdbb130b0 con 0x556cdbc046c0 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556cdbbfd890 con 0x556cdbbf55f0 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc06ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc8b062740 10 client.0 _ll_register_callbacks cb 0x556cdbafef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 -- 172.21.15.7:0/4183050973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 msgr2=0x556cdbbff690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbbff690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 -- 172.21.15.7:0/4183050973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 msgr2=0x556cdbbff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.289+0000 7fcc86200640 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdb938620 con 0x556cdbc046c0 2024-06-13T07:10:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc6c00ebb0 con 0x556cdbc046c0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc86200640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc06ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc7802ae00 tx=0x7fcc78031da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2498b1ff3739998c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc67400640 10 client.27398 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc67400640 1 -- 172.21.15.7:0/4183050973 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78003b70 con 0x556cdbc046c0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc67400640 1 -- 172.21.15.7:0/4183050973 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcc78003d10 con 0x556cdbc046c0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.290+0000 7fcc67400640 1 -- 172.21.15.7:0/4183050973 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcc78004040 con 0x556cdbc046c0 2024-06-13T07:10:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.291+0000 7fcc8b062740 10 client.27398 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.291+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=41}) -- 0x556cdbc09d60 con 0x556cdbc046c0 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.291+0000 7fcc67400640 1 -- 172.21.15.7:0/4183050973 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fcc780041e0 con 0x556cdbc046c0 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.291+0000 7fcc8b062740 10 client.27398 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.291+0000 7fcc8b062740 10 client.27398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.292+0000 7fcc8b062740 20 client.27398 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.292+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc0b380 0x556cdbc0d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.292+0000 7fcc8b062740 4 client.27398 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.292+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cdbb65b50 con 0x556cdbc0b380 2024-06-13T07:10:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.292+0000 7fcc85800640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc0b380 0x556cdbc0d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.293+0000 7fcc85800640 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc0b380 0x556cdbc0d810 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daee5b5041a407d4 server_cookie=6345c5eb66742500 in_seq=0 out_seq=0 2024-06-13T07:10:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.293+0000 7fcc67400640 10 client.27398 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.293+0000 7fcc84e00640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 0x7fcc680a5400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e6d0 tx=0x7fcc6c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.293+0000 7fcc67400640 1 -- 172.21.15.7:0/4183050973 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556cdbb65b50 con 0x556cdbc0b380 2024-06-13T07:10:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.293+0000 7fcc67400640 10 client.27398 handle_command_reply: tid=0 2024-06-13T07:10:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.295+0000 7fcc8b062740 1 client.27398 shutdown 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.298+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 msgr2=0x556cdbc06ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.298+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc06ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc7802ae00 tx=0x7fcc78031da0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.298+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 shutdown_connections 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.298+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc0b380 0x556cdbc0d810 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.298+0000 7fcc67400640 0 client.27398 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.299+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc06ab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.299+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbbff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.299+0000 7fcc8b062740 1 --2- 172.21.15.7:0/4183050973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf4c00 0x556cdbbff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.299+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 >> 172.21.15.7:0/4183050973 conn(0x556cdbbbcfb0 msgr2=0x556cdbbef840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.300+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 shutdown_connections 2024-06-13T07:10:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.301+0000 7fcc8b062740 1 -- 172.21.15.7:0/4183050973 wait complete. 2024-06-13T07:10:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.301+0000 7fcc8b062740 20 client.27398 trim_cache size 0 max 16384 2024-06-13T07:10:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.364+0000 7fcc8b062740 1 Processor -- start 2024-06-13T07:10:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.364+0000 7fcc8b062740 1 -- start start 2024-06-13T07:10:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf55f0 0x556cdbbf06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc85800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc130c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801c4510 con 0x556cdbbf55f0 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801c1d50 con 0x556cdbc046c0 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.365+0000 7fcc8b062740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010cd70 con 0x556cdbbf4c00 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.366+0000 7fcc85800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55316/0 (socket says 172.21.15.7:55316) 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.366+0000 7fcc85800640 1 -- 172.21.15.7:0/358779366 learned_addr learned my addr 172.21.15.7:0/358779366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.366+0000 7fcc86200640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc84e00640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf55f0 0x556cdbbf06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc85800640 1 -- 172.21.15.7:0/358779366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 msgr2=0x556cdbc130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc85800640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc85800640 1 -- 172.21.15.7:0/358779366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf55f0 msgr2=0x556cdbbf06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc85800640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf55f0 0x556cdbbf06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.367+0000 7fcc85800640 1 -- 172.21.15.7:0/358779366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdbc06e20 con 0x556cdbbf4c00 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.368+0000 7fcc86200640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.368+0000 7fcc85800640 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409ade0 tx=0x7fcc740cc250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad0053932ce4c47 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.368+0000 7fcc72000640 1 -- 172.21.15.7:0/358779366 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740b4070 con 0x556cdbbf4c00 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.368+0000 7fcc72000640 1 -- 172.21.15.7:0/358779366 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc74099180 con 0x556cdbbf4c00 2024-06-13T07:10:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.368+0000 7fcc72000640 1 -- 172.21.15.7:0/358779366 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7401d040 con 0x556cdbbf4c00 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.369+0000 7fcc8b062740 1 -- 172.21.15.7:0/358779366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 msgr2=0x556cdbbf0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.369+0000 7fcc8b062740 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409ade0 tx=0x7fcc740cc250 comp rx=0 tx=0).stop 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 -- 172.21.15.7:0/358779366 shutdown_connections 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc046c0 0x556cdbc130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbbf55f0 0x556cdbbf06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 --2- 172.21.15.7:0/358779366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf4c00 0x556cdbbf0190 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 -- 172.21.15.7:0/358779366 >> 172.21.15.7:0/358779366 conn(0x556cdbbbcfb0 msgr2=0x556cdbbff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.370+0000 7fcc8b062740 1 -- 172.21.15.7:0/358779366 shutdown_connections 2024-06-13T07:10:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.371+0000 7fcc8b062740 1 -- 172.21.15.7:0/358779366 wait complete. 2024-06-13T07:10:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.372+0000 7fcc8b062740 1 -- start start 2024-06-13T07:10:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.372+0000 7fcc8b062740 1 Processor -- start 2024-06-13T07:10:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbc046c0 0x556cdbc1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc0b380 0x556cdbc1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801aa8f0 con 0x556cdbc046c0 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801c1680 con 0x556cdbc0b380 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.374+0000 7fcc8b062740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc801c08e0 con 0x556cdbbf55f0 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.375+0000 7fcc85800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.375+0000 7fcc8b062740 10 client.0 _ll_register_callbacks cb 0x7fcc78009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.375+0000 7fcc85800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55328/0 (socket says 172.21.15.7:55328) 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.375+0000 7fcc85800640 1 -- 172.21.15.7:0/3576275693 learned_addr learned my addr 172.21.15.7:0/3576275693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.376+0000 7fcc84e00640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbc046c0 0x556cdbc1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc86200640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc0b380 0x556cdbc1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc85800640 1 -- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc0b380 msgr2=0x556cdbc1e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc85800640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc0b380 0x556cdbc1e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc85800640 1 -- 172.21.15.7:0/3576275693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbc046c0 msgr2=0x556cdbc1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc85800640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbc046c0 0x556cdbc1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.377+0000 7fcc85800640 1 -- 172.21.15.7:0/3576275693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdbc1e830 con 0x556cdbbf55f0 2024-06-13T07:10:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.379+0000 7fcc67400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.379+0000 7fcc85800640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc74097ab0 tx=0x7fcc7409b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14326a08a2d55653 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.379+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7401d040 con 0x556cdbbf55f0 2024-06-13T07:10:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.379+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fcc74097750 con 0x556cdbbf55f0 2024-06-13T07:10:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.379+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc74099180 con 0x556cdbbf55f0 2024-06-13T07:10:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.380+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcc740b4040 con 0x556cdbbf55f0 2024-06-13T07:10:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.381+0000 7fcc8b062740 10 client.27280 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.381+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556cdbc75f10 con 0x556cdbbf55f0 2024-06-13T07:10:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.381+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc74099180 con 0x556cdbbf55f0 2024-06-13T07:10:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7fcc74099fa0 con 0x556cdbbf55f0 2024-06-13T07:10:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 10 client.27280 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 10 client.27280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 20 client.27280 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc76740 0x556cdbc78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 4 client.27280 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.384+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cdbb5fea0 con 0x556cdbc76740 2024-06-13T07:10:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.385+0000 7fcc84e00640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc76740 0x556cdbc78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.385+0000 7fcc67400640 10 client.27280 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.385+0000 7fcc84e00640 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc76740 0x556cdbc78b30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce03754d5245318 server_cookie=9ee9a08e7cb3c33d in_seq=0 out_seq=0 2024-06-13T07:10:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.407+0000 7f73f6e00700 1 -- 172.21.15.7:0/4126643456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102540 msgr2=0x7f73f01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.407+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4126643456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102540 0x7f73f01029a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009990 tx=0x7f73d8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.408+0000 7f73f6e00700 1 -- 172.21.15.7:0/4126643456 shutdown_connections 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.408+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4126643456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f0102ee0 0x7f73f010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.409+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4126643456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0107450 0x7f73f0107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.409+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4126643456 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102540 0x7f73f01029a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.409+0000 7f73f6e00700 1 -- 172.21.15.7:0/4126643456 >> 172.21.15.7:0/4126643456 conn(0x7f73f00795f0 msgr2=0x7f73f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.409+0000 7f73f6e00700 1 -- 172.21.15.7:0/4126643456 shutdown_connections 2024-06-13T07:10:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.409+0000 7f73f6e00700 1 -- 172.21.15.7:0/4126643456 wait complete. 2024-06-13T07:10:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.411+0000 7f73f6e00700 1 Processor -- start 2024-06-13T07:10:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.411+0000 7f73f6e00700 1 -- start start 2024-06-13T07:10:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.413+0000 7f73f6e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.413+0000 7f73f6e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102ee0 0x7f73f0118110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.413+0000 7f73efe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.413+0000 7f73efe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55346/0 (socket says 172.21.15.7:55346) 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73f6e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f0107450 0x7f73f0111d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73f6e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01106d0 con 0x7f73f0107450 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73f6e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0110550 con 0x7f73f0102ee0 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73f6e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0110850 con 0x7f73f0102540 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73efe00700 1 -- 172.21.15.7:0/4054641247 learned_addr learned my addr 172.21.15.7:0/4054641247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.414+0000 7f73e7400700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102ee0 0x7f73f0118110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.415+0000 7f73efe00700 1 -- 172.21.15.7:0/4054641247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102ee0 msgr2=0x7f73f0118110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.415+0000 7f73efe00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102ee0 0x7f73f0118110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.415+0000 7f73efe00700 1 -- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f0107450 msgr2=0x7f73f0111d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.415+0000 7f73efe00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f0107450 0x7f73f0111d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.416+0000 7f73efe00700 1 -- 172.21.15.7:0/4054641247 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f0112620 con 0x7f73f0102540 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.416+0000 7f73efe00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f73e000d980 tx=0x7f73e000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.416+0000 7f73ed600700 1 -- 172.21.15.7:0/4054641247 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0017030 con 0x7f73f0102540 2024-06-13T07:10:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.417+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f0112910 con 0x7f73f0102540 2024-06-13T07:10:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.417+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f0072100 con 0x7f73f0102540 2024-06-13T07:10:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.417+0000 7f73ed600700 1 -- 172.21.15.7:0/4054641247 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e0004500 con 0x7f73f0102540 2024-06-13T07:10:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.417+0000 7f73ed600700 1 -- 172.21.15.7:0/4054641247 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0003b20 con 0x7f73f0102540 2024-06-13T07:10:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.425+0000 7f74016a8b80 1 Processor -- start 2024-06-13T07:10:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.426+0000 7f74016a8b80 1 -- start start 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.426+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d7208f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.427+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d7202d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.427+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.427+0000 7f74016a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5557d7111a10 con 0x5557d7208b90 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.427+0000 7f74016a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557d7154b00 con 0x5557d72094b0 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.427+0000 7f74016a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557d710f8f0 con 0x5557d7203240 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.428+0000 7f73e7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.428+0000 7f73e7400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55358/0 (socket says 172.21.15.7:55358) 2024-06-13T07:10:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.428+0000 7f73e7400700 1 -- 172.21.15.7:0/540076940 learned_addr learned my addr 172.21.15.7:0/540076940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.429+0000 7f73e7400700 1 -- 172.21.15.7:0/540076940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 msgr2=0x5557d7202d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.429+0000 7f73e7400700 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d7202d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.429+0000 7f73e7400700 1 -- 172.21.15.7:0/540076940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 msgr2=0x5557d7208f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.431+0000 7f73ed600700 1 -- 172.21.15.7:0/4054641247 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f73e001a070 con 0x7f73f0102540 2024-06-13T07:10:01.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.432+0000 7f73f4e00700 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d7208f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.432+0000 7f73e7400700 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d7208f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.432+0000 7f73e7400700 1 -- 172.21.15.7:0/540076940 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d70fcf90 con 0x5557d7203240 2024-06-13T07:10:01.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.432+0000 7f73f4e00700 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d7208f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:01.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.433+0000 7f73e7400700 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f73d8015000 tx=0x7f73d80096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698c99c5f55679bf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.433+0000 7f73e5600700 1 -- 172.21.15.7:0/540076940 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8003780 con 0x5557d7203240 2024-06-13T07:10:01.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.433+0000 7f73e5600700 1 -- 172.21.15.7:0/540076940 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8003e40 con 0x5557d7203240 2024-06-13T07:10:01.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.433+0000 7f73e5600700 1 -- 172.21.15.7:0/540076940 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d800bac0 con 0x5557d7203240 2024-06-13T07:10:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.435+0000 7f73ed600700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 0x7f73d40a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.435+0000 7f73ed600700 1 -- 172.21.15.7:0/4054641247 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f73e00ced20 con 0x7f73f0102540 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.435+0000 7f74016a8b80 1 -- 172.21.15.7:0/540076940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 msgr2=0x5557d7203680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.435+0000 7f74016a8b80 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f73d8015000 tx=0x7f73d80096a0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 -- 172.21.15.7:0/540076940 shutdown_connections 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7203680 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d7208f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 --2- 172.21.15.7:0/540076940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d7202d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 -- 172.21.15.7:0/540076940 >> 172.21.15.7:0/540076940 conn(0x5557d712f170 msgr2=0x5557d71fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f73f4e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 0x7f73d40a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 -- 172.21.15.7:0/540076940 shutdown_connections 2024-06-13T07:10:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.438+0000 7f74016a8b80 1 -- 172.21.15.7:0/540076940 wait complete. 2024-06-13T07:10:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.439+0000 7f74016a8b80 1 -- start start 2024-06-13T07:10:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.439+0000 7f74016a8b80 1 Processor -- start 2024-06-13T07:10:01.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.442+0000 7f73f4e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 0x7f73d40a51a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73f0118790 tx=0x7f73e80072c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d720f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d720c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5557d70f0f60 con 0x5557d7208b90 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557d7150ac0 con 0x5557d7203240 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557d720fe30 con 0x5557d72094b0 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.443+0000 7f74016a8b80 10 client.0 _ll_register_callbacks cb 0x5557d70ea380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73e7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d720c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55372/0 (socket says 172.21.15.7:55372) 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 -- 172.21.15.7:0/3824777127 learned_addr learned my addr 172.21.15.7:0/3824777127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 -- 172.21.15.7:0/3824777127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 msgr2=0x5557d720f1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d720f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 -- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 msgr2=0x5557d720c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d720c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.444+0000 7f73f4e00700 1 -- 172.21.15.7:0/3824777127 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d70a38d0 con 0x5557d72094b0 2024-06-13T07:10:01.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.449+0000 7f73f4e00700 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f73e800f780 tx=0x7f73e8004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581b0ae04e63675e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.449+0000 7f73de000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.449+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8042020 con 0x5557d72094b0 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.449+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f73e803e650 con 0x5557d72094b0 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.450+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557d7162000 con 0x5557d72094b0 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.450+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73e8008040 con 0x5557d72094b0 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.450+0000 7f74016a8b80 10 client.27292 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:01.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.450+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5557d721a5c0 con 0x5557d72094b0 2024-06-13T07:10:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.451+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5557d7162000 con 0x5557d72094b0 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f73e800e040 con 0x5557d72094b0 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 10 client.27292 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 10 client.27292 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 20 client.27292 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d721f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 4 client.27292 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557d715c580 con 0x5557d721cca0 2024-06-13T07:10:01.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.453+0000 7f73efe00700 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d721f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.457+0000 7f73efe00700 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d721f090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5e96565845e0e38 server_cookie=25ca6a7480fd0d3d in_seq=0 out_seq=0 2024-06-13T07:10:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.457+0000 7f73de000700 10 client.27292 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:10:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.458+0000 7f73de000700 1 -- 172.21.15.7:0/3824777127 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5557d715c580 con 0x5557d721cca0 2024-06-13T07:10:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.458+0000 7f73de000700 10 client.27292 handle_command_reply: tid=0 2024-06-13T07:10:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.458+0000 7f74016a8b80 1 client.27292 shutdown 2024-06-13T07:10:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.460+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 msgr2=0x5557d720ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.460+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f73e800f780 tx=0x7f73e8004cf0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.460+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 shutdown_connections 2024-06-13T07:10:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.460+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d721f090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f73de000700 0 client.27292 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:10:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d72094b0 0x5557d720ca30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d720f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 --2- 172.21.15.7:0/3824777127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d720c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 >> 172.21.15.7:0/3824777127 conn(0x5557d712f170 msgr2=0x5557d71f7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 shutdown_connections 2024-06-13T07:10:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.461+0000 7f74016a8b80 1 -- 172.21.15.7:0/3824777127 wait complete. 2024-06-13T07:10:01.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.464+0000 7f74016a8b80 20 client.27292 trim_cache size 0 max 16384 2024-06-13T07:10:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc67400640 1 -- 172.21.15.7:0/3576275693 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556cdbb5fea0 con 0x556cdbc76740 2024-06-13T07:10:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc67400640 10 client.27280 handle_command_reply: tid=0 2024-06-13T07:10:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 client.27280 shutdown 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 msgr2=0x556cdbc1cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc74097ab0 tx=0x7fcc7409b260 comp rx=0 tx=0).stop 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 shutdown_connections 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556cdbc76740 0x556cdbc78b30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556cdbc0b380 0x556cdbc1e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556cdbc046c0 0x556cdbc1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 --2- 172.21.15.7:0/3576275693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cdbbf55f0 0x556cdbc1cd60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.484+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 >> 172.21.15.7:0/3576275693 conn(0x556cdbbbcfb0 msgr2=0x556cdbbf0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.485+0000 7fcc67400640 0 client.27280 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.485+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 shutdown_connections 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.485+0000 7fcc8b062740 1 -- 172.21.15.7:0/3576275693 wait complete. 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.485+0000 7fcc8b062740 20 client.27280 trim_cache size 0 max 16384 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:10:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.486+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 msgr2=0x7fcc680a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.486+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 0x7fcc680a5400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e6d0 tx=0x7fcc6c000bc0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.486+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 msgr2=0x7fcc8010f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.486+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 0x7fcc8010f630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c970 tx=0x7fcc7400ce30 comp rx=0 tx=0).stop 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 shutdown_connections 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fcc680a2f40 0x7fcc680a5400 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc80110220 0x7fcc8010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc80109300 0x7fcc8010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 --2- 172.21.15.7:0/3359605053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc801089a0 0x7fcc8010f630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 >> 172.21.15.7:0/3359605053 conn(0x7fcc8006ea10 msgr2=0x7fcc80109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 shutdown_connections 2024-06-13T07:10:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:01.487+0000 7fcc88200640 1 -- 172.21.15.7:0/3359605053 wait complete. 2024-06-13T07:10:01.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.505+0000 7f74016a8b80 1 Processor -- start 2024-06-13T07:10:01.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 -- start start 2024-06-13T07:10:01.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d71f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d720ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc011290 con 0x5557d7208b90 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc00ea90 con 0x5557d72094b0 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f74016a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc00c8e0 con 0x5557d7203240 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.506+0000 7f73efe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.507+0000 7f73efe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55382/0 (socket says 172.21.15.7:55382) 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.507+0000 7f73efe00700 1 -- 172.21.15.7:0/2751663938 learned_addr learned my addr 172.21.15.7:0/2751663938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.507+0000 7f73e7400700 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d71f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.507+0000 7f73f4e00700 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d720ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.508+0000 7f73efe00700 1 -- 172.21.15.7:0/2751663938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 msgr2=0x5557d720ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.508+0000 7f73efe00700 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d720ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.508+0000 7f73efe00700 1 -- 172.21.15.7:0/2751663938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 msgr2=0x5557d71f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.508+0000 7f73efe00700 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d71f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.508+0000 7f73efe00700 1 -- 172.21.15.7:0/2751663938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d71fe100 con 0x5557d7203240 2024-06-13T07:10:01.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.509+0000 7f73efe00700 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f73e00cc480 tx=0x7f73e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=782f44c3eddbf02e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.512+0000 7f73e4c00700 1 -- 172.21.15.7:0/2751663938 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00b4070 con 0x5557d7203240 2024-06-13T07:10:01.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.512+0000 7f73e4c00700 1 -- 172.21.15.7:0/2751663938 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e009f590 con 0x5557d7203240 2024-06-13T07:10:01.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.512+0000 7f73e4c00700 1 -- 172.21.15.7:0/2751663938 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0014070 con 0x5557d7203240 2024-06-13T07:10:01.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 -- 172.21.15.7:0/2751663938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 msgr2=0x5557d7202d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f73e00cc480 tx=0x7f73e0004750 comp rx=0 tx=0).stop 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 -- 172.21.15.7:0/2751663938 shutdown_connections 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d72094b0 0x5557d720ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7203240 0x5557d7202d00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.513+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2751663938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d7208b90 0x5557d71f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.514+0000 7f74016a8b80 1 -- 172.21.15.7:0/2751663938 >> 172.21.15.7:0/2751663938 conn(0x5557d712f170 msgr2=0x5557d72098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.514+0000 7f74016a8b80 1 -- 172.21.15.7:0/2751663938 shutdown_connections 2024-06-13T07:10:01.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.514+0000 7f74016a8b80 1 -- 172.21.15.7:0/2751663938 wait complete. 2024-06-13T07:10:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.515+0000 7f74016a8b80 1 -- start start 2024-06-13T07:10:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.515+0000 7f74016a8b80 1 Processor -- start 2024-06-13T07:10:01.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.517+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.517+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 0x5557d7215a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.517+0000 7f74016a8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 0x5557d7224170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.517+0000 7f73efe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.517+0000 7f73efe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43298/0 (socket says 172.21.15.7:43298) 2024-06-13T07:10:01.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f73efe00700 1 -- 172.21.15.7:0/2385562505 learned_addr learned my addr 172.21.15.7:0/2385562505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:01.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc00cf50 con 0x5557d72094b0 2024-06-13T07:10:01.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc00e3c0 con 0x5557d7203240 2024-06-13T07:10:01.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc00d620 con 0x5557d7208b90 2024-06-13T07:10:01.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f74016a8b80 10 client.0 _ll_register_callbacks cb 0x5557d70ea380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f73e7400700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 0x5557d7215a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.518+0000 7f73f4e00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 0x5557d7224170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73f4e00700 1 -- 172.21.15.7:0/2385562505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 msgr2=0x5557d7215a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73f4e00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 0x5557d7215a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73f4e00700 1 -- 172.21.15.7:0/2385562505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 msgr2=0x5557d7215510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73f4e00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73f4e00700 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d7224950 con 0x5557d72094b0 2024-06-13T07:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.519+0000 7f73efe00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.520+0000 7f73e7400700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 0x5557d7215a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.520+0000 7f73de000700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.520+0000 7f73f4e00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 0x5557d7224170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73e8011040 tx=0x7f73e800fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edec7def4ad1bbeb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.521+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8042040 con 0x5557d72094b0 2024-06-13T07:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.522+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f73e804e070 con 0x5557d72094b0 2024-06-13T07:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.522+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e009f590 con 0x5557d72094b0 2024-06-13T07:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.523+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73e8042040 con 0x5557d72094b0 2024-06-13T07:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.523+0000 7f74016a8b80 10 client.16605 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.523+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5557d7287590 con 0x5557d72094b0 2024-06-13T07:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e009f590 con 0x5557d72094b0 2024-06-13T07:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f73e800e070 con 0x5557d72094b0 2024-06-13T07:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 10 client.16605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:10:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 20 client.16605 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d7289cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 4 client.16605 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:10:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.524+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557d6fd6250 con 0x5557d721cca0 2024-06-13T07:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.526+0000 7f73efe00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d7289cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.531+0000 7f73efe00700 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d7289cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae7887969c3bf528 server_cookie=ea7b9dd90986336 in_seq=0 out_seq=0 2024-06-13T07:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.531+0000 7f73de000700 10 client.16605 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.532+0000 7f73de000700 1 -- 172.21.15.7:0/2385562505 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557d6fd6250 con 0x5557d721cca0 2024-06-13T07:10:01.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.532+0000 7f73de000700 10 client.16605 handle_command_reply: tid=0 2024-06-13T07:10:01.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.532+0000 7f74016a8b80 1 client.16605 shutdown 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 msgr2=0x5557d7224170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 0x5557d7224170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73e8011040 tx=0x7f73e800fed0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 shutdown_connections 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d72094b0 0x5557d7224170 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557d7203240 0x5557d7215510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.533+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5557d721cca0 0x5557d7289cd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.534+0000 7f74016a8b80 1 --2- 172.21.15.7:0/2385562505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557d7208b90 0x5557d7215a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.534+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 >> 172.21.15.7:0/2385562505 conn(0x5557d712f170 msgr2=0x5557d720dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.534+0000 7f73de000700 0 client.16605 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:10:01.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.534+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 shutdown_connections 2024-06-13T07:10:01.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.535+0000 7f74016a8b80 1 -- 172.21.15.7:0/2385562505 wait complete. 2024-06-13T07:10:01.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.535+0000 7f74016a8b80 20 client.16605 trim_cache size 0 max 16384 2024-06-13T07:10:01.535 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.537+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 msgr2=0x7f73d40a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.537+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 0x7f73d40a51a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73f0118790 tx=0x7f73e80072c0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.537+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 msgr2=0x7f73f0117bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.537+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f73e000d980 tx=0x7f73e000dd40 comp rx=0 tx=0).stop 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 shutdown_connections 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f0107450 0x7f73f0111d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f0102540 0x7f73f0117bd0 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f73e000d980 tx=0x7f73e000dd40 comp rx=0 tx=0).stop 2024-06-13T07:10:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f73d40a2ce0 0x7f73d40a51a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 --2- 172.21.15.7:0/4054641247 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f0102ee0 0x7f73f0118110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.538+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 >> 172.21.15.7:0/4054641247 conn(0x7f73f00795f0 msgr2=0x7f73f01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.539+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 shutdown_connections 2024-06-13T07:10:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:01.539+0000 7f73f6e00700 1 -- 172.21.15.7:0/4054641247 wait complete. 2024-06-13T07:10:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:01 smithi175 ceph-mon[36886]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.5 MiB/s wr, 855 op/s 2024-06-13T07:10:01.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:01 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:01 smithi007 ceph-mon[32077]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.5 MiB/s wr, 855 op/s 2024-06-13T07:10:01.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:01 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:01.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:01 smithi189 ceph-mon[36616]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 9.5 MiB/s wr, 855 op/s 2024-06-13T07:10:01.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:01 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:02.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:10:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 840.0 s, 621.9 tps, lat 157.634 ms stddev 164.340 2024-06-13T07:10:03.285 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:10:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:03 smithi175 ceph-mon[36886]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-06-13T07:10:03.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:03 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:03 smithi007 ceph-mon[32077]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-06-13T07:10:03.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:03 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:03 smithi189 ceph-mon[36616]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 10 MiB/s wr, 920 op/s 2024-06-13T07:10:03.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:03 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:10:04.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:04 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:04 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:04.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:04 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T07:10:05.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T07:10:05.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:05 smithi175 ceph-mon[36886]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T07:10:05.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:05 smithi007 ceph-mon[32077]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T07:10:05.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi189 to 176.2M 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi189 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: Adjusting osd_memory_target on smithi175 to 176.2M 2024-06-13T07:10:05.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:05 smithi189 ceph-mon[36616]: Unable to set osd_memory_target on smithi175 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-06-13T07:10:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:06.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:10:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:06.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:10:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:10:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:10:06.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:10:07.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:07 smithi007 ceph-mon[32077]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-06-13T07:10:07.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:07 smithi189 ceph-mon[36616]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-06-13T07:10:07.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:07 smithi175 ceph-mon[36886]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 10 MiB/s wr, 883 op/s 2024-06-13T07:10:07.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 845.0 s, 210.0 tps, lat 465.498 ms stddev 652.857 2024-06-13T07:10:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.712+0000 7f89abe00700 1 -- 172.21.15.7:0/1435168827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a4096210 msgr2=0x7f89a40965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.712+0000 7f89abe00700 1 --2- 172.21.15.7:0/1435168827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a4096210 0x7f89a40965f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f899801a9d0 tx=0x7f899801ace0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 -- 172.21.15.7:0/1435168827 shutdown_connections 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 --2- 172.21.15.7:0/1435168827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a409b0f0 0x7f89a40a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 --2- 172.21.15.7:0/1435168827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a4096210 0x7f89a40965f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 --2- 172.21.15.7:0/1435168827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 -- 172.21.15.7:0/1435168827 >> 172.21.15.7:0/1435168827 conn(0x7f89a400b8e0 msgr2=0x7f89a400bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 -- 172.21.15.7:0/1435168827 shutdown_connections 2024-06-13T07:10:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.713+0000 7f89abe00700 1 -- 172.21.15.7:0/1435168827 wait complete. 2024-06-13T07:10:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.716+0000 7f89abe00700 1 Processor -- start 2024-06-13T07:10:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.716+0000 7f89abe00700 1 -- start start 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 0x7f89a409f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a409b0f0 0x7f89a409fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4008600 con 0x7f89a4096210 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4008480 con 0x7f89a409b0f0 2024-06-13T07:10:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.718+0000 7f89abe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4008780 con 0x7f89a4096bc0 2024-06-13T07:10:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.719+0000 7f89aa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.719+0000 7f89aa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55408/0 (socket says 172.21.15.7:55408) 2024-06-13T07:10:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.719+0000 7f89aa000700 1 -- 172.21.15.7:0/2901526140 learned_addr learned my addr 172.21.15.7:0/2901526140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.719+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 0x7f89a409f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89ab400700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a409b0f0 0x7f89a409fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89aaa00700 1 -- 172.21.15.7:0/2901526140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 msgr2=0x7f89a409f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89aaa00700 1 -- 172.21.15.7:0/2901526140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a409b0f0 msgr2=0x7f89a409fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a409b0f0 0x7f89a409fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.720+0000 7f89aaa00700 1 -- 172.21.15.7:0/2901526140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a413eec0 con 0x7f89a4096210 2024-06-13T07:10:08.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.722+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 0x7f89a409f0f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f899801a8f0 tx=0x7f8998003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.723+0000 7f89a3400700 1 -- 172.21.15.7:0/2901526140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998036070 con 0x7f89a4096210 2024-06-13T07:10:08.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.723+0000 7f89a3400700 1 -- 172.21.15.7:0/2901526140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8998005da0 con 0x7f89a4096210 2024-06-13T07:10:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.723+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a413f150 con 0x7f89a4096210 2024-06-13T07:10:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.723+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a413f680 con 0x7f89a4096210 2024-06-13T07:10:08.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.723+0000 7f89a3400700 1 -- 172.21.15.7:0/2901526140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998031d60 con 0x7f89a4096210 2024-06-13T07:10:08.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.731+0000 7f89bc1d9b80 1 Processor -- start 2024-06-13T07:10:08.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.732+0000 7f89bc1d9b80 1 -- start start 2024-06-13T07:10:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.735+0000 7f89a3400700 1 -- 172.21.15.7:0/2901526140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8998031410 con 0x7f89a4096210 2024-06-13T07:10:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acd4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acdf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a92abe1a10 con 0x55a92ac4ee40 2024-06-13T07:10:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a92ac24b00 con 0x55a92acdd570 2024-06-13T07:10:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.737+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a92abdf8f0 con 0x55a92acd4960 2024-06-13T07:10:08.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55420/0 (socket says 172.21.15.7:55420) 2024-06-13T07:10:08.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 -- 172.21.15.7:0/2883934536 learned_addr learned my addr 172.21.15.7:0/2883934536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:08.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 -- 172.21.15.7:0/2883934536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 msgr2=0x55a92acdf960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acdf960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 -- 172.21.15.7:0/2883934536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 msgr2=0x55a92acd4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acd4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.738+0000 7f89aaa00700 1 -- 172.21.15.7:0/2883934536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92abccf90 con 0x55a92acd4960 2024-06-13T07:10:08.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.740+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f89980e6ba0 tx=0x7f89980e6eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e68c4e4270ba176 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.740+0000 7f8997e00700 1 -- 172.21.15.7:0/2883934536 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980ef070 con 0x55a92acd4960 2024-06-13T07:10:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.741+0000 7f89a3400700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 0x7f89900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.741+0000 7f89a3400700 1 -- 172.21.15.7:0/2901526140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f89980e09a0 con 0x7f89a4096210 2024-06-13T07:10:08.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.742+0000 7f8997e00700 1 -- 172.21.15.7:0/2883934536 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89980e0e50 con 0x55a92acd4960 2024-06-13T07:10:08.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.743+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2883934536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 msgr2=0x55a92acd4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.743+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f89980e6ba0 tx=0x7f89980e6eb0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89ab400700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 0x7f89900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2883934536 shutdown_connections 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acdf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acd4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2883934536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acd4de0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.744+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2883934536 >> 172.21.15.7:0/2883934536 conn(0x55a92abff170 msgr2=0x55a92acce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.746+0000 7f89ab400700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 0x7f89900a5160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f89ac051000 tx=0x7f89ac06e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.747+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2883934536 shutdown_connections 2024-06-13T07:10:08.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.747+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2883934536 wait complete. 2024-06-13T07:10:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.749+0000 7f89bc1d9b80 1 -- start start 2024-06-13T07:10:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.749+0000 7f89bc1d9b80 1 Processor -- start 2024-06-13T07:10:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.753+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.754+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.754+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acdd570 0x55a92acda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.754+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.754+0000 7f89aa000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43330/0 (socket says 172.21.15.7:43330) 2024-06-13T07:10:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 -- 172.21.15.7:0/3667065484 learned_addr learned my addr 172.21.15.7:0/3667065484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a92abc0f60 con 0x55a92ac4ee40 2024-06-13T07:10:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a92ac20ac0 con 0x55a92acd4960 2024-06-13T07:10:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a92acdff90 con 0x55a92acdd570 2024-06-13T07:10:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89ab400700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acdd570 0x55a92acda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 -- 172.21.15.7:0/3667065484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acdd570 msgr2=0x55a92acda390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acdd570 0x55a92acda390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 -- 172.21.15.7:0/3667065484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 msgr2=0x55a92acdcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.755+0000 7f89aa000700 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92ab738d0 con 0x55a92acd4960 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.758+0000 7f89bc1d9b80 10 client.0 _ll_register_callbacks cb 0x55a92abbe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.758+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89980e0e50 con 0x55a92acd4960 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.758+0000 7f8994c00700 10 client.27422 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.758+0000 7f89aa000700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f899c00d1e0 tx=0x7f899c00d5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60984df5dc13cf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.759+0000 7f8994c00700 1 -- 172.21.15.7:0/3667065484 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c007380 con 0x55a92acd4960 2024-06-13T07:10:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.759+0000 7f8994c00700 1 -- 172.21.15.7:0/3667065484 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f899c013480 con 0x55a92acd4960 2024-06-13T07:10:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.759+0000 7f8994c00700 1 -- 172.21.15.7:0/3667065484 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f899c0136e0 con 0x55a92acd4960 2024-06-13T07:10:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.759+0000 7f89bc1d9b80 10 client.27422 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.760+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55a92acea3e0 con 0x55a92acd4960 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.760+0000 7f8994c00700 1 -- 172.21.15.7:0/3667065484 <== mon.2 v2:172.21.15.175:3300/0 4 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f899c013880 con 0x55a92acd4960 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 10 client.27422 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 20 client.27422 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92acecdc0 0x55a92acef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 4 client.27422 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a92ac32000 con 0x55a92acecdc0 2024-06-13T07:10:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.761+0000 7f89aaa00700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92acecdc0 0x55a92acef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.762+0000 7f89aaa00700 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92acecdc0 0x55a92acef280 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbf13f72c747eb1 server_cookie=cd07beaebad3ccbd in_seq=0 out_seq=0 2024-06-13T07:10:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.762+0000 7f8994c00700 10 client.27422 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.765+0000 7f8994c00700 1 -- 172.21.15.7:0/3667065484 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a92ac32000 con 0x55a92acecdc0 2024-06-13T07:10:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.765+0000 7f8994c00700 10 client.27422 handle_command_reply: tid=0 2024-06-13T07:10:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.765+0000 7f89bc1d9b80 1 client.27422 shutdown 2024-06-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.765+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 msgr2=0x55a92acd9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.765+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f899c00d1e0 tx=0x7f899c00d5a0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 shutdown_connections 2024-06-13T07:10:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92acecdc0 0x55a92acef280 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f8994c00700 0 client.27422 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acdd570 0x55a92acda390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/3667065484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acd4960 0x55a92acd9e50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.766+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 >> 172.21.15.7:0/3667065484 conn(0x55a92abff170 msgr2=0x55a92acce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.769+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 shutdown_connections 2024-06-13T07:10:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.769+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/3667065484 wait complete. 2024-06-13T07:10:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.770+0000 7f89bc1d9b80 20 client.27422 trim_cache size 0 max 16384 2024-06-13T07:10:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.805+0000 7f89bc1d9b80 1 Processor -- start 2024-06-13T07:10:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 -- start start 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acf4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89a414ab70 con 0x55a92ac4ee40 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4148370 con 0x55a92acdd570 2024-06-13T07:10:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.806+0000 7f89bc1d9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41461c0 con 0x55a92acd4960 2024-06-13T07:10:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.807+0000 7f89ab400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.807+0000 7f89ab400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43346/0 (socket says 172.21.15.7:43346) 2024-06-13T07:10:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60352/0 (socket says 172.21.15.7:60352) 2024-06-13T07:10:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 -- 172.21.15.7:0/2655249555 learned_addr learned my addr 172.21.15.7:0/2655249555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.807+0000 7f89aa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acf4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 -- 172.21.15.7:0/2655249555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 msgr2=0x55a92acf4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acf4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 -- 172.21.15.7:0/2655249555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 msgr2=0x55a92acf50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89aaa00700 1 -- 172.21.15.7:0/2655249555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92acced60 con 0x55a92ac4ee40 2024-06-13T07:10:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.808+0000 7f89ab400700 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.809+0000 7f89aaa00700 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89980de1b0 tx=0x7f899801a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b57f4c249e70ce server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.810+0000 7f8997400700 1 -- 172.21.15.7:0/2655249555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980a91f0 con 0x55a92ac4ee40 2024-06-13T07:10:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.810+0000 7f8997400700 1 -- 172.21.15.7:0/2655249555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a92ac32000 con 0x55a92ac4ee40 2024-06-13T07:10:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.810+0000 7f8997400700 1 -- 172.21.15.7:0/2655249555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980a91f0 con 0x55a92ac4ee40 2024-06-13T07:10:08.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2655249555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 msgr2=0x55a92acdbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f89980de1b0 tx=0x7f899801a320 comp rx=0 tx=0).stop 2024-06-13T07:10:08.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2655249555 shutdown_connections 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92acdd570 0x55a92acf50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92ac4ee40 0x55a92acdbd90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.812+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/2655249555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92acf4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.813+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2655249555 >> 172.21.15.7:0/2655249555 conn(0x55a92abff170 msgr2=0x55a92acdb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.814+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2655249555 shutdown_connections 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.815+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/2655249555 wait complete. 2024-06-13T07:10:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.815+0000 7f89bc1d9b80 1 -- start start 2024-06-13T07:10:08.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.816+0000 7f89bc1d9b80 1 Processor -- start 2024-06-13T07:10:08.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.817+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.818+0000 7f89bc1d9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92ace0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.818+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.819+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:43354/0 (socket says 172.21.15.7:43354) 2024-06-13T07:10:08.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.819+0000 7f89aaa00700 1 -- 172.21.15.7:0/386361886 learned_addr learned my addr 172.21.15.7:0/386361886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:08.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.820+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 0x55a92ad55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.821+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4146830 con 0x55a92acdd570 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.821+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4147ca0 con 0x55a92ac4ee40 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.821+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4146f00 con 0x55a92acd4960 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.821+0000 7f89bc1d9b80 10 client.0 _ll_register_callbacks cb 0x55a92abbe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.819+0000 7f89aa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92ace0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.821+0000 7f89ab400700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 0x55a92ad55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.822+0000 7f89aaa00700 1 -- 172.21.15.7:0/386361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 msgr2=0x55a92ace0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.823+0000 7f89aaa00700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92ace0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.823+0000 7f89aaa00700 1 -- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 msgr2=0x55a92ad55b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.823+0000 7f89aaa00700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 0x55a92ad55b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.823+0000 7f89aaa00700 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92ace10e0 con 0x55a92ac4ee40 2024-06-13T07:10:08.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.824+0000 7f89aa000700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92ace0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:08.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.824+0000 7f89ab400700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 0x55a92ad55b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.825+0000 7f89aaa00700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f89980e61b0 tx=0x7f8998031a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51408c1984959f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.825+0000 7f8994c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:08.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.826+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980acd20 con 0x55a92ac4ee40 2024-06-13T07:10:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.826+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f89980de1e0 con 0x55a92ac4ee40 2024-06-13T07:10:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.827+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a92ac32000 con 0x55a92ac4ee40 2024-06-13T07:10:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.827+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f89980a91f0 con 0x55a92ac4ee40 2024-06-13T07:10:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.827+0000 7f89bc1d9b80 10 client.27425 fetch_fsmap learned FSMap version 41 2024-06-13T07:10:08.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.827+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a92ad570d0 con 0x55a92ac4ee40 2024-06-13T07:10:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.827+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a92ac32000 con 0x55a92ac4ee40 2024-06-13T07:10:08.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.828+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f89980a95c0 con 0x55a92ac4ee40 2024-06-13T07:10:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 10 client.27425 fetch_fsmap finished waiting for FSMap version 41 2024-06-13T07:10:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 10 client.27425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 20 client.27425 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92ad57210 0x55a92ad59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:08.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 4 client.27425 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:10:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a92ac2c580 con 0x55a92ad57210 2024-06-13T07:10:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.829+0000 7f89aa000700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92ad57210 0x55a92ad59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.831+0000 7f89aa000700 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92ad57210 0x55a92ad59600 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f148de4c08efdcb server_cookie=3cedc51e489a581d in_seq=0 out_seq=0 2024-06-13T07:10:08.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.831+0000 7f8994c00700 10 client.27425 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:08.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.832+0000 7f8994c00700 1 -- 172.21.15.7:0/386361886 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a92ac2c580 con 0x55a92ad57210 2024-06-13T07:10:08.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f8994c00700 10 client.27425 handle_command_reply: tid=0 2024-06-13T07:10:08.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 client.27425 shutdown 2024-06-13T07:10:08.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 msgr2=0x55a92ace0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f89980e61b0 tx=0x7f8998031a60 comp rx=0 tx=0).stop 2024-06-13T07:10:08.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 shutdown_connections 2024-06-13T07:10:08.835 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55a92ad57210 0x55a92ad59600 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a92acdd570 0x55a92ad55b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a92ac4ee40 0x55a92ace0300 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 --2- 172.21.15.7:0/386361886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a92acd4960 0x55a92ace0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 >> 172.21.15.7:0/386361886 conn(0x55a92abff170 msgr2=0x55a92acd8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.835+0000 7f8994c00700 0 client.27425 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.836+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 shutdown_connections 2024-06-13T07:10:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.836+0000 7f89bc1d9b80 1 -- 172.21.15.7:0/386361886 wait complete. 2024-06-13T07:10:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.836+0000 7f89bc1d9b80 20 client.27425 trim_cache size 0 max 16384 2024-06-13T07:10:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.837+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 msgr2=0x7f89900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.837+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 0x7f89900a5160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f89ac051000 tx=0x7f89ac06e040 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.837+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 msgr2=0x7f89a409f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.837+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 0x7f89a409f0f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f899801a8f0 tx=0x7f8998003ce0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 shutdown_connections 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f89900a2ca0 0x7f89900a5160 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89a409b0f0 0x7f89a409fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4096210 0x7f89a409f0f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 --2- 172.21.15.7:0/2901526140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89a4096bc0 0x7f89a409f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 >> 172.21.15.7:0/2901526140 conn(0x7f89a400b8e0 msgr2=0x7f89a4095740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 shutdown_connections 2024-06-13T07:10:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:08.838+0000 7f89abe00700 1 -- 172.21.15.7:0/2901526140 wait complete. 2024-06-13T07:10:09.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:09 smithi007 ceph-mon[32077]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-13T07:10:09.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:09 smithi189 ceph-mon[36616]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-13T07:10:09.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:09 smithi175 ceph-mon[36886]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-06-13T07:10:11.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:11 smithi007 ceph-mon[32077]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-06-13T07:10:11.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:11 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:11 smithi189 ceph-mon[36616]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-06-13T07:10:11.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:11 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:11 smithi175 ceph-mon[36886]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-06-13T07:10:11.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:11 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 850.0 s, 392.0 tps, lat 256.416 ms stddev 364.424 2024-06-13T07:10:13.222 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-13T07:10:13.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-13T07:10:13.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:13 smithi007 ceph-mon[32077]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-06-13T07:10:13.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:13 smithi175 ceph-mon[36886]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-06-13T07:10:13.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:13 smithi189 ceph-mon[36616]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-06-13T07:10:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.694+0000 7fb26a000640 1 -- 172.21.15.7:0/2562587369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2640743a0 msgr2=0x7fb264074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.694+0000 7fb26a000640 1 --2- 172.21.15.7:0/2562587369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2640743a0 0x7fb264074800 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb254009900 tx=0x7fb25402a4b0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 -- 172.21.15.7:0/2562587369 shutdown_connections 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 --2- 172.21.15.7:0/2562587369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2641098c0 0x7fb26410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 --2- 172.21.15.7:0/2562587369 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2640743a0 0x7fb264074800 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 --2- 172.21.15.7:0/2562587369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb264075980 0x7fb264073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 -- 172.21.15.7:0/2562587369 >> 172.21.15.7:0/2562587369 conn(0x7fb2640fcfc0 msgr2=0x7fb2640ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 -- 172.21.15.7:0/2562587369 shutdown_connections 2024-06-13T07:10:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.695+0000 7fb26a000640 1 -- 172.21.15.7:0/2562587369 wait complete. 2024-06-13T07:10:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.697+0000 7fb26a000640 1 Processor -- start 2024-06-13T07:10:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 -- start start 2024-06-13T07:10:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2640743a0 0x7fb2640702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 0x7fb2640707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 0x7fb2641ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410f8e0 con 0x7fb2640743a0 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410f760 con 0x7fb2641098c0 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.699+0000 7fb26a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb26410fa60 con 0x7fb264075980 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb263e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 0x7fb2641ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 0x7fb2640707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb263e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 0x7fb2641ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38710/0 (socket says 172.21.15.7:38710) 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb263e00640 1 -- 172.21.15.7:0/2828086186 learned_addr learned my addr 172.21.15.7:0/2828086186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb263400640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2640743a0 0x7fb2640702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 -- 172.21.15.7:0/2828086186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 msgr2=0x7fb2641ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 0x7fb2641ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 -- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2640743a0 msgr2=0x7fb2640702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2640743a0 0x7fb2640702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.700+0000 7fb262a00640 1 -- 172.21.15.7:0/2828086186 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb264073960 con 0x7fb264075980 2024-06-13T07:10:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.701+0000 7fb262a00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 0x7fb2640707e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb254009230 tx=0x7fb254005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.701+0000 7fb25be00640 1 -- 172.21.15.7:0/2828086186 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254040030 con 0x7fb264075980 2024-06-13T07:10:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.701+0000 7fb25be00640 1 -- 172.21.15.7:0/2828086186 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25403cb30 con 0x7fb264075980 2024-06-13T07:10:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.701+0000 7fb25be00640 1 -- 172.21.15.7:0/2828086186 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254031cc0 con 0x7fb264075980 2024-06-13T07:10:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.702+0000 7fb26a000640 1 -- 172.21.15.7:0/2828086186 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2641054c0 con 0x7fb264075980 2024-06-13T07:10:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.703+0000 7fb26a000640 1 -- 172.21.15.7:0/2828086186 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2641a61b0 con 0x7fb264075980 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 Processor -- start 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 -- start start 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa96fef0 0x55d7fa970310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa977ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa8d3420 con 0x55d7fa96fef0 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa87ac50 con 0x55d7fa9708e0 2024-06-13T07:10:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.710+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa8cf3e0 con 0x55d7fa975a50 2024-06-13T07:10:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.711+0000 7fb25be00640 1 -- 172.21.15.7:0/2828086186 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb254043030 con 0x7fb264075980 2024-06-13T07:10:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb262a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa977ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38720/0 (socket says 172.21.15.7:38720) 2024-06-13T07:10:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 -- 172.21.15.7:0/1352188446 learned_addr learned my addr 172.21.15.7:0/1352188446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 -- 172.21.15.7:0/1352188446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 msgr2=0x55d7fa977ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa977ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 -- 172.21.15.7:0/1352188446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa96fef0 msgr2=0x55d7fa970310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa96fef0 0x55d7fa970310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.712+0000 7fb263400640 1 -- 172.21.15.7:0/1352188446 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fa5e7210 con 0x55d7fa9708e0 2024-06-13T07:10:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.713+0000 7fb263400640 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009e70 tx=0x7fb25c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1753237bc9310f6c server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.714+0000 7fb24f400640 1 -- 172.21.15.7:0/1352188446 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c01a680 con 0x55d7fa9708e0 2024-06-13T07:10:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.714+0000 7fb24f400640 1 -- 172.21.15.7:0/1352188446 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c01ad00 con 0x55d7fa9708e0 2024-06-13T07:10:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.714+0000 7fb24f400640 1 -- 172.21.15.7:0/1352188446 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c011e00 con 0x55d7fa9708e0 2024-06-13T07:10:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.718+0000 7fb25be00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 0x7fb2480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.719+0000 7fb25be00640 1 -- 172.21.15.7:0/2828086186 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb2540ef550 con 0x7fb264075980 2024-06-13T07:10:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.720+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1352188446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 msgr2=0x55d7fa9753e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.720+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009e70 tx=0x7fb25c00f730 comp rx=0 tx=0).stop 2024-06-13T07:10:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.721+0000 7fb263e00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 0x7fb2480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.722+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1352188446 shutdown_connections 2024-06-13T07:10:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.722+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa977ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.723+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa9708e0 0x55d7fa9753e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.723+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1352188446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa96fef0 0x55d7fa970310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.723+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1352188446 >> 172.21.15.7:0/1352188446 conn(0x55d7fa937fb0 msgr2=0x55d7fa96b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.723+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1352188446 shutdown_connections 2024-06-13T07:10:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.724+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1352188446 wait complete. 2024-06-13T07:10:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.724+0000 7fb26cdd5740 1 -- start start 2024-06-13T07:10:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.724+0000 7fb26cdd5740 1 Processor -- start 2024-06-13T07:10:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.725+0000 7fb263e00640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 0x7fb2480a51f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.727+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.727+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.727+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa981560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.728+0000 7fb262a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.728+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa978910 con 0x55d7fa975a50 2024-06-13T07:10:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.728+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa88e0b0 con 0x55d7fa96fef0 2024-06-13T07:10:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.728+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7fa978a90 con 0x55d7fa9708e0 2024-06-13T07:10:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.728+0000 7fb26cdd5740 10 client.0 _ll_register_callbacks cb 0x55d7fa879f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.729+0000 7fb263400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.729+0000 7fb262a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35800/0 (socket says 172.21.15.7:35800) 2024-06-13T07:10:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.729+0000 7fb262a00640 1 -- 172.21.15.7:0/1353354242 learned_addr learned my addr 172.21.15.7:0/1353354242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb263e00640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa981560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb262a00640 1 -- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 msgr2=0x55d7fa979fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb262a00640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb262a00640 1 -- 172.21.15.7:0/1353354242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 msgr2=0x55d7fa981560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb262a00640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa981560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.730+0000 7fb262a00640 1 -- 172.21.15.7:0/1353354242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fa6b3620 con 0x55d7fa9708e0 2024-06-13T07:10:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.731+0000 7fb262a00640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2540bbdb0 tx=0x7fb2540bbbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e36274fb1c6103d server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.731+0000 7fb24cc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.732+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2540ee040 con 0x55d7fa9708e0 2024-06-13T07:10:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.732+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb2540ba120 con 0x55d7fa9708e0 2024-06-13T07:10:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.732+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25c01ad00 con 0x55d7fa9708e0 2024-06-13T07:10:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.733+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb254031e60 con 0x55d7fa9708e0 2024-06-13T07:10:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.733+0000 7fb26cdd5740 10 client.27322 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.733+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d7fa978980 con 0x55d7fa9708e0 2024-06-13T07:10:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.734+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c01ad00 con 0x55d7fa9708e0 2024-06-13T07:10:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.734+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fb254031440 con 0x55d7fa9708e0 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.734+0000 7fb26cdd5740 10 client.27322 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.735+0000 7fb26cdd5740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.735+0000 7fb26cdd5740 20 client.27322 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.735+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa986040 0x55d7fa988480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.735+0000 7fb26cdd5740 4 client.27322 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.735+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7fa8e0b50 con 0x55d7fa986040 2024-06-13T07:10:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.736+0000 7fb263400640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa986040 0x55d7fa988480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.738+0000 7fb263400640 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa986040 0x55d7fa988480 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd322dc35df01add server_cookie=d8b83c14ec069fb in_seq=0 out_seq=0 2024-06-13T07:10:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.738+0000 7fb24cc00640 10 client.27322 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.740+0000 7fb24cc00640 1 -- 172.21.15.7:0/1353354242 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d7fa8e0b50 con 0x55d7fa986040 2024-06-13T07:10:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.740+0000 7fb24cc00640 10 client.27322 handle_command_reply: tid=0 2024-06-13T07:10:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.740+0000 7fb26cdd5740 1 client.27322 shutdown 2024-06-13T07:10:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 msgr2=0x55d7fa97a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2540bbdb0 tx=0x7fb2540bbbf0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 shutdown_connections 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa986040 0x55d7fa988480 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa981560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa97a520 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/1353354242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 >> 172.21.15.7:0/1353354242 conn(0x55d7fa937fb0 msgr2=0x55d7fa96aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.741+0000 7fb24cc00640 0 client.27322 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.742+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 shutdown_connections 2024-06-13T07:10:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.742+0000 7fb26cdd5740 1 -- 172.21.15.7:0/1353354242 wait complete. 2024-06-13T07:10:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.742+0000 7fb26cdd5740 20 client.27322 trim_cache size 0 max 16384 2024-06-13T07:10:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.786+0000 7fb26cdd5740 1 Processor -- start 2024-06-13T07:10:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 -- start start 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa981520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa96b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c4320 con 0x55d7fa975a50 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c1b30 con 0x55d7fa96fef0 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.790+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641ae430 con 0x55d7fa9708e0 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38742/0 (socket says 172.21.15.7:38742) 2024-06-13T07:10:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 -- 172.21.15.7:0/2474069966 learned_addr learned my addr 172.21.15.7:0/2474069966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb262a00640 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa981520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 -- 172.21.15.7:0/2474069966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 msgr2=0x55d7fa981520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa981520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 -- 172.21.15.7:0/2474069966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 msgr2=0x55d7fa96b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa96b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 -- 172.21.15.7:0/2474069966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fa974ce0 con 0x55d7fa96fef0 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb262a00640 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa981520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.791+0000 7fb263400640 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009d90 tx=0x7fb25c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90577826fbc70fe0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.792+0000 7fb24ea00640 1 -- 172.21.15.7:0/2474069966 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004ba0 con 0x55d7fa96fef0 2024-06-13T07:10:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.792+0000 7fb24ea00640 1 -- 172.21.15.7:0/2474069966 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c004d40 con 0x55d7fa96fef0 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.792+0000 7fb24ea00640 1 -- 172.21.15.7:0/2474069966 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c01a870 con 0x55d7fa96fef0 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2474069966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 msgr2=0x55d7fa979600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009d90 tx=0x7fb25c009800 comp rx=0 tx=0).stop 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2474069966 shutdown_connections 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa975a50 0x55d7fa96b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa9708e0 0x55d7fa981520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2474069966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa979600 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.793+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2474069966 >> 172.21.15.7:0/2474069966 conn(0x55d7fa937fb0 msgr2=0x55d7fa9666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.794+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2474069966 shutdown_connections 2024-06-13T07:10:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.794+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2474069966 wait complete. 2024-06-13T07:10:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.796+0000 7fb26cdd5740 1 -- start start 2024-06-13T07:10:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.796+0000 7fb26cdd5740 1 Processor -- start 2024-06-13T07:10:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa97e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa9708e0 0x55d7fa97e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641bfff0 con 0x55d7fa9708e0 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c1460 con 0x55d7fa96fef0 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2641c06c0 con 0x55d7fa975a50 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.797+0000 7fb26cdd5740 10 client.0 _ll_register_callbacks cb 0x7fb254009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35810/0 (socket says 172.21.15.7:35810) 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 -- 172.21.15.7:0/2751515529 learned_addr learned my addr 172.21.15.7:0/2751515529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 -- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 msgr2=0x55d7fa97e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa97e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 -- 172.21.15.7:0/2751515529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa9708e0 msgr2=0x55d7fa97e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa9708e0 0x55d7fa97e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.798+0000 7fb263e00640 1 -- 172.21.15.7:0/2751515529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fa9ef250 con 0x55d7fa975a50 2024-06-13T07:10:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.799+0000 7fb263e00640 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb25001da60 tx=0x7fb250004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84a5e6f33d4d2e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.799+0000 7fb243e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.800+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25002a060 con 0x55d7fa975a50 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.800+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb250010e00 con 0x55d7fa975a50 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.800+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25c004d40 con 0x55d7fa975a50 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.800+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb250031050 con 0x55d7fa975a50 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.801+0000 7fb26cdd5740 10 client.27325 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.801+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d7fa9f0520 con 0x55d7fa975a50 2024-06-13T07:10:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.802+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c004d40 con 0x55d7fa975a50 2024-06-13T07:10:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.804+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fb250024030 con 0x55d7fa975a50 2024-06-13T07:10:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.804+0000 7fb26cdd5740 10 client.27325 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.804+0000 7fb26cdd5740 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.804+0000 7fb26cdd5740 20 client.27325 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.804+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa9f0e80 0x55d7fa9f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.805+0000 7fb26cdd5740 4 client.27325 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.805+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7fa8daea0 con 0x55d7fa9f0e80 2024-06-13T07:10:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.805+0000 7fb263400640 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa9f0e80 0x55d7fa9f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.806+0000 7fb263400640 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa9f0e80 0x55d7fa9f32c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a253a41a39297136 server_cookie=b1b319ff0415c42d in_seq=0 out_seq=0 2024-06-13T07:10:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.806+0000 7fb243e00640 10 client.27325 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.873+0000 7fb243e00640 1 -- 172.21.15.7:0/2751515529 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d7fa8daea0 con 0x55d7fa9f0e80 2024-06-13T07:10:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.873+0000 7fb243e00640 10 client.27325 handle_command_reply: tid=0 2024-06-13T07:10:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.873+0000 7fb26cdd5740 1 client.27325 shutdown 2024-06-13T07:10:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 msgr2=0x55d7fa9eed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb25001da60 tx=0x7fb250004730 comp rx=0 tx=0).stop 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 shutdown_connections 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55d7fa9f0e80 0x55d7fa9f32c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7fa975a50 0x55d7fa9eed10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7fa9708e0 0x55d7fa97e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 --2- 172.21.15.7:0/2751515529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d7fa96fef0 0x55d7fa97e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.874+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 >> 172.21.15.7:0/2751515529 conn(0x55d7fa937fb0 msgr2=0x55d7fa9666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.875+0000 7fb243e00640 0 client.27325 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.875+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 shutdown_connections 2024-06-13T07:10:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.875+0000 7fb26cdd5740 1 -- 172.21.15.7:0/2751515529 wait complete. 2024-06-13T07:10:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.875+0000 7fb26cdd5740 20 client.27325 trim_cache size 0 max 16384 2024-06-13T07:10:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:10:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:10:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:10:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.877+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 msgr2=0x7fb2480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.877+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 0x7fb2480a51f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005850 comp rx=0 tx=0).stop 2024-06-13T07:10:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 msgr2=0x7fb2640707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 0x7fb2640707e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb254009230 tx=0x7fb254005620 comp rx=0 tx=0).stop 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 shutdown_connections 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2480a2d30 0x7fb2480a51f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2641098c0 0x7fb2641ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.878+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb264075980 0x7fb2640707e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.879+0000 7fb259200640 1 --2- 172.21.15.7:0/2828086186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2640743a0 0x7fb2640702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.879+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 >> 172.21.15.7:0/2828086186 conn(0x7fb2640fcfc0 msgr2=0x7fb264105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.879+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 shutdown_connections 2024-06-13T07:10:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:14.879+0000 7fb259200640 1 -- 172.21.15.7:0/2828086186 wait complete. 2024-06-13T07:10:15.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:15 smithi189 ceph-mon[36616]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-06-13T07:10:15.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:15 smithi175 ceph-mon[36886]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-06-13T07:10:15.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:15 smithi007 ceph-mon[32077]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 -- 172.21.15.7:0/2947803031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 msgr2=0x7f49b01115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 --2- 172.21.15.7:0/2947803031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b01115a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b0a0 tx=0x7f49a800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 -- 172.21.15.7:0/2947803031 shutdown_connections 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 --2- 172.21.15.7:0/2947803031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b01115a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 --2- 172.21.15.7:0/2947803031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 --2- 172.21.15.7:0/2947803031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 -- 172.21.15.7:0/2947803031 >> 172.21.15.7:0/2947803031 conn(0x7f49b01001c0 msgr2=0x7f49b01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 -- 172.21.15.7:0/2947803031 shutdown_connections 2024-06-13T07:10:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.833+0000 7f49b6600700 1 -- 172.21.15.7:0/2947803031 wait complete. 2024-06-13T07:10:17.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:17 smithi007 ceph-mon[32077]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 883 op/s 2024-06-13T07:10:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.834+0000 7f49b6600700 1 Processor -- start 2024-06-13T07:10:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.835+0000 7f49b6600700 1 -- start start 2024-06-13T07:10:17.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:17 smithi189 ceph-mon[36616]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 883 op/s 2024-06-13T07:10:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.836+0000 7f49b6600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.836+0000 7f49b6600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b010e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49b6600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49af400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49af400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38774/0 (socket says 172.21.15.7:38774) 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49b6600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0114a60 con 0x7f49b010a6d0 2024-06-13T07:10:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01148e0 con 0x7f49b01046d0 2024-06-13T07:10:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49af400700 1 -- 172.21.15.7:0/1723028568 learned_addr learned my addr 172.21.15.7:0/1723028568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0114be0 con 0x7f49b0105090 2024-06-13T07:10:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49af400700 1 -- 172.21.15.7:0/1723028568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 msgr2=0x7f49b010e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.837+0000 7f49aea00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b010e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.838+0000 7f49afe00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.838+0000 7f49af400700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b010e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.838+0000 7f49af400700 1 -- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 msgr2=0x7f49b010e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.838+0000 7f49af400700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.838+0000 7f49af400700 1 -- 172.21.15.7:0/1723028568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b010f290 con 0x7f49b01046d0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.839+0000 7f49aea00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.839+0000 7f49afe00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.839+0000 7f49af400700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f49a000e9a0 tx=0x7f49a000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.839+0000 7f49a7e00700 1 -- 172.21.15.7:0/1723028568 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a000cbd0 con 0x7f49b01046d0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.840+0000 7f49a7e00700 1 -- 172.21.15.7:0/1723028568 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a000cd70 con 0x7f49b01046d0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.840+0000 7f49a7e00700 1 -- 172.21.15.7:0/1723028568 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00187a0 con 0x7f49b01046d0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.840+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01ab800 con 0x7f49b01046d0 2024-06-13T07:10:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.840+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01abcf0 con 0x7f49b01046d0 2024-06-13T07:10:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.843+0000 7f49a7e00700 1 -- 172.21.15.7:0/1723028568 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f49a0010aa0 con 0x7f49b01046d0 2024-06-13T07:10:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.844+0000 7f49c0e9db80 1 Processor -- start 2024-06-13T07:10:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49a7e00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 0x7f49940a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 -- start start 2024-06-13T07:10:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49a7e00700 1 -- 172.21.15.7:0/1723028568 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49a00cf7c0 con 0x7f49b01046d0 2024-06-13T07:10:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304593a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045947770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563045849a90 con 0x563045945380 2024-06-13T07:10:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56304588cb80 con 0x5630458b6ec0 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.845+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563045847970 con 0x56304593a630 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49afe00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 0x7f49940a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49af400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304593a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35848/0 (socket says 172.21.15.7:35848) 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 -- 172.21.15.7:0/2611385970 learned_addr learned my addr 172.21.15.7:0/2611385970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49afe00700 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045947770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 -- 172.21.15.7:0/2611385970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 msgr2=0x56304593a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304593a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 -- 172.21.15.7:0/2611385970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 msgr2=0x563045947770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045947770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.846+0000 7f49aea00700 1 -- 172.21.15.7:0/2611385970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563045835010 con 0x56304593a630 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.847+0000 7f49aea00700 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f499c00a9f0 tx=0x7f499c00ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520cecb454de2470 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.847+0000 7f49afe00700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 0x7f49940a52b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009e60 tx=0x7f49a800bbe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.847+0000 7f49a4c00700 1 -- 172.21.15.7:0/2611385970 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c015070 con 0x56304593a630 2024-06-13T07:10:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.847+0000 7f49a4c00700 1 -- 172.21.15.7:0/2611385970 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499c0113f0 con 0x56304593a630 2024-06-13T07:10:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.847+0000 7f49a4c00700 1 -- 172.21.15.7:0/2611385970 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0185c0 con 0x56304593a630 2024-06-13T07:10:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 -- 172.21.15.7:0/2611385970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 msgr2=0x56304593aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f499c00a9f0 tx=0x7f499c00ad00 comp rx=0 tx=0).stop 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 -- 172.21.15.7:0/2611385970 shutdown_connections 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045947770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304593a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/2611385970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x56304593aab0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.849+0000 7f49c0e9db80 1 -- 172.21.15.7:0/2611385970 >> 172.21.15.7:0/2611385970 conn(0x5630458671f0 msgr2=0x563045936530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.850+0000 7f49c0e9db80 1 -- 172.21.15.7:0/2611385970 shutdown_connections 2024-06-13T07:10:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.850+0000 7f49c0e9db80 1 -- 172.21.15.7:0/2611385970 wait complete. 2024-06-13T07:10:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 -- start start 2024-06-13T07:10:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 Processor -- start 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045942380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563045828fe0 con 0x563045945380 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.851+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563045888b40 con 0x56304593a630 2024-06-13T07:10:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563045948040 con 0x5630458b6ec0 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49c0e9db80 10 client.0 _ll_register_callbacks cb 0x5630458267a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49aea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35862/0 (socket says 172.21.15.7:35862) 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49aea00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38794/0 (socket says 172.21.15.7:38794) 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 -- 172.21.15.7:0/1939949843 learned_addr learned my addr 172.21.15.7:0/1939949843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49afe00700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045942380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 -- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 msgr2=0x563045941e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 -- 172.21.15.7:0/1939949843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 msgr2=0x563045942380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045942380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49af400700 1 -- 172.21.15.7:0/1939949843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630457db950 con 0x5630458b6ec0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.852+0000 7f49aea00700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f49af400700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f49a00ccf20 tx=0x7f49a009c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d4b381935a91b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f4999600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a009b380 con 0x5630458b6ec0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49a0097a80 con 0x5630458b6ec0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499c0113f0 con 0x5630458b6ec0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49a00a48f0 con 0x5630458b6ec0 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f49c0e9db80 10 client.27334 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.853+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563045952820 con 0x5630458b6ec0 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499c0113f0 con 0x5630458b6ec0 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f49a00a11c0 con 0x5630458b6ec0 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 10 client.27334 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 20 client.27334 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563045954db0 0x5630459571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 4 client.27334 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56304589a080 con 0x563045954db0 2024-06-13T07:10:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.854+0000 7f49aea00700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563045954db0 0x5630459571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.855+0000 7f49aea00700 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563045954db0 0x5630459571a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631948221539edb9 server_cookie=b4b5620e0d9e4899 in_seq=0 out_seq=0 2024-06-13T07:10:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.855+0000 7f4999600700 10 client.27334 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f4999600700 1 -- 172.21.15.7:0/1939949843 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56304589a080 con 0x563045954db0 2024-06-13T07:10:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f4999600700 10 client.27334 handle_command_reply: tid=0 2024-06-13T07:10:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 client.27334 shutdown 2024-06-13T07:10:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 msgr2=0x563045944b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f49a00ccf20 tx=0x7f49a009c270 comp rx=0 tx=0).stop 2024-06-13T07:10:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 shutdown_connections 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x563045954db0 0x5630459571a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045942380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630458b6ec0 0x563045944b40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1939949843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56304593a630 0x563045941e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 >> 172.21.15.7:0/1939949843 conn(0x5630458671f0 msgr2=0x5630459357c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f4999600700 0 client.27334 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 shutdown_connections 2024-06-13T07:10:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.856+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1939949843 wait complete. 2024-06-13T07:10:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.857+0000 7f49c0e9db80 20 client.27334 trim_cache size 0 max 16384 2024-06-13T07:10:17.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:17 smithi175 ceph-mon[36886]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 883 op/s 2024-06-13T07:10:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 855.0 s, 444.0 tps, lat 237.014 ms stddev 305.774 2024-06-13T07:10:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.898+0000 7f49c0e9db80 1 Processor -- start 2024-06-13T07:10:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.900+0000 7f49c0e9db80 1 -- start start 2024-06-13T07:10:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.900+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630458b6ec0 0x563045946770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.900+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045946cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.903+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.903+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b71a0 con 0x5630458b6ec0 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.903+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b4970 con 0x563045945380 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.903+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b27c0 con 0x56304593a630 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.904+0000 7f49afe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.904+0000 7f49afe00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38798/0 (socket says 172.21.15.7:38798) 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.904+0000 7f49afe00700 1 -- 172.21.15.7:0/1922863649 learned_addr learned my addr 172.21.15.7:0/1922863649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.904+0000 7f49af400700 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630458b6ec0 0x563045946770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.904+0000 7f49aea00700 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045946cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.906+0000 7f49afe00700 1 -- 172.21.15.7:0/1922863649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 msgr2=0x563045946cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.906+0000 7f49afe00700 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045946cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.906+0000 7f49afe00700 1 -- 172.21.15.7:0/1922863649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630458b6ec0 msgr2=0x563045946770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.906+0000 7f49afe00700 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630458b6ec0 0x563045946770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.906+0000 7f49afe00700 1 -- 172.21.15.7:0/1922863649 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304592de90 con 0x563045945380 2024-06-13T07:10:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.907+0000 7f49afe00700 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f49a8015000 tx=0x7f49a8022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be1ae7e4b87ecca server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.907+0000 7f499be00700 1 -- 172.21.15.7:0/1922863649 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8026070 con 0x563045945380 2024-06-13T07:10:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.908+0000 7f499be00700 1 -- 172.21.15.7:0/1922863649 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a8007b30 con 0x563045945380 2024-06-13T07:10:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.908+0000 7f499be00700 1 -- 172.21.15.7:0/1922863649 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8020670 con 0x563045945380 2024-06-13T07:10:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1922863649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 msgr2=0x56304593c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f49a8015000 tx=0x7f49a8022000 comp rx=0 tx=0).stop 2024-06-13T07:10:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1922863649 shutdown_connections 2024-06-13T07:10:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563045945380 0x56304593c750 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5630458b6ec0 0x563045946770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/1922863649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045946cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1922863649 >> 172.21.15.7:0/1922863649 conn(0x5630458671f0 msgr2=0x56304593c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1922863649 shutdown_connections 2024-06-13T07:10:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.909+0000 7f49c0e9db80 1 -- 172.21.15.7:0/1922863649 wait complete. 2024-06-13T07:10:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.910+0000 7f49c0e9db80 1 -- start start 2024-06-13T07:10:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.910+0000 7f49c0e9db80 1 Processor -- start 2024-06-13T07:10:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.916+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.916+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045948bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.917+0000 7f49c0e9db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045949110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.917+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b2e30 con 0x563045945380 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.917+0000 7f49aea00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045948bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49af400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b42a0 con 0x5630458b6ec0 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49c0e9db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b3500 con 0x56304593a630 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49af400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:38800/0 (socket says 172.21.15.7:38800) 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49af400700 1 -- 172.21.15.7:0/3186240948 learned_addr learned my addr 172.21.15.7:0/3186240948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49c0e9db80 10 client.0 _ll_register_callbacks cb 0x5630458267a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.918+0000 7f49afe00700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045949110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 -- 172.21.15.7:0/3186240948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 msgr2=0x563045948bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045948bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 -- 172.21.15.7:0/3186240948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 msgr2=0x563045949110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045949110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 -- 172.21.15.7:0/3186240948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563045949750 con 0x5630458b6ec0 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49afe00700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045949110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49af400700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f49a00a4ab0 tx=0x7f49a00a14a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2e45aac707b993 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f4999600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00a4d40 con 0x5630458b6ec0 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f49a009b180 con 0x5630458b6ec0 2024-06-13T07:10:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.920+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a8007b30 con 0x5630458b6ec0 2024-06-13T07:10:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.921+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49a009b380 con 0x5630458b6ec0 2024-06-13T07:10:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.921+0000 7f49c0e9db80 10 client.27455 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.921+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5630459bf8a0 con 0x5630458b6ec0 2024-06-13T07:10:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.922+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a8007b30 con 0x5630458b6ec0 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.923+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f49a00ba2d0 con 0x5630458b6ec0 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.923+0000 7f49c0e9db80 10 client.27455 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.923+0000 7f49c0e9db80 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.923+0000 7f49c0e9db80 20 client.27455 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.925+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5630459bfe10 0x5630459c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.925+0000 7f49c0e9db80 4 client.27455 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-13T07:10:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.925+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563045894600 con 0x5630459bfe10 2024-06-13T07:10:17.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.926+0000 7f49aea00700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5630459bfe10 0x5630459c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.929+0000 7f49aea00700 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5630459bfe10 0x5630459c2200 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9df56fd2415fbb1 server_cookie=ef0d92218bb72987 in_seq=0 out_seq=0 2024-06-13T07:10:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.929+0000 7f4999600700 10 client.27455 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.930+0000 7f4999600700 1 -- 172.21.15.7:0/3186240948 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563045894600 con 0x5630459bfe10 2024-06-13T07:10:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.930+0000 7f4999600700 10 client.27455 handle_command_reply: tid=0 2024-06-13T07:10:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.930+0000 7f49c0e9db80 1 client.27455 shutdown 2024-06-13T07:10:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.931+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 msgr2=0x56304594d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.931+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f49a00a4ab0 tx=0x7f49a00a14a0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 shutdown_connections 2024-06-13T07:10:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5630459bfe10 0x5630459c2200 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563045945380 0x563045949110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5630458b6ec0 0x56304594d850 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 --2- 172.21.15.7:0/3186240948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56304593a630 0x563045948bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 >> 172.21.15.7:0/3186240948 conn(0x5630458671f0 msgr2=0x56304593c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.934+0000 7f4999600700 0 client.27455 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.935+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 shutdown_connections 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.935+0000 7f49c0e9db80 1 -- 172.21.15.7:0/3186240948 wait complete. 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.936+0000 7f49c0e9db80 20 client.27455 trim_cache size 0 max 16384 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "645669b7-cf5e-4483-a2e2-bec73910897a", 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-06-13T07:10:17.936 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:10:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.936+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 msgr2=0x7f49940a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.936+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 0x7f49940a52b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009e60 tx=0x7f49a800bbe0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 msgr2=0x7f49b0111350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f49a000e9a0 tx=0x7f49a000ed60 comp rx=0 tx=0).stop 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 shutdown_connections 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49b010a6d0 0x7f49b010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f49940a2df0 0x7f49940a52b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49b01046d0 0x7f49b0111350 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 --2- 172.21.15.7:0/1723028568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b0105090 0x7f49b010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 >> 172.21.15.7:0/1723028568 conn(0x7f49b01001c0 msgr2=0x7f49b0100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 shutdown_connections 2024-06-13T07:10:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:17.937+0000 7f49b6600700 1 -- 172.21.15.7:0/1723028568 wait complete. 2024-06-13T07:10:18.823 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:18 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:18.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:18 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:18.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:18 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:10:19.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub queued for path: / 2024-06-13T07:10:19.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:19.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: active paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 965 op/s 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:19 smithi007 ceph-mon[32077]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub queued for path: / 2024-06-13T07:10:19.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: active paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 965 op/s 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:19 smithi189 ceph-mon[36616]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub queued for path: / 2024-06-13T07:10:19.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: active paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 965 op/s 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:19.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:19 smithi175 ceph-mon[36886]: scrub summary: idle+waiting paths [/] 2024-06-13T07:10:20.349 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:10:21.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:21 smithi007 ceph-mon[32077]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-06-13T07:10:21.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:21 smithi189 ceph-mon[36616]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-06-13T07:10:21.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:21 smithi175 ceph-mon[36886]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-06-13T07:10:22.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 860.0 s, 428.2 tps, lat 227.644 ms stddev 283.884 2024-06-13T07:10:23.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:23 smithi007 ceph-mon[32077]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-06-13T07:10:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:23 smithi189 ceph-mon[36616]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-06-13T07:10:23.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:23 smithi175 ceph-mon[36886]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 -- 172.21.15.7:0/2840568027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 msgr2=0x7f333410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 --2- 172.21.15.7:0/2840568027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f333410f320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f332c009920 tx=0x7f332c009c30 comp rx=0 tx=0).stop 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 -- 172.21.15.7:0/2840568027 shutdown_connections 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 --2- 172.21.15.7:0/2840568027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f333410f320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 --2- 172.21.15.7:0/2840568027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f33341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 --2- 172.21.15.7:0/2840568027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f333406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.922+0000 7f3339e00700 1 -- 172.21.15.7:0/2840568027 >> 172.21.15.7:0/2840568027 conn(0x7f33340782c0 msgr2=0x7f33340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:23.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.923+0000 7f3339e00700 1 -- 172.21.15.7:0/2840568027 shutdown_connections 2024-06-13T07:10:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.926+0000 7f3339e00700 1 -- 172.21.15.7:0/2840568027 wait complete. 2024-06-13T07:10:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.928+0000 7f3339e00700 1 Processor -- start 2024-06-13T07:10:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.928+0000 7f3339e00700 1 -- start start 2024-06-13T07:10:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.929+0000 7f3339e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3339e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f3334074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3332a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3332a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55394/0 (socket says 172.21.15.7:55394) 2024-06-13T07:10:23.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f332a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f3334074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3332a00700 1 -- 172.21.15.7:0/2129077311 learned_addr learned my addr 172.21.15.7:0/2129077311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:23.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f333406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3334112730 con 0x7f333406ab10 2024-06-13T07:10:23.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33341125b0 con 0x7f33341084e0 2024-06-13T07:10:23.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.930+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33341128b0 con 0x7f333406a170 2024-06-13T07:10:23.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f333406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 -- 172.21.15.7:0/2129077311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 msgr2=0x7f3334073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:23.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 -- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 msgr2=0x7f3334074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f3334074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.931+0000 7f3333400700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333406d7c0 con 0x7f33341084e0 2024-06-13T07:10:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.932+0000 7f332a000700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f3334074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.932+0000 7f3332a00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:23.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.932+0000 7f3333400700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f333406cec0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f332c009810 tx=0x7f332c00fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.933+0000 7f3338a00700 1 -- 172.21.15.7:0/2129077311 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c00b760 con 0x7f33341084e0 2024-06-13T07:10:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.933+0000 7f3338a00700 1 -- 172.21.15.7:0/2129077311 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c005ce0 con 0x7f33341084e0 2024-06-13T07:10:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.933+0000 7f3338a00700 1 -- 172.21.15.7:0/2129077311 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c017660 con 0x7f33341084e0 2024-06-13T07:10:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.933+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333406da50 con 0x7f33341084e0 2024-06-13T07:10:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.933+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3334071930 con 0x7f33341084e0 2024-06-13T07:10:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.940+0000 7f3338a00700 1 -- 172.21.15.7:0/2129077311 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f332c003810 con 0x7f33341084e0 2024-06-13T07:10:23.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.945+0000 7f3338a00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 0x7f33180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.945+0000 7f3338a00700 1 -- 172.21.15.7:0/2129077311 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f332c0d0370 con 0x7f33341084e0 2024-06-13T07:10:23.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.947+0000 7f3332a00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 0x7f33180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.949+0000 7f3332a00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 0x7f33180a50e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3324009c10 tx=0x7f3324016040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:23.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.951+0000 7f3344744b80 1 Processor -- start 2024-06-13T07:10:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.952+0000 7f3344744b80 1 -- start start 2024-06-13T07:10:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 0x556044fb0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 0x556044fb0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 0x556044fbd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556044ebfa10 con 0x556044fb05b0 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556044f02b00 con 0x556044fbb300 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3344744b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556044ebd8f0 con 0x556044f2ce40 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3332a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 0x556044fb0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3332a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 0x556044fb0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55410/0 (socket says 172.21.15.7:55410) 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.953+0000 7f3332a00700 1 -- 172.21.15.7:0/3465339668 learned_addr learned my addr 172.21.15.7:0/3465339668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.954+0000 7f3333400700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 0x556044fbd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f332a000700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 0x556044fb0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f3333400700 1 -- 172.21.15.7:0/3465339668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 msgr2=0x556044fb0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f3333400700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 0x556044fb0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f3333400700 1 -- 172.21.15.7:0/3465339668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 msgr2=0x556044fb0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f3333400700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 0x556044fb0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.955+0000 7f3333400700 1 -- 172.21.15.7:0/3465339668 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556044eaaf90 con 0x556044fbb300 2024-06-13T07:10:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.957+0000 7f332a000700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 0x556044fb0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.957+0000 7f3333400700 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 0x556044fbd6f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a010 tx=0x7f332c098c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feff67d367f91556 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.957+0000 7f3328c00700 1 -- 172.21.15.7:0/3465339668 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0a1e60 con 0x556044fbb300 2024-06-13T07:10:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.957+0000 7f3328c00700 1 -- 172.21.15.7:0/3465339668 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c00bdc0 con 0x556044fbb300 2024-06-13T07:10:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.958+0000 7f3328c00700 1 -- 172.21.15.7:0/3465339668 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c028e50 con 0x556044fbb300 2024-06-13T07:10:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 -- 172.21.15.7:0/3465339668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 msgr2=0x556044fbd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 0x556044fbd6f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a010 tx=0x7f332c098c40 comp rx=0 tx=0).stop 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 -- 172.21.15.7:0/3465339668 shutdown_connections 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbb300 0x556044fbd6f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044f2ce40 0x556044fb0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 --2- 172.21.15.7:0/3465339668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fb05b0 0x556044fb0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 -- 172.21.15.7:0/3465339668 >> 172.21.15.7:0/3465339668 conn(0x556044edd170 msgr2=0x556044fac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 -- 172.21.15.7:0/3465339668 shutdown_connections 2024-06-13T07:10:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.959+0000 7f3344744b80 1 -- 172.21.15.7:0/3465339668 wait complete. 2024-06-13T07:10:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.960+0000 7f3344744b80 1 -- start start 2024-06-13T07:10:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.960+0000 7f3344744b80 1 Processor -- start 2024-06-13T07:10:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 0x556044fb7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fb81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556044e9ef60 con 0x556044fbb300 2024-06-13T07:10:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556044efeac0 con 0x556044f2ce40 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556044f02b00 con 0x556044fb05b0 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.961+0000 7f3344744b80 10 client.0 _ll_register_callbacks cb 0x556044e9c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f3332a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f3332a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fba9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41620/0 (socket says 172.21.15.7:41620) 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f3332a00700 1 -- 172.21.15.7:0/91721027 learned_addr learned my addr 172.21.15.7:0/91721027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 0x556044fb7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f3333400700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fb81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 -- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 msgr2=0x556044fba9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fba9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 -- 172.21.15.7:0/91721027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 msgr2=0x556044fb81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fb81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f332a000700 1 -- 172.21.15.7:0/91721027 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556044e518d0 con 0x556044fb05b0 2024-06-13T07:10:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.962+0000 7f3333400700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.964+0000 7f332a000700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 0x556044fb7cb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f333406e510 tx=0x7f331c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f5c7afaa3d3c9a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.964+0000 7f3321600700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.966+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c004a30 con 0x556044fb05b0 2024-06-13T07:10:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.966+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f331c004bd0 con 0x556044fb05b0 2024-06-13T07:10:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.966+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c00bdc0 con 0x556044fb05b0 2024-06-13T07:10:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.967+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f331c004dd0 con 0x556044fb05b0 2024-06-13T07:10:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.967+0000 7f3344744b80 10 client.27352 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.967+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556044fc81c0 con 0x556044fb05b0 2024-06-13T07:10:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.967+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c00bdc0 con 0x556044fb05b0 2024-06-13T07:10:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f331c005540 con 0x556044fb05b0 2024-06-13T07:10:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 10 client.27352 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 20 client.27352 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556044fca750 0x556044fccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 4 client.27352 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.968+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556044f10000 con 0x556044fca750 2024-06-13T07:10:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.969+0000 7f3332a00700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556044fca750 0x556044fccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.972+0000 7f3321600700 10 client.27352 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.972+0000 7f3332a00700 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556044fca750 0x556044fccb40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4ad6acb97f6357 server_cookie=8f354cdfabc0f58b in_seq=0 out_seq=0 2024-06-13T07:10:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.973+0000 7f3321600700 1 -- 172.21.15.7:0/91721027 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556044f10000 con 0x556044fca750 2024-06-13T07:10:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.973+0000 7f3321600700 10 client.27352 handle_command_reply: tid=0 2024-06-13T07:10:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.973+0000 7f3344744b80 1 client.27352 shutdown 2024-06-13T07:10:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.973+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 msgr2=0x556044fb7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.973+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 0x556044fb7cb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f333406e510 tx=0x7f331c0119b0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 shutdown_connections 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x556044fca750 0x556044fccb40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3321600700 0 client.27352 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 --2- 172.21.15.7:0/91721027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fb05b0 0x556044fb7cb0 secure :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f333406e510 tx=0x7f331c0119b0 comp rx=0 tx=0).stop 2024-06-13T07:10:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.974+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 >> 172.21.15.7:0/91721027 conn(0x556044edd170 msgr2=0x556044fab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:23.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.976+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 shutdown_connections 2024-06-13T07:10:23.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.976+0000 7f3344744b80 1 -- 172.21.15.7:0/91721027 wait complete. 2024-06-13T07:10:23.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:23.976+0000 7f3344744b80 20 client.27352 trim_cache size 0 max 16384 2024-06-13T07:10:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.030+0000 7f3344744b80 1 Processor -- start 2024-06-13T07:10:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.030+0000 7f3344744b80 1 -- start start 2024-06-13T07:10:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.030+0000 7f3344744b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044f2ce40 0x556044fba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3344744b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3344744b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 0x556044fd2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3344744b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b71c0 con 0x556044f2ce40 2024-06-13T07:10:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3344744b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b4990 con 0x556044fbaa30 2024-06-13T07:10:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3344744b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b2800 con 0x556044fbb300 2024-06-13T07:10:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f332a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3332a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044f2ce40 0x556044fba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f332a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55438/0 (socket says 172.21.15.7:55438) 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f332a000700 1 -- 172.21.15.7:0/3537300069 learned_addr learned my addr 172.21.15.7:0/3537300069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.031+0000 7f3333400700 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 0x556044fd2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 -- 172.21.15.7:0/3537300069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 msgr2=0x556044fd2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 -- 172.21.15.7:0/3537300069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044f2ce40 msgr2=0x556044fba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044f2ce40 0x556044fba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 -- 172.21.15.7:0/3537300069 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556044fb38f0 con 0x556044fbaa30 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f332a000700 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.032+0000 7f3333400700 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 0x556044fd2890 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f332c0a0e90 tx=0x7f332c009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e3720d951e48c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.033+0000 7f3323e00700 1 -- 172.21.15.7:0/3537300069 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0a0cd0 con 0x556044fbaa30 2024-06-13T07:10:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.033+0000 7f3323e00700 1 -- 172.21.15.7:0/3537300069 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c00bdc0 con 0x556044fbaa30 2024-06-13T07:10:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.033+0000 7f3323e00700 1 -- 172.21.15.7:0/3537300069 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c09ac20 con 0x556044fbaa30 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 -- 172.21.15.7:0/3537300069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 msgr2=0x556044fd2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 0x556044fd2890 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f332c0a0e90 tx=0x7f332c009630 comp rx=0 tx=0).stop 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 -- 172.21.15.7:0/3537300069 shutdown_connections 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044fbaa30 0x556044fd2890 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044f2ce40 0x556044fba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 --2- 172.21.15.7:0/3537300069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbb300 0x556044fd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.034+0000 7f3344744b80 1 -- 172.21.15.7:0/3537300069 >> 172.21.15.7:0/3537300069 conn(0x556044edd170 msgr2=0x556044fb65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.036+0000 7f3344744b80 1 -- 172.21.15.7:0/3537300069 shutdown_connections 2024-06-13T07:10:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.036+0000 7f3344744b80 1 -- 172.21.15.7:0/3537300069 wait complete. 2024-06-13T07:10:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.036+0000 7f3344744b80 1 -- start start 2024-06-13T07:10:24.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.037+0000 7f3344744b80 1 Processor -- start 2024-06-13T07:10:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fc3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 0x556044fc3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fbfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b2e50 con 0x556044fbb300 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b42c0 con 0x556044f2ce40 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b3520 con 0x556044fbaa30 2024-06-13T07:10:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.038+0000 7f3344744b80 10 client.0 _ll_register_callbacks cb 0x556044e9c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f332a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 0x556044fc3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f332a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 0x556044fc3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55454/0 (socket says 172.21.15.7:55454) 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f332a000700 1 -- 172.21.15.7:0/2154309294 learned_addr learned my addr 172.21.15.7:0/2154309294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fc3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3333400700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fbfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 -- 172.21.15.7:0/2154309294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 msgr2=0x556044fc3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 0x556044fc3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 -- 172.21.15.7:0/2154309294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 msgr2=0x556044fbfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fbfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3332a00700 1 -- 172.21.15.7:0/2154309294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556044fc0400 con 0x556044f2ce40 2024-06-13T07:10:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.039+0000 7f3333400700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fbfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.040+0000 7f3332a00700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fc3020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x556044fcd2c0 tx=0x7f332400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2aae8917a6c629b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.040+0000 7f3321600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.040+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324004830 con 0x556044f2ce40 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.040+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f332c00bdc0 con 0x556044f2ce40 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.040+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f33240143e0 con 0x556044f2ce40 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.041+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f332403b050 con 0x556044f2ce40 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.041+0000 7f3344744b80 10 client.27473 fetch_fsmap learned FSMap version 42 2024-06-13T07:10:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.041+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556045046c90 con 0x556044f2ce40 2024-06-13T07:10:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c00bdc0 con 0x556044f2ce40 2024-06-13T07:10:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f332402f400 con 0x556044f2ce40 2024-06-13T07:10:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 10 client.27473 fetch_fsmap finished waiting for FSMap version 42 2024-06-13T07:10:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 20 client.27473 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5560450473a0 0x5560450497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 4 client.27473 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:10:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.042+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556044f0a580 con 0x5560450473a0 2024-06-13T07:10:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.043+0000 7f332a000700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5560450473a0 0x5560450497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:24.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f332a000700 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5560450473a0 0x5560450497c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4f2eb793cdc302 server_cookie=f65129a4bbc9477e in_seq=0 out_seq=0 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3321600700 10 client.27473 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3321600700 1 -- 172.21.15.7:0/2154309294 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556044f0a580 con 0x5560450473a0 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3321600700 10 client.27473 handle_command_reply: tid=0 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3344744b80 1 client.27473 shutdown 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 msgr2=0x556044fc3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fc3020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x556044fcd2c0 tx=0x7f332400b040 comp rx=0 tx=0).stop 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 shutdown_connections 2024-06-13T07:10:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.044+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5560450473a0 0x5560450497c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556044fbb300 0x556044fbfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556044f2ce40 0x556044fc3020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 --2- 172.21.15.7:0/2154309294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556044fbaa30 0x556044fc3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 >> 172.21.15.7:0/2154309294 conn(0x556044edd170 msgr2=0x556044faaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3321600700 0 client.27473 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 shutdown_connections 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 1 -- 172.21.15.7:0/2154309294 wait complete. 2024-06-13T07:10:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.045+0000 7f3344744b80 20 client.27473 trim_cache size 0 max 16384 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "645669b7-cf5e-4483-a2e2-bec73910897a": { 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "645669b7-cf5e-4483-a2e2-bec73910897a", 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:10:24.047 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.046+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 msgr2=0x7f33180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.046+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 0x7f33180a50e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3324009c10 tx=0x7f3324016040 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.046+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 msgr2=0x7f333406cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.046+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f333406cec0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f332c009810 tx=0x7f332c00fa80 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 shutdown_connections 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f33180a2c20 0x7f33180a50e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33341084e0 0x7f333406cec0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f333406a170 0x7f3334073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 --2- 172.21.15.7:0/2129077311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333406ab10 0x7f3334074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 >> 172.21.15.7:0/2129077311 conn(0x7f33340782c0 msgr2=0x7f33340ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 shutdown_connections 2024-06-13T07:10:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:10:24.047+0000 7f3339e00700 1 -- 172.21.15.7:0/2129077311 wait complete. 2024-06-13T07:10:24.944 INFO:tasks.cephfs.filesystem:scrub status for tag:645669b7-cf5e-4483-a2e2-bec73910897a - {'path': '/', 'tag': '645669b7-cf5e-4483-a2e2-bec73910897a', 'options': 'recursive,force'} 2024-06-13T07:10:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:25 smithi189 ceph-mon[36616]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.5 MiB/s wr, 573 op/s 2024-06-13T07:10:25.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:25 smithi175 ceph-mon[36886]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.5 MiB/s wr, 573 op/s 2024-06-13T07:10:25.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:25 smithi007 ceph-mon[32077]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.5 MiB/s wr, 573 op/s 2024-06-13T07:10:26.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:26 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:26.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:26 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:26.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:26 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:27.768 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:27 smithi175 ceph-mon[36886]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-06-13T07:10:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:27 smithi189 ceph-mon[36616]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-06-13T07:10:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 865.0 s, 472.2 tps, lat 218.209 ms stddev 266.421 2024-06-13T07:10:27.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:27 smithi007 ceph-mon[32077]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.4 MiB/s wr, 460 op/s 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.082+0000 7f75ce400640 1 -- 172.21.15.7:0/2860767405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c80743a0 msgr2=0x7f75c8074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.082+0000 7f75ce400640 1 --2- 172.21.15.7:0/2860767405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c80743a0 0x7f75c8074800 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f75b8013f10 tx=0x7f75b8032a50 comp rx=0 tx=0).stop 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.083+0000 7f75ce400640 1 -- 172.21.15.7:0/2860767405 shutdown_connections 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.083+0000 7f75ce400640 1 --2- 172.21.15.7:0/2860767405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c8109890 0x7f75c810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.083+0000 7f75ce400640 1 --2- 172.21.15.7:0/2860767405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c80743a0 0x7f75c8074800 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f75b8013f10 tx=0x7f75b8032a50 comp rx=0 tx=0).stop 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.083+0000 7f75ce400640 1 --2- 172.21.15.7:0/2860767405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8075980 0x7f75c8073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.083+0000 7f75ce400640 1 -- 172.21.15.7:0/2860767405 >> 172.21.15.7:0/2860767405 conn(0x7f75c80fcf70 msgr2=0x7f75c80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.084+0000 7f75ce400640 1 -- 172.21.15.7:0/2860767405 shutdown_connections 2024-06-13T07:10:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.084+0000 7f75ce400640 1 -- 172.21.15.7:0/2860767405 wait complete. 2024-06-13T07:10:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.085+0000 7f75ce400640 1 Processor -- start 2024-06-13T07:10:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.085+0000 7f75ce400640 1 -- start start 2024-06-13T07:10:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 0x7f75c8195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 0x7f75c8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c819b250 0x7f75c819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81040f0 con 0x7f75c819b250 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8103f70 con 0x7f75c8075980 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75ce400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8104270 con 0x7f75c8109890 2024-06-13T07:10:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 0x7f75c8195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 0x7f75c8195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41656/0 (socket says 172.21.15.7:41656) 2024-06-13T07:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.086+0000 7f75c7400640 1 -- 172.21.15.7:0/235281128 learned_addr learned my addr 172.21.15.7:0/235281128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c7e00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c819b250 0x7f75c819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 0x7f75c8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 -- 172.21.15.7:0/235281128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 msgr2=0x7f75c8195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 0x7f75c8195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 -- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c819b250 msgr2=0x7f75c819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c819b250 0x7f75c819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.087+0000 7f75c6a00640 1 -- 172.21.15.7:0/235281128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c819dce0 con 0x7f75c8109890 2024-06-13T07:10:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.091+0000 7f75c6a00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 0x7f75c8196200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75b8008000 tx=0x7f75b8005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.092+0000 7f75bfe00640 1 -- 172.21.15.7:0/235281128 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b803a050 con 0x7f75c8109890 2024-06-13T07:10:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.092+0000 7f75bfe00640 1 -- 172.21.15.7:0/235281128 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b80053a0 con 0x7f75c8109890 2024-06-13T07:10:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.092+0000 7f75bfe00640 1 -- 172.21.15.7:0/235281128 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8003b50 con 0x7f75c8109890 2024-06-13T07:10:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.092+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c819df70 con 0x7f75c8109890 2024-06-13T07:10:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.092+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c819e3a0 con 0x7f75c8109890 2024-06-13T07:10:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.098+0000 7f75bfe00640 1 -- 172.21.15.7:0/235281128 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f75b8003cf0 con 0x7f75c8109890 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.099+0000 7f75d1239740 1 Processor -- start 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.099+0000 7f75d1239740 1 -- start start 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 0x55880e92c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e92ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55880e88a420 con 0x55880e9275f0 2024-06-13T07:10:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55880e831c50 con 0x55880e926c00 2024-06-13T07:10:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75d1239740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55880e8863e0 con 0x55880e92c760 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41662/0 (socket says 172.21.15.7:41662) 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75c7e00640 1 -- 172.21.15.7:0/3951796176 learned_addr learned my addr 172.21.15.7:0/3951796176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75c7400640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 0x55880e92c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.100+0000 7f75c6a00640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e92ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75bfe00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 0x7f75ac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7e00640 1 -- 172.21.15.7:0/3951796176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 msgr2=0x55880e92ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7e00640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e92ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7e00640 1 -- 172.21.15.7:0/3951796176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 msgr2=0x55880e92c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7e00640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 0x55880e92c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7e00640 1 -- 172.21.15.7:0/3951796176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880e59e210 con 0x55880e926c00 2024-06-13T07:10:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75bfe00640 1 -- 172.21.15.7:0/235281128 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f75b80f7110 con 0x7f75c8109890 2024-06-13T07:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.101+0000 7f75c7400640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 0x55880e92c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.102+0000 7f75c7e00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 0x7f75ac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.103+0000 7f75c7e00640 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009f40 tx=0x7f75c000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4c9a09f6fb05ea server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.104+0000 7f75bcc00640 1 -- 172.21.15.7:0/3951796176 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00077e0 con 0x55880e926c00 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.104+0000 7f75bcc00640 1 -- 172.21.15.7:0/3951796176 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c0018e10 con 0x55880e926c00 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.104+0000 7f75bcc00640 1 -- 172.21.15.7:0/3951796176 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00077e0 con 0x55880e926c00 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 -- 172.21.15.7:0/3951796176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 msgr2=0x55880e927020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009f40 tx=0x7f75c000c890 comp rx=0 tx=0).stop 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75c7e00640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 0x7f75ac0a51f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009fd0 tx=0x7f75c0019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 -- 172.21.15.7:0/3951796176 shutdown_connections 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e92ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e9275f0 0x55880e92c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 --2- 172.21.15.7:0/3951796176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e927020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.105+0000 7f75d1239740 1 -- 172.21.15.7:0/3951796176 >> 172.21.15.7:0/3951796176 conn(0x55880e8eefb0 msgr2=0x55880e922680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.106+0000 7f75d1239740 1 -- 172.21.15.7:0/3951796176 shutdown_connections 2024-06-13T07:10:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.107+0000 7f75d1239740 1 -- 172.21.15.7:0/3951796176 wait complete. 2024-06-13T07:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.108+0000 7f75d1239740 1 -- start start 2024-06-13T07:10:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.108+0000 7f75d1239740 1 Processor -- start 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e931470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e92c760 0x55880e9384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55880e92f710 con 0x55880e92c760 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55880e8450b0 con 0x55880e926c00 2024-06-13T07:10:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55880e92f890 con 0x55880e9275f0 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75d1239740 10 client.0 _ll_register_callbacks cb 0x55880e830f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41678/0 (socket says 172.21.15.7:41678) 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.109+0000 7f75c7400640 1 -- 172.21.15.7:0/1655146523 learned_addr learned my addr 172.21.15.7:0/1655146523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.110+0000 7f75c7400640 1 -- 172.21.15.7:0/1655146523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 msgr2=0x55880e931470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.110+0000 7f75c7400640 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e931470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.111+0000 7f75c7400640 1 -- 172.21.15.7:0/1655146523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e92c760 msgr2=0x55880e9384b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:10:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.111+0000 7f75c7400640 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e92c760 0x55880e9384b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.111+0000 7f75c7400640 1 -- 172.21.15.7:0/1655146523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880e66a620 con 0x55880e926c00 2024-06-13T07:10:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.111+0000 7f75c7400640 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75b40136e0 tx=0x7f75b4013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178ad24357b9abd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.111+0000 7f75b1600640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:10:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.113+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004310 con 0x55880e926c00 2024-06-13T07:10:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.113+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f75b40119c0 con 0x55880e926c00 2024-06-13T07:10:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.113+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c0018e10 con 0x55880e926c00 2024-06-13T07:10:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.115+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75b40044b0 con 0x55880e926c00 2024-06-13T07:10:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.115+0000 7f75d1239740 10 client.27485 fetch_fsmap learned FSMap version 43 2024-06-13T07:10:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.115+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55880e93b850 con 0x55880e926c00 2024-06-13T07:10:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.116+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c0018e10 con 0x55880e926c00 2024-06-13T07:10:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.116+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f75b401a070 con 0x55880e926c00 2024-06-13T07:10:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.117+0000 7f75d1239740 10 client.27485 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:10:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.117+0000 7f75d1239740 10 client.27485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.117+0000 7f75d1239740 20 client.27485 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.118+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e93cf90 0x55880e93f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.118+0000 7f75d1239740 4 client.27485 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.118+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55880e897b50 con 0x55880e93cf90 2024-06-13T07:10:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.119+0000 7f75c6a00640 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e93cf90 0x55880e93f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.121+0000 7f75b1600640 10 client.27485 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.121+0000 7f75c6a00640 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e93cf90 0x55880e93f3a0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ebc3f11d6e5f81 server_cookie=85cab36be66a345a in_seq=0 out_seq=0 2024-06-13T07:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.121+0000 7f75b1600640 1 -- 172.21.15.7:0/1655146523 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55880e897b50 con 0x55880e93cf90 2024-06-13T07:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.121+0000 7f75b1600640 10 client.27485 handle_command_reply: tid=0 2024-06-13T07:10:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.122+0000 7f75d1239740 1 client.27485 shutdown 2024-06-13T07:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 msgr2=0x55880e930f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75b40136e0 tx=0x7f75b4013ba0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 shutdown_connections 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e93cf90 0x55880e93f3a0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e92c760 0x55880e9384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e931470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 --2- 172.21.15.7:0/1655146523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e926c00 0x55880e930f30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75b1600640 0 client.27485 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.123+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 >> 172.21.15.7:0/1655146523 conn(0x55880e8eefb0 msgr2=0x55880e921840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.124+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 shutdown_connections 2024-06-13T07:10:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.126+0000 7f75d1239740 1 -- 172.21.15.7:0/1655146523 wait complete. 2024-06-13T07:10:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.127+0000 7f75d1239740 20 client.27485 trim_cache size 0 max 16384 2024-06-13T07:10:28.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.183+0000 7f75d1239740 1 Processor -- start 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.183+0000 7f75d1239740 1 -- start start 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e928040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 0x55880e925d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81ab1e0 con 0x55880e926c00 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81a8a20 con 0x55880e92c760 2024-06-13T07:10:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75d1239740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81a6870 con 0x55880e9275f0 2024-06-13T07:10:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 0x55880e925d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75c7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 0x55880e925d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41694/0 (socket says 172.21.15.7:41694) 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e928040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.184+0000 7f75c6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.185+0000 7f75c6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:55488/0 (socket says 172.21.15.7:55488) 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.185+0000 7f75c6a00640 1 -- 172.21.15.7:0/1030245472 learned_addr learned my addr 172.21.15.7:0/1030245472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.186+0000 7f75c6a00640 1 -- 172.21.15.7:0/1030245472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 msgr2=0x55880e925d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.186+0000 7f75c6a00640 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 0x55880e925d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.186+0000 7f75c6a00640 1 -- 172.21.15.7:0/1030245472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 msgr2=0x55880e928040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.186+0000 7f75c6a00640 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e928040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.186+0000 7f75c6a00640 1 -- 172.21.15.7:0/1030245472 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880e91dd40 con 0x55880e9275f0 2024-06-13T07:10:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.187+0000 7f75c6a00640 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f75b80c32d0 tx=0x7f75b80f4740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb037296d2a63843 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.189+0000 7f75b3e00640 1 -- 172.21.15.7:0/1030245472 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b80c3110 con 0x55880e9275f0 2024-06-13T07:10:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.189+0000 7f75b3e00640 1 -- 172.21.15.7:0/1030245472 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b80c3cd0 con 0x55880e9275f0 2024-06-13T07:10:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.189+0000 7f75b3e00640 1 -- 172.21.15.7:0/1030245472 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b80c2f30 con 0x55880e9275f0 2024-06-13T07:10:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 -- 172.21.15.7:0/1030245472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 msgr2=0x55880e938470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f75b80c32d0 tx=0x7f75b80f4740 comp rx=0 tx=0).stop 2024-06-13T07:10:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 -- 172.21.15.7:0/1030245472 shutdown_connections 2024-06-13T07:10:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e92c760 0x55880e925d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e9275f0 0x55880e938470 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 --2- 172.21.15.7:0/1030245472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e928040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.190+0000 7f75d1239740 1 -- 172.21.15.7:0/1030245472 >> 172.21.15.7:0/1030245472 conn(0x55880e8eefb0 msgr2=0x55880e925650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.192+0000 7f75d1239740 1 -- 172.21.15.7:0/1030245472 shutdown_connections 2024-06-13T07:10:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.192+0000 7f75d1239740 1 -- 172.21.15.7:0/1030245472 wait complete. 2024-06-13T07:10:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.192+0000 7f75d1239740 1 -- start start 2024-06-13T07:10:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.193+0000 7f75d1239740 1 Processor -- start 2024-06-13T07:10:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.195+0000 7f75d1239740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e9336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.195+0000 7f75d1239740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 0x55880e933c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75d1239740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e936bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75c6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 0x55880e933c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75c6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 0x55880e933c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:41702/0 (socket says 172.21.15.7:41702) 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75c6a00640 1 -- 172.21.15.7:0/1119476418 learned_addr learned my addr 172.21.15.7:0/1119476418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81a6ee0 con 0x55880e926c00 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81a8350 con 0x55880e9275f0 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75c81a75b0 con 0x55880e92c760 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75d1239740 10 client.0 _ll_register_callbacks cb 0x7f75b4009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75c7e00640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e936bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.196+0000 7f75c7400640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e9336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.197+0000 7f75c7e00640 1 -- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 msgr2=0x55880e933c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.197+0000 7f75c7e00640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 0x55880e933c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.197+0000 7f75c7e00640 1 -- 172.21.15.7:0/1119476418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 msgr2=0x55880e9336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.197+0000 7f75c7e00640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e9336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.197+0000 7f75c7e00640 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880e9371a0 con 0x55880e92c760 2024-06-13T07:10:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.198+0000 7f75c7400640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e9336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.199+0000 7f75c7e00640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e936bb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f75c0008010 tx=0x7f75c0018ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15abc8ec2b7971bf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.199+0000 7f75b1600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.199+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0031070 con 0x55880e92c760 2024-06-13T07:10:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.200+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f75c0004040 con 0x55880e92c760 2024-06-13T07:10:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.200+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b80c3cd0 con 0x55880e92c760 2024-06-13T07:10:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.202+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75c003c050 con 0x55880e92c760 2024-06-13T07:10:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.202+0000 7f75d1239740 10 client.27373 fetch_fsmap learned FSMap version 43 2024-06-13T07:10:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.202+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55880e938520 con 0x55880e92c760 2024-06-13T07:10:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.202+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b80c3cd0 con 0x55880e92c760 2024-06-13T07:10:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.203+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f75c002a050 con 0x55880e92c760 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 10 client.27373 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 20 client.27373 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e9b8f80 0x55880e9bb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 4 client.27373 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55880e891ea0 con 0x55880e9b8f80 2024-06-13T07:10:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.205+0000 7f75c7400640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e9b8f80 0x55880e9bb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.207+0000 7f75b1600640 10 client.27373 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.207+0000 7f75c7400640 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e9b8f80 0x55880e9bb370 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862484c676824a05 server_cookie=a2d1b01dddbeb53a in_seq=0 out_seq=0 2024-06-13T07:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75b1600640 1 -- 172.21.15.7:0/1119476418 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55880e891ea0 con 0x55880e9b8f80 2024-06-13T07:10:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75b1600640 10 client.27373 handle_command_reply: tid=0 2024-06-13T07:10:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75d1239740 1 client.27373 shutdown 2024-06-13T07:10:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 msgr2=0x55880e936bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e936bb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f75c0008010 tx=0x7f75c0018ae0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.259+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 shutdown_connections 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55880e9b8f80 0x55880e9bb370 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55880e92c760 0x55880e936bb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55880e9275f0 0x55880e933c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 --2- 172.21.15.7:0/1119476418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55880e926c00 0x55880e9336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 >> 172.21.15.7:0/1119476418 conn(0x55880e8eefb0 msgr2=0x55880e920fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75b1600640 0 client.27373 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 shutdown_connections 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 1 -- 172.21.15.7:0/1119476418 wait complete. 2024-06-13T07:10:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.260+0000 7f75d1239740 20 client.27373 trim_cache size 0 max 16384 2024-06-13T07:10:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:10:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:10:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:10:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.263+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 msgr2=0x7f75ac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.263+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 0x7f75ac0a51f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009fd0 tx=0x7f75c0019000 comp rx=0 tx=0).stop 2024-06-13T07:10:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.263+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 msgr2=0x7f75c8196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.263+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 0x7f75c8196200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75b8008000 tx=0x7f75b8005020 comp rx=0 tx=0).stop 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 shutdown_connections 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f75ac0a2d30 0x7f75ac0a51f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c819b250 0x7f75c819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75c8109890 0x7f75c8196200 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 --2- 172.21.15.7:0/235281128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c8075980 0x7f75c8195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 >> 172.21.15.7:0/235281128 conn(0x7f75c80fcf70 msgr2=0x7f75c8104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 shutdown_connections 2024-06-13T07:10:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:28.264+0000 7f75ce400640 1 -- 172.21.15.7:0/235281128 wait complete. 2024-06-13T07:10:28.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:28 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:10:28.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:28 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:10:28.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:28 smithi189 ceph-mon[36616]: scrub summary: idle 2024-06-13T07:10:28.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:28 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:10:28.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:28 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:10:28.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:28 smithi175 ceph-mon[36886]: scrub summary: idle 2024-06-13T07:10:28.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:28 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:10:28.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:28 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:10:28.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:28 smithi007 ceph-mon[32077]: scrub summary: idle 2024-06-13T07:10:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:29 smithi189 ceph-mon[36616]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 550 op/s 2024-06-13T07:10:29.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:29 smithi175 ceph-mon[36886]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 550 op/s 2024-06-13T07:10:29.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:29 smithi007 ceph-mon[32077]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.2 MiB/s wr, 550 op/s 2024-06-13T07:10:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:31 smithi189 ceph-mon[36616]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-06-13T07:10:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:31 smithi175 ceph-mon[36886]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-06-13T07:10:31.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:31 smithi007 ceph-mon[32077]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-06-13T07:10:32.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 870.0 s, 470.2 tps, lat 214.556 ms stddev 268.907 2024-06-13T07:10:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:33 smithi189 ceph-mon[36616]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.7 MiB/s wr, 466 op/s 2024-06-13T07:10:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:33 smithi175 ceph-mon[36886]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.7 MiB/s wr, 466 op/s 2024-06-13T07:10:33.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:33 smithi007 ceph-mon[32077]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.7 MiB/s wr, 466 op/s 2024-06-13T07:10:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:35 smithi189 ceph-mon[36616]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-06-13T07:10:35.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:35 smithi175 ceph-mon[36886]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-06-13T07:10:35.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:35 smithi007 ceph-mon[32077]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-06-13T07:10:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:37 smithi189 ceph-mon[36616]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-06-13T07:10:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:37 smithi175 ceph-mon[36886]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-06-13T07:10:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 875.0 s, 535.4 tps, lat 183.427 ms stddev 226.241 2024-06-13T07:10:37.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:37 smithi007 ceph-mon[32077]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-06-13T07:10:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:39 smithi189 ceph-mon[36616]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 487 op/s 2024-06-13T07:10:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:39 smithi175 ceph-mon[36886]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 487 op/s 2024-06-13T07:10:39.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:39 smithi007 ceph-mon[32077]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.2 MiB/s wr, 487 op/s 2024-06-13T07:10:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 -- 172.21.15.7:0/434062137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 msgr2=0x7f5cd80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 --2- 172.21.15.7:0/434062137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd80787e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009860 tx=0x7f5cc402a340 comp rx=0 tx=0).stop 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 -- 172.21.15.7:0/434062137 shutdown_connections 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 --2- 172.21.15.7:0/434062137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd80787e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 --2- 172.21.15.7:0/434062137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd80775a0 0x7f5cd8077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 --2- 172.21.15.7:0/434062137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd8076360 0x7f5cd8076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 -- 172.21.15.7:0/434062137 >> 172.21.15.7:0/434062137 conn(0x7f5cd80ff1c0 msgr2=0x7f5cd81015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.479+0000 7f5cdec00640 1 -- 172.21.15.7:0/434062137 shutdown_connections 2024-06-13T07:10:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.480+0000 7f5cdec00640 1 -- 172.21.15.7:0/434062137 wait complete. 2024-06-13T07:10:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.483+0000 7f5cdec00640 1 Processor -- start 2024-06-13T07:10:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.484+0000 7f5cdec00640 1 -- start start 2024-06-13T07:10:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.484+0000 7f5cdec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 0x7f5cd819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.485+0000 7f5cdec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 0x7f5cd819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cdec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd81a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cdec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8113cc0 con 0x7f5cd8076360 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cdec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8113b40 con 0x7f5cd8077f40 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cdec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8113e40 con 0x7f5cd80775a0 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.485+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 0x7f5cd819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 0x7f5cd819e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53912/0 (socket says 172.21.15.7:53912) 2024-06-13T07:10:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.486+0000 7f5cd7e00640 1 -- 172.21.15.7:0/2319500868 learned_addr learned my addr 172.21.15.7:0/2319500868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 0x7f5cd819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cdcc00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd81a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 -- 172.21.15.7:0/2319500868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 msgr2=0x7f5cd81a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd81a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 -- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 msgr2=0x7f5cd819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 0x7f5cd819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.487+0000 7f5cd7400640 1 -- 172.21.15.7:0/2319500868 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd81a5e40 con 0x7f5cd80775a0 2024-06-13T07:10:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.488+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 0x7f5cd819e7b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800d6d0 tx=0x7f5cc800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.488+0000 7f5cd4c00640 1 -- 172.21.15.7:0/2319500868 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8017070 con 0x7f5cd80775a0 2024-06-13T07:10:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.489+0000 7f5cd4c00640 1 -- 172.21.15.7:0/2319500868 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8010460 con 0x7f5cd80775a0 2024-06-13T07:10:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.489+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd81a6130 con 0x7f5cd80775a0 2024-06-13T07:10:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.489+0000 7f5cd4c00640 1 -- 172.21.15.7:0/2319500868 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800f620 con 0x7f5cd80775a0 2024-06-13T07:10:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.489+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd81a66d0 con 0x7f5cd80775a0 2024-06-13T07:10:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.493+0000 7f5ce1a42740 1 Processor -- start 2024-06-13T07:10:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.496+0000 7f5cd4c00640 1 -- 172.21.15.7:0/2319500868 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5cc801a070 con 0x7f5cd80775a0 2024-06-13T07:10:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 -- start start 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e1d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 0x557806e220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 0x557806e24bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557806d80420 con 0x557806e1cc00 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557806d27c50 con 0x557806e1d5f0 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.498+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557806d7c3e0 con 0x557806e22760 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.499+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e1d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.499+0000 7f5cdcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 0x557806e24bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.499+0000 7f5cdcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 0x557806e24bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:39640/0 (socket says 172.21.15.7:39640) 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.499+0000 7f5cdcc00640 1 -- 172.21.15.7:0/2196279351 learned_addr learned my addr 172.21.15.7:0/2196279351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.499+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 0x557806e220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7400640 1 -- 172.21.15.7:0/2196279351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 msgr2=0x557806e24bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd4c00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 0x7f5cbc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 0x557806e24bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7400640 1 -- 172.21.15.7:0/2196279351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 msgr2=0x557806e1d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e1d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7400640 1 -- 172.21.15.7:0/2196279351 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557806a94210 con 0x557806e1d5f0 2024-06-13T07:10:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.500+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e1d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5cd4c00640 1 -- 172.21.15.7:0/2319500868 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5cc80ce970 con 0x7f5cd80775a0 2024-06-13T07:10:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5cd7400640 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 0x557806e220f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cc100 tx=0x7f5cc80d7650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e782076475625028 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5cdcc00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 0x7f5cbc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5ccd200640 1 -- 172.21.15.7:0/2196279351 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80d3040 con 0x557806e1d5f0 2024-06-13T07:10:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5ccd200640 1 -- 172.21.15.7:0/2196279351 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8010610 con 0x557806e1d5f0 2024-06-13T07:10:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.501+0000 7f5ccd200640 1 -- 172.21.15.7:0/2196279351 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc809a9d0 con 0x557806e1d5f0 2024-06-13T07:10:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.502+0000 7f5ce1a42740 1 -- 172.21.15.7:0/2196279351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 msgr2=0x557806e220f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.502+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 0x557806e220f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cc100 tx=0x7f5cc80d7650 comp rx=0 tx=0).stop 2024-06-13T07:10:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5ce1a42740 1 -- 172.21.15.7:0/2196279351 shutdown_connections 2024-06-13T07:10:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e22760 0x557806e24bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1d5f0 0x557806e220f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/2196279351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e1d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5ce1a42740 1 -- 172.21.15.7:0/2196279351 >> 172.21.15.7:0/2196279351 conn(0x557806de4fb0 msgr2=0x557806e18680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.506+0000 7f5cdcc00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 0x7f5cbc0a51c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40095f0 tx=0x7f5cc4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.508+0000 7f5ce1a42740 1 -- 172.21.15.7:0/2196279351 shutdown_connections 2024-06-13T07:10:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.508+0000 7f5ce1a42740 1 -- 172.21.15.7:0/2196279351 wait complete. 2024-06-13T07:10:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.508+0000 7f5ce1a42740 1 -- start start 2024-06-13T07:10:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.509+0000 7f5ce1a42740 1 Processor -- start 2024-06-13T07:10:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.509+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e37f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.509+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806e3f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:39648/0 (socket says 172.21.15.7:39648) 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e37f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e37f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.510+0000 7f5cd7400640 1 -- 172.21.15.7:0/807104899 learned_addr learned my addr 172.21.15.7:0/807104899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557806e366c0 con 0x557806e1cc00 2024-06-13T07:10:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557806d3b0b0 con 0x557806e22760 2024-06-13T07:10:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557806e36840 con 0x557806e1d5f0 2024-06-13T07:10:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5ce1a42740 10 client.0 _ll_register_callbacks cb 0x557806d26f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5cd7400640 1 -- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 msgr2=0x557806e3f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5cd7400640 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806e3f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5cd7400640 1 -- 172.21.15.7:0/807104899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 msgr2=0x557806e37f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5cd7400640 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e37f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.511+0000 7f5cd7400640 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557806b60620 con 0x557806e1d5f0 2024-06-13T07:10:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.512+0000 7f5cd7400640 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cc0d0 tx=0x7f5cc8010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f12e6388746cd8a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.512+0000 7f5cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.512+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc809a7f0 con 0x557806e1d5f0 2024-06-13T07:10:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.512+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557806d8db50 con 0x557806e1d5f0 2024-06-13T07:10:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.512+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5cc8099190 con 0x557806e1d5f0 2024-06-13T07:10:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.518+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5cc809a9d0 con 0x557806e1d5f0 2024-06-13T07:10:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.519+0000 7f5ce1a42740 10 client.27385 fetch_fsmap learned FSMap version 43 2024-06-13T07:10:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.519+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557806e366c0 con 0x557806e1d5f0 2024-06-13T07:10:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.519+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557806d8db50 con 0x557806e1d5f0 2024-06-13T07:10:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f5cc80d9040 con 0x557806e1d5f0 2024-06-13T07:10:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 10 client.27385 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:10:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 10 client.27385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 20 client.27385 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806e45e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 4 client.27385 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.520+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557806d87ea0 con 0x557806e43a50 2024-06-13T07:10:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.521+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806e45e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.521+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806e45e40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4fa6277d0cec9e server_cookie=d90e362083d1b607 in_seq=0 out_seq=0 2024-06-13T07:10:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.521+0000 7f5cc1600640 10 client.27385 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.522+0000 7f5cc1600640 1 -- 172.21.15.7:0/807104899 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557806d87ea0 con 0x557806e43a50 2024-06-13T07:10:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.522+0000 7f5cc1600640 10 client.27385 handle_command_reply: tid=0 2024-06-13T07:10:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.523+0000 7f5ce1a42740 1 client.27385 shutdown 2024-06-13T07:10:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 msgr2=0x557806e38450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cc0d0 tx=0x7f5cc8010d80 comp rx=0 tx=0).stop 2024-06-13T07:10:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 shutdown_connections 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806e45e40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5cc1600640 0 client.27385 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806e3f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e38450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/807104899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e37f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.527+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 >> 172.21.15.7:0/807104899 conn(0x557806de4fb0 msgr2=0x557806e17840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.528+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 shutdown_connections 2024-06-13T07:10:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.529+0000 7f5ce1a42740 1 -- 172.21.15.7:0/807104899 wait complete. 2024-06-13T07:10:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.530+0000 7f5ce1a42740 20 client.27385 trim_cache size 0 max 16384 2024-06-13T07:10:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.564+0000 7f5ce1a42740 1 Processor -- start 2024-06-13T07:10:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.564+0000 7f5ce1a42740 1 -- start start 2024-06-13T07:10:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.564+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e22760 0x557806e1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81aaca0 con 0x557806e22760 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a84e0 con 0x557806e1cc00 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8073500 con 0x557806e1d5f0 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50436/0 (socket says 172.21.15.7:50436) 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5cd7e00640 1 -- 172.21.15.7:0/4267627220 learned_addr learned my addr 172.21.15.7:0/4267627220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5cd7400640 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.565+0000 7f5cdcc00640 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e22760 0x557806e1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 -- 172.21.15.7:0/4267627220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 msgr2=0x557806e3f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 -- 172.21.15.7:0/4267627220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e22760 msgr2=0x557806e1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e22760 0x557806e1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 -- 172.21.15.7:0/4267627220 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557806e461b0 con 0x557806e1cc00 2024-06-13T07:10:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x557806e1de50 tx=0x7f5cd0009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7deefd06b2918272 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cc3e00640 1 -- 172.21.15.7:0/4267627220 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0011980 con 0x557806e1cc00 2024-06-13T07:10:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.566+0000 7f5cc3e00640 1 -- 172.21.15.7:0/4267627220 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd001f430 con 0x557806e1cc00 2024-06-13T07:10:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.567+0000 7f5cc3e00640 1 -- 172.21.15.7:0/4267627220 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd001e680 con 0x557806e1cc00 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.567+0000 7f5ce1a42740 1 -- 172.21.15.7:0/4267627220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 msgr2=0x557806e1e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x557806e1de50 tx=0x7f5cd0009720 comp rx=0 tx=0).stop 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 -- 172.21.15.7:0/4267627220 shutdown_connections 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e22760 0x557806e1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/4267627220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e1cc00 0x557806e1e970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.568+0000 7f5ce1a42740 1 -- 172.21.15.7:0/4267627220 >> 172.21.15.7:0/4267627220 conn(0x557806de4fb0 msgr2=0x557806e18100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.569+0000 7f5ce1a42740 1 -- 172.21.15.7:0/4267627220 shutdown_connections 2024-06-13T07:10:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.569+0000 7f5ce1a42740 1 -- 172.21.15.7:0/4267627220 wait complete. 2024-06-13T07:10:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.569+0000 7f5ce1a42740 1 -- start start 2024-06-13T07:10:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.573+0000 7f5ce1a42740 1 Processor -- start 2024-06-13T07:10:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806eac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a69f0 con 0x557806e1cc00 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a7e10 con 0x557806e22760 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a7070 con 0x557806e1d5f0 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.574+0000 7f5ce1a42740 10 client.0 _ll_register_callbacks cb 0x7f5cc8013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cdcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806eac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cdcc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806eac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50442/0 (socket says 172.21.15.7:50442) 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cdcc00640 1 -- 172.21.15.7:0/664813661 learned_addr learned my addr 172.21.15.7:0/664813661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cd7400640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cd7400640 1 -- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 msgr2=0x557806eac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.575+0000 7f5cd7400640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806eac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.576+0000 7f5cd7400640 1 -- 172.21.15.7:0/664813661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 msgr2=0x557806e3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.576+0000 7f5cd7400640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.576+0000 7f5cd7400640 1 -- 172.21.15.7:0/664813661 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557806eacc80 con 0x557806e1d5f0 2024-06-13T07:10:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.576+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.577+0000 7f5cd7400640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3c940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5cc809ae20 tx=0x7f5cc8010cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0756c8bd8806ac9 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.577+0000 7f5cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.577+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc809a7f0 con 0x557806e1d5f0 2024-06-13T07:10:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.577+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f5cc80d2790 con 0x557806e1d5f0 2024-06-13T07:10:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.578+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd001f430 con 0x557806e1d5f0 2024-06-13T07:10:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.581+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5cc8097130 con 0x557806e1d5f0 2024-06-13T07:10:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.581+0000 7f5ce1a42740 10 client.27391 fetch_fsmap learned FSMap version 43 2024-06-13T07:10:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.581+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557806eadf90 con 0x557806e1d5f0 2024-06-13T07:10:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.581+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd001f430 con 0x557806e1d5f0 2024-06-13T07:10:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.581+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 43) ==== 8358+0+0 (secure 0 0 0) 0x7f5cc80d9070 con 0x557806e1d5f0 2024-06-13T07:10:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 10 client.27391 fetch_fsmap finished waiting for FSMap version 43 2024-06-13T07:10:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 10 client.27391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 20 client.27391 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806eb0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 4 client.27391 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557806d057a0 con 0x557806e43a50 2024-06-13T07:10:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.584+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806eb0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.585+0000 7f5cd7e00640 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806eb0770 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a289528cc0f27606 server_cookie=323db7d21ae84f89 in_seq=0 out_seq=0 2024-06-13T07:10:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.585+0000 7f5cc1600640 10 client.27391 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:41.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:41 smithi189 ceph-mon[36616]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-13T07:10:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:41 smithi175 ceph-mon[36886]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-13T07:10:41.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:41 smithi007 ceph-mon[32077]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 MiB/s wr, 480 op/s 2024-06-13T07:10:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.940+0000 7f5cc1600640 1 -- 172.21.15.7:0/664813661 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557806d057a0 con 0x557806e43a50 2024-06-13T07:10:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.940+0000 7f5cc1600640 10 client.27391 handle_command_reply: tid=0 2024-06-13T07:10:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.940+0000 7f5ce1a42740 1 client.27391 shutdown 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 msgr2=0x557806e3c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3c940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5cc809ae20 tx=0x7f5cc8010cf0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 shutdown_connections 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x557806e43a50 0x557806eb0770 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557806e22760 0x557806eac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557806e1d5f0 0x557806e3c940 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 --2- 172.21.15.7:0/664813661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557806e1cc00 0x557806e3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 >> 172.21.15.7:0/664813661 conn(0x557806de4fb0 msgr2=0x557806e37ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.941+0000 7f5cc1600640 0 client.27391 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.942+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 shutdown_connections 2024-06-13T07:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.942+0000 7f5ce1a42740 1 -- 172.21.15.7:0/664813661 wait complete. 2024-06-13T07:10:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.942+0000 7f5ce1a42740 20 client.27391 trim_cache size 0 max 16384 2024-06-13T07:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:10:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:10:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 msgr2=0x7f5cbc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 0x7f5cbc0a51c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40095f0 tx=0x7f5cc4030550 comp rx=0 tx=0).stop 2024-06-13T07:10:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 msgr2=0x7f5cd819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 0x7f5cd819e7b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800d6d0 tx=0x7f5cc800db90 comp rx=0 tx=0).stop 2024-06-13T07:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 shutdown_connections 2024-06-13T07:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f5cbc0a2d00 0x7f5cbc0a51c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd8077f40 0x7f5cd81a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd80775a0 0x7f5cd819e7b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 --2- 172.21.15.7:0/2319500868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cd8076360 0x7f5cd819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.946+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 >> 172.21.15.7:0/2319500868 conn(0x7f5cd80ff1c0 msgr2=0x7f5cd8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.947+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 shutdown_connections 2024-06-13T07:10:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:41.947+0000 7f5cdec00640 1 -- 172.21.15.7:0/2319500868 wait complete. 2024-06-13T07:10:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 880.0 s, 502.1 tps, lat 193.759 ms stddev 233.262 2024-06-13T07:10:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:43 smithi189 ceph-mon[36616]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-13T07:10:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:43 smithi175 ceph-mon[36886]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-13T07:10:43.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:43 smithi007 ceph-mon[32077]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 MiB/s wr, 441 op/s 2024-06-13T07:10:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:45 smithi175 ceph-mon[36886]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-13T07:10:45.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:45 smithi007 ceph-mon[32077]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-13T07:10:46.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:45 smithi189 ceph-mon[36616]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-06-13T07:10:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:47 smithi175 ceph-mon[36886]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-06-13T07:10:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 885.0 s, 426.8 tps, lat 232.193 ms stddev 309.003 2024-06-13T07:10:47.887 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:47 smithi189 ceph-mon[36616]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-06-13T07:10:47.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:47 smithi007 ceph-mon[32077]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-06-13T07:10:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:49 smithi175 ceph-mon[36886]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-13T07:10:49.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:49 smithi007 ceph-mon[32077]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-13T07:10:50.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:49 smithi189 ceph-mon[36616]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-06-13T07:10:51.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:51 smithi007 ceph-mon[32077]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-06-13T07:10:51.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:51 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:51 smithi189 ceph-mon[36616]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-06-13T07:10:52.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:51 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:51 smithi175 ceph-mon[36886]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-06-13T07:10:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:51 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:10:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 890.0 s, 418.4 tps, lat 245.278 ms stddev 288.060 2024-06-13T07:10:53.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:53 smithi007 ceph-mon[32077]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-06-13T07:10:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:53 smithi189 ceph-mon[36616]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-06-13T07:10:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:53 smithi175 ceph-mon[36886]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 424 op/s 2024-06-13T07:10:54.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 -- 172.21.15.7:0/3462673548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8108030 msgr2=0x7f0be8108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 --2- 172.21.15.7:0/3462673548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8108030 0x7f0be8108450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009900 tx=0x7f0bd802a3f0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 -- 172.21.15.7:0/3462673548 shutdown_connections 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 --2- 172.21.15.7:0/3462673548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109be0 0x7f0be8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 --2- 172.21.15.7:0/3462673548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8109220 0x7f0be81096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 --2- 172.21.15.7:0/3462673548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8108030 0x7f0be8108450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.176+0000 7f0bf0000640 1 -- 172.21.15.7:0/3462673548 >> 172.21.15.7:0/3462673548 conn(0x7f0be8076ff0 msgr2=0x7f0be8079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.177+0000 7f0bf0000640 1 -- 172.21.15.7:0/3462673548 shutdown_connections 2024-06-13T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.177+0000 7f0bf0000640 1 -- 172.21.15.7:0/3462673548 wait complete. 2024-06-13T07:10:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.178+0000 7f0bf0000640 1 Processor -- start 2024-06-13T07:10:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.178+0000 7f0bf0000640 1 -- start start 2024-06-13T07:10:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.179+0000 7f0bf0000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.179+0000 7f0bf0000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109220 0x7f0be807ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.179+0000 7f0bf0000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 0x7f0be8081ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bf0000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8113d70 con 0x7f0be8108030 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bf0000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8113bf0 con 0x7f0be8109be0 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bf0000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8113ef0 con 0x7f0be8109220 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58030/0 (socket says 172.21.15.7:58030) 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 -- 172.21.15.7:0/508192941 learned_addr learned my addr 172.21.15.7:0/508192941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bee000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 0x7f0be8081ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0becc00640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109220 0x7f0be807ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 -- 172.21.15.7:0/508192941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109220 msgr2=0x7f0be807ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109220 0x7f0be807ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 -- 172.21.15.7:0/508192941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 msgr2=0x7f0be8081ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 0x7f0be8081ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.180+0000 7f0bed600640 1 -- 172.21.15.7:0/508192941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8082210 con 0x7f0be8108030 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.181+0000 7f0bee000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 0x7f0be8081ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.181+0000 7f0bed600640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8031440 tx=0x7f0bd8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.182+0000 7f0be6000640 1 -- 172.21.15.7:0/508192941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd803f030 con 0x7f0be8108030 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.182+0000 7f0be6000640 1 -- 172.21.15.7:0/508192941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd8003ce0 con 0x7f0be8108030 2024-06-13T07:10:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.182+0000 7f0be6000640 1 -- 172.21.15.7:0/508192941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8035070 con 0x7f0be8108030 2024-06-13T07:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.182+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be80824a0 con 0x7f0be8108030 2024-06-13T07:10:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.182+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81aef00 con 0x7f0be8108030 2024-06-13T07:10:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.188+0000 7f0be6000640 1 -- 172.21.15.7:0/508192941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0bd8042030 con 0x7f0be8108030 2024-06-13T07:10:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.190+0000 7f0bf2d84740 1 Processor -- start 2024-06-13T07:10:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.190+0000 7f0be6000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 0x7f0bd00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0be6000640 1 -- 172.21.15.7:0/508192941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0bd80ef720 con 0x7f0be8108030 2024-06-13T07:10:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 -- start start 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b0b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 0x55b648b100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b648a6e420 con 0x55b648b0ac00 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b648a15c50 con 0x55b648b0b5f0 2024-06-13T07:10:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.191+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b648a6a3e0 con 0x55b648b10760 2024-06-13T07:10:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.192+0000 7f0becc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 0x55b648b100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.192+0000 7f0becc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 0x55b648b100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:54978/0 (socket says 172.21.15.7:54978) 2024-06-13T07:10:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.192+0000 7f0becc00640 1 -- 172.21.15.7:0/2911243575 learned_addr learned my addr 172.21.15.7:0/2911243575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.192+0000 7f0bed600640 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b0b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bee000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 0x7f0bd00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 -- 172.21.15.7:0/2911243575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 msgr2=0x55b648b12bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b12bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 -- 172.21.15.7:0/2911243575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 msgr2=0x55b648b100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 0x55b648b100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 -- 172.21.15.7:0/2911243575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b648782210 con 0x55b648b0ac00 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bee000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 0x7f0bd00a51f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009fd0 tx=0x7f0be0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.193+0000 7f0bed600640 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b0b020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd80bb720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2ba549f036792a server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.194+0000 7f0bd6600640 1 -- 172.21.15.7:0/2911243575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a6050 con 0x55b648b0ac00 2024-06-13T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.194+0000 7f0bd6600640 1 -- 172.21.15.7:0/2911243575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd80bc2e0 con 0x55b648b0ac00 2024-06-13T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.194+0000 7f0bd6600640 1 -- 172.21.15.7:0/2911243575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80b8250 con 0x55b648b0ac00 2024-06-13T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 -- 172.21.15.7:0/2911243575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 msgr2=0x55b648b0b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b0b020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8000c00 tx=0x7f0bd80bb720 comp rx=0 tx=0).stop 2024-06-13T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 -- 172.21.15.7:0/2911243575 shutdown_connections 2024-06-13T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b12bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0b5f0 0x55b648b100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/2911243575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b0b020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.196+0000 7f0bf2d84740 1 -- 172.21.15.7:0/2911243575 >> 172.21.15.7:0/2911243575 conn(0x55b648ad2fb0 msgr2=0x55b648b06680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.197+0000 7f0bf2d84740 1 -- 172.21.15.7:0/2911243575 shutdown_connections 2024-06-13T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.197+0000 7f0bf2d84740 1 -- 172.21.15.7:0/2911243575 wait complete. 2024-06-13T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.197+0000 7f0bf2d84740 1 -- start start 2024-06-13T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.198+0000 7f0bf2d84740 1 Processor -- start 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.198+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b14c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.198+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 0x55b648b151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.198+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b1c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bed600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b14c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b648b134d0 con 0x55b648b0b5f0 2024-06-13T07:10:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b648a290b0 con 0x55b648b0ac00 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b648b13650 con 0x55b648b10760 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bf2d84740 10 client.0 _ll_register_callbacks cb 0x55b648a14f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b1c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b1c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38994/0 (socket says 172.21.15.7:38994) 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0bee000640 1 -- 172.21.15.7:0/552665345 learned_addr learned my addr 172.21.15.7:0/552665345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.199+0000 7f0becc00640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 0x55b648b151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0becc00640 1 -- 172.21.15.7:0/552665345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 msgr2=0x55b648b1c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0becc00640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b1c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0becc00640 1 -- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 msgr2=0x55b648b14c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0becc00640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b14c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0becc00640 1 -- 172.21.15.7:0/552665345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64884e620 con 0x55b648b0b5f0 2024-06-13T07:10:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.201+0000 7f0bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:10:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.202+0000 7f0becc00640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 0x55b648b151a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b648b0b9f0 tx=0x7f0bdc011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611b8314a6090d47 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.202+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc014650 con 0x55b648b0b5f0 2024-06-13T07:10:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.202+0000 7f0bed600640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b14c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.202+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd80bc2e0 con 0x55b648b0b5f0 2024-06-13T07:10:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.203+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0bdc00f040 con 0x55b648b0b5f0 2024-06-13T07:10:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.203+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc014650 con 0x55b648b0b5f0 2024-06-13T07:10:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.204+0000 7f0bf2d84740 10 client.16683 fetch_fsmap learned FSMap version 44 2024-06-13T07:10:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.204+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b648b1f6a0 con 0x55b648b0b5f0 2024-06-13T07:10:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.204+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd80bc2e0 con 0x55b648b0b5f0 2024-06-13T07:10:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f0bdc027a90 con 0x55b648b0b5f0 2024-06-13T07:10:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 10 client.16683 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:10:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 20 client.16683 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b20d80 0x55b648b231c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 4 client.16683 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:10:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.205+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b648a7bb50 con 0x55b648b20d80 2024-06-13T07:10:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.207+0000 7f0bed600640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b20d80 0x55b648b231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.208+0000 7f0bed600640 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b20d80 0x55b648b231c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ae31eddde9ff52 server_cookie=fb9df7c7117848c1 in_seq=0 out_seq=0 2024-06-13T07:10:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.208+0000 7f0bcaa00640 10 client.16683 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.209+0000 7f0bcaa00640 1 -- 172.21.15.7:0/552665345 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b648a7bb50 con 0x55b648b20d80 2024-06-13T07:10:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.209+0000 7f0bcaa00640 10 client.16683 handle_command_reply: tid=0 2024-06-13T07:10:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.209+0000 7f0bf2d84740 1 client.16683 shutdown 2024-06-13T07:10:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 msgr2=0x55b648b151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 0x55b648b151a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55b648b0b9f0 tx=0x7f0bdc011910 comp rx=0 tx=0).stop 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 shutdown_connections 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b20d80 0x55b648b231c0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b10760 0x55b648b1c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0b5f0 0x55b648b151a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/552665345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b14c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bcaa00640 0 client.16683 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.210+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 >> 172.21.15.7:0/552665345 conn(0x55b648ad2fb0 msgr2=0x55b648b05840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.211+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 shutdown_connections 2024-06-13T07:10:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.211+0000 7f0bf2d84740 1 -- 172.21.15.7:0/552665345 wait complete. 2024-06-13T07:10:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.211+0000 7f0bf2d84740 20 client.16683 trim_cache size 0 max 16384 2024-06-13T07:10:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.264+0000 7f0bf2d84740 1 Processor -- start 2024-06-13T07:10:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.264+0000 7f0bf2d84740 1 -- start start 2024-06-13T07:10:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 0x55b648b1c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc012480 con 0x55b648b0ac00 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00fcf0 con 0x55b648b10760 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00dae0 con 0x55b648b069f0 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bee000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:39010/0 (socket says 172.21.15.7:39010) 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.265+0000 7f0bee000640 1 -- 172.21.15.7:0/637217641 learned_addr learned my addr 172.21.15.7:0/637217641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0becc00640 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 0x55b648b1c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0bee000640 1 -- 172.21.15.7:0/637217641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 msgr2=0x55b648b1c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0bee000640 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 0x55b648b1c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0bee000640 1 -- 172.21.15.7:0/637217641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 msgr2=0x55b648b09cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0bee000640 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.266+0000 7f0bee000640 1 -- 172.21.15.7:0/637217641 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b648b07330 con 0x55b648b069f0 2024-06-13T07:10:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.267+0000 7f0becc00640 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 0x55b648b1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:10:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.268+0000 7f0bee000640 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0be0011a90 tx=0x7f0be0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b22d6300badcd8b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.269+0000 7f0bd5c00640 1 -- 172.21.15.7:0/637217641 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004300 con 0x55b648b069f0 2024-06-13T07:10:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.269+0000 7f0bd5c00640 1 -- 172.21.15.7:0/637217641 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be0028440 con 0x55b648b069f0 2024-06-13T07:10:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.269+0000 7f0bd5c00640 1 -- 172.21.15.7:0/637217641 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be001e660 con 0x55b648b069f0 2024-06-13T07:10:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 -- 172.21.15.7:0/637217641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 msgr2=0x55b648b13980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0be0011a90 tx=0x7f0be0014000 comp rx=0 tx=0).stop 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 -- 172.21.15.7:0/637217641 shutdown_connections 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b13980 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b10760 0x55b648b1c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/637217641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b0ac00 0x55b648b09cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 -- 172.21.15.7:0/637217641 >> 172.21.15.7:0/637217641 conn(0x55b648ad2fb0 msgr2=0x55b648b095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 -- 172.21.15.7:0/637217641 shutdown_connections 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.270+0000 7f0bf2d84740 1 -- 172.21.15.7:0/637217641 wait complete. 2024-06-13T07:10:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.271+0000 7f0bf2d84740 1 -- start start 2024-06-13T07:10:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.271+0000 7f0bf2d84740 1 Processor -- start 2024-06-13T07:10:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b1a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b10760 0x55b648b1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00e1b0 con 0x55b648b10760 2024-06-13T07:10:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00f620 con 0x55b648b0ac00 2024-06-13T07:10:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00e880 con 0x55b648b069f0 2024-06-13T07:10:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bf2d84740 10 client.0 _ll_register_callbacks cb 0x7f0bdc009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:10:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.275+0000 7f0bed600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.276+0000 7f0bed600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:39020/0 (socket says 172.21.15.7:39020) 2024-06-13T07:10:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.276+0000 7f0bed600640 1 -- 172.21.15.7:0/339597498 learned_addr learned my addr 172.21.15.7:0/339597498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:10:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.276+0000 7f0becc00640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b1a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.277+0000 7f0bed600640 1 -- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 msgr2=0x55b648b1a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.277+0000 7f0bed600640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b1a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.277+0000 7f0bed600640 1 -- 172.21.15.7:0/339597498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b10760 msgr2=0x55b648b1b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.277+0000 7f0bed600640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b10760 0x55b648b1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.277+0000 7f0bed600640 1 -- 172.21.15.7:0/339597498 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b648b1b730 con 0x55b648b069f0 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.278+0000 7f0bed600640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0bd803b870 tx=0x7f0bd8009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b9676b058400f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.278+0000 7f0becc00640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b1a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.278+0000 7f0bcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:10:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.278+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a6040 con 0x55b648b069f0 2024-06-13T07:10:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.278+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f0bd80ecf90 con 0x55b648b069f0 2024-06-13T07:10:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.279+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be0028440 con 0x55b648b069f0 2024-06-13T07:10:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.280+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0bd80bff90 con 0x55b648b069f0 2024-06-13T07:10:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.280+0000 7f0bf2d84740 10 client.27406 fetch_fsmap learned FSMap version 44 2024-06-13T07:10:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.280+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b648b8be20 con 0x55b648b069f0 2024-06-13T07:10:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be0028440 con 0x55b648b069f0 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f0bd803b460 con 0x55b648b069f0 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 10 client.27406 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 10 client.27406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 20 client.27406 populate_metadata read hostname 'smithi007' 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b8c4a0 0x55b648b8e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 4 client.27406 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.281+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b648a75ea0 con 0x55b648b8c4a0 2024-06-13T07:10:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.282+0000 7f0becc00640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b8c4a0 0x55b648b8e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:10:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.285+0000 7f0bcaa00640 10 client.27406 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.285+0000 7f0becc00640 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b8c4a0 0x55b648b8e8c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d15811e6950f8b server_cookie=f88d2c2b966cd829 in_seq=0 out_seq=0 2024-06-13T07:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.500+0000 7f0bcaa00640 1 -- 172.21.15.7:0/339597498 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b648a75ea0 con 0x55b648b8c4a0 2024-06-13T07:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.500+0000 7f0bcaa00640 10 client.27406 handle_command_reply: tid=0 2024-06-13T07:10:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.500+0000 7f0bf2d84740 1 client.27406 shutdown 2024-06-13T07:10:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 msgr2=0x55b648b19c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0bd803b870 tx=0x7f0bd8009680 comp rx=0 tx=0).stop 2024-06-13T07:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 shutdown_connections 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55b648b8c4a0 0x55b648b8e8c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b648b10760 0x55b648b1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b648b0ac00 0x55b648b1a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 --2- 172.21.15.7:0/339597498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b648b069f0 0x55b648b19c90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.501+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 >> 172.21.15.7:0/339597498 conn(0x55b648ad2fb0 msgr2=0x55b648b04fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.502+0000 7f0bcaa00640 0 client.27406 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.502+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 shutdown_connections 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.502+0000 7f0bf2d84740 1 -- 172.21.15.7:0/339597498 wait complete. 2024-06-13T07:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.502+0000 7f0bf2d84740 20 client.27406 trim_cache size 0 max 16384 2024-06-13T07:10:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:10:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:10:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:10:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 msgr2=0x7f0bd00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 0x7f0bd00a51f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009fd0 tx=0x7f0be0009500 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 msgr2=0x7f0be807a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8031440 tx=0x7f0bd8032ce0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 shutdown_connections 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f0bd00a2d30 0x7f0bd00a51f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0be8109be0 0x7f0be8081ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0be8109220 0x7f0be807ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 --2- 172.21.15.7:0/508192941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0be8108030 0x7f0be807a5f0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8031440 tx=0x7f0bd8032ce0 comp rx=0 tx=0).stop 2024-06-13T07:10:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 >> 172.21.15.7:0/508192941 conn(0x7f0be8076ff0 msgr2=0x7f0be810c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:10:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.504+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 shutdown_connections 2024-06-13T07:10:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:10:55.508+0000 7f0bf0000640 1 -- 172.21.15.7:0/508192941 wait complete. 2024-06-13T07:10:55.725 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:10:55.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:55 smithi007 ceph-mon[32077]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-06-13T07:10:56.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:55 smithi189 ceph-mon[36616]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-06-13T07:10:56.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:55 smithi175 ceph-mon[36886]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-06-13T07:10:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 895.0 s, 415.2 tps, lat 239.352 ms stddev 267.059 2024-06-13T07:10:57.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:57 smithi007 ceph-mon[32077]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-06-13T07:10:57.919 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:57 smithi175 ceph-mon[36886]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-06-13T07:10:58.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:57 smithi189 ceph-mon[36616]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-06-13T07:10:59.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:10:59 smithi007 ceph-mon[32077]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-06-13T07:11:00.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:10:59 smithi189 ceph-mon[36616]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-06-13T07:11:00.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:10:59 smithi175 ceph-mon[36886]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-06-13T07:11:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 -- 172.21.15.7:0/3324121896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc240047d0 msgr2=0x7fbc24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3324121896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc240047d0 0x7fbc24004c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009990 tx=0x7fbc18009ca0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 -- 172.21.15.7:0/3324121896 shutdown_connections 2024-06-13T07:11:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3324121896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc24005190 0x7fbc240a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3324121896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc2409be40 0x7fbc2409c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3324121896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc240047d0 0x7fbc24004c50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.197+0000 7fbc2be00700 1 -- 172.21.15.7:0/3324121896 >> 172.21.15.7:0/3324121896 conn(0x7fbc2400a7a0 msgr2=0x7fbc2400abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.198+0000 7fbc2be00700 1 -- 172.21.15.7:0/3324121896 shutdown_connections 2024-06-13T07:11:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.198+0000 7fbc2be00700 1 -- 172.21.15.7:0/3324121896 wait complete. 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.199+0000 7fbc2be00700 1 Processor -- start 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.199+0000 7fbc2be00700 1 -- start start 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 0x7fbc2412eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2409be40 0x7fbc24132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240a6000 con 0x7fbc2409be40 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240a5e80 con 0x7fbc24005190 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2be00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240a6180 con 0x7fbc240047d0 2024-06-13T07:11:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 0x7fbc2412eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 0x7fbc2412eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51770/0 (socket says 172.21.15.7:51770) 2024-06-13T07:11:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48560/0 (socket says 172.21.15.7:48560) 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.200+0000 7fbc2aa00700 1 -- 172.21.15.7:0/3718226407 learned_addr learned my addr 172.21.15.7:0/3718226407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 -- 172.21.15.7:0/3718226407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 msgr2=0x7fbc2412eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 0x7fbc2412eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 -- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2409be40 msgr2=0x7fbc24132ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2409be40 0x7fbc24132ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 -- 172.21.15.7:0/3718226407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc24133620 con 0x7fbc240047d0 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00e9a0 tx=0x7fbc1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc30a00700 1 -- 172.21.15.7:0/3718226407 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00cbd0 con 0x7fbc240047d0 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc30a00700 1 -- 172.21.15.7:0/3718226407 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc1c00cd70 con 0x7fbc240047d0 2024-06-13T07:11:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc30a00700 1 -- 172.21.15.7:0/3718226407 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c018750 con 0x7fbc240047d0 2024-06-13T07:11:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.201+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc24133910 con 0x7fbc240047d0 2024-06-13T07:11:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.202+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2414c1a0 con 0x7fbc240047d0 2024-06-13T07:11:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.205+0000 7fbc30a00700 1 -- 172.21.15.7:0/3718226407 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbc1c010430 con 0x7fbc240047d0 2024-06-13T07:11:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.207+0000 7fbc30a00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 0x7fbc100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.207+0000 7fbc30a00700 1 -- 172.21.15.7:0/3718226407 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbc1c0cf650 con 0x7fbc240047d0 2024-06-13T07:11:01.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.208+0000 7fbc2a000700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 0x7fbc100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.209+0000 7fbc3c69bb80 1 Processor -- start 2024-06-13T07:11:01.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.210+0000 7fbc3c69bb80 1 -- start start 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.210+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 0x55df98675140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.210+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98688800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.210+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 0x55df9868b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.211+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98688800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.211+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98688800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48562/0 (socket says 172.21.15.7:48562) 2024-06-13T07:11:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.211+0000 7fbc2a000700 1 -- 172.21.15.7:0/1253875335 learned_addr learned my addr 172.21.15.7:0/1253875335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.212+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 0x55df9868b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.212+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df98584a10 con 0x55df98674d60 2024-06-13T07:11:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.212+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55df985c7b00 con 0x55df98688d40 2024-06-13T07:11:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.212+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df985828f0 con 0x55df98675680 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.212+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 0x55df98675140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 -- 172.21.15.7:0/1253875335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 msgr2=0x55df98688800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98688800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 -- 172.21.15.7:0/1253875335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 msgr2=0x55df98675140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 0x55df98675140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 -- 172.21.15.7:0/1253875335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9856ff90 con 0x55df98688d40 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 0x55df98675140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2a000700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 0x7fbc100a5160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbc24092f00 tx=0x7fbc1800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.213+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 0x55df9868b130 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c068f20 tx=0x7fbc2c068aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff59f7db59dad003 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.214+0000 7fbc20c00700 1 -- 172.21.15.7:0/1253875335 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c069810 con 0x55df98688d40 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.214+0000 7fbc20c00700 1 -- 172.21.15.7:0/1253875335 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c07c3a0 con 0x55df98688d40 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.214+0000 7fbc20c00700 1 -- 172.21.15.7:0/1253875335 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c078560 con 0x55df98688d40 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 msgr2=0x55df9868b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 0x55df9868b130 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c068f20 tx=0x7fbc2c068aa0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 shutdown_connections 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98688d40 0x55df9868b130 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98674d60 0x55df98675140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1253875335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98688800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 >> 172.21.15.7:0/1253875335 conn(0x55df985a2170 msgr2=0x55df98671280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 shutdown_connections 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.215+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1253875335 wait complete. 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.216+0000 7fbc3c69bb80 1 -- start start 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.216+0000 7fbc3c69bb80 1 Processor -- start 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98685780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df98563f60 con 0x55df98688d40 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55df985c3ac0 con 0x55df98674d60 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df9868b7d0 con 0x55df98675680 2024-06-13T07:11:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.217+0000 7fbc3c69bb80 10 client.0 _ll_register_callbacks cb 0x55df9855d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98685780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51786/0 (socket says 172.21.15.7:51786) 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2541378714 learned_addr learned my addr 172.21.15.7:0/2541378714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98685780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48578/0 (socket says 172.21.15.7:48578) 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2b400700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2541378714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 msgr2=0x55df98685780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98685780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2541378714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 msgr2=0x55df98685cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2541378714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df985168d0 con 0x55df98674d60 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.218+0000 7fbc2b400700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c09baa0 tx=0x7fbc1c09bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1a0666face5757 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc15600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0a4780 con 0x55df98674d60 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbc1c0109a0 con 0x55df98674d60 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c07c3a0 con 0x55df98674d60 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc1c0a42f0 con 0x55df98674d60 2024-06-13T07:11:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc3c69bb80 10 client.27530 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.219+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55df98695ee0 con 0x55df98674d60 2024-06-13T07:11:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c07c3a0 con 0x55df98674d60 2024-06-13T07:11:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fbc1c097860 con 0x55df98674d60 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 10 client.27530 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 10 client.27530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 20 client.27530 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df986985c0 0x55df9869aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 4 client.27530 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.220+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df985d5000 con 0x55df986985c0 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.221+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df986985c0 0x55df9869aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.221+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df986985c0 0x55df9869aa10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3748446175e264b0 server_cookie=bc30ec21eefc760e in_seq=0 out_seq=0 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.221+0000 7fbc15600700 10 client.27530 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:01.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.222+0000 7fbc15600700 1 -- 172.21.15.7:0/2541378714 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df985d5000 con 0x55df986985c0 2024-06-13T07:11:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.222+0000 7fbc15600700 10 client.27530 handle_command_reply: tid=0 2024-06-13T07:11:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.222+0000 7fbc3c69bb80 1 client.27530 shutdown 2024-06-13T07:11:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.222+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 msgr2=0x55df98688480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.222+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c09baa0 tx=0x7fbc1c09bb10 comp rx=0 tx=0).stop 2024-06-13T07:11:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 shutdown_connections 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df986985c0 0x55df9869aa10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc15600700 0 client.27530 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98674d60 0x55df98688480 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c09baa0 tx=0x7fbc1c09bb10 comp rx=0 tx=0).stop 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2541378714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df98675680 0x55df98685780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.223+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 >> 172.21.15.7:0/2541378714 conn(0x55df985a2170 msgr2=0x55df98670e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.224+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 shutdown_connections 2024-06-13T07:11:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.225+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2541378714 wait complete. 2024-06-13T07:11:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.225+0000 7fbc3c69bb80 20 client.27530 trim_cache size 0 max 16384 2024-06-13T07:11:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 Processor -- start 2024-06-13T07:11:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 -- start start 2024-06-13T07:11:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98685770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24153200 con 0x55df98688d40 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24150a30 con 0x55df98675680 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.259+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2414e880 con 0x55df986985c0 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98685770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48582/0 (socket says 172.21.15.7:48582) 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2a000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 -- 172.21.15.7:0/1503046690 learned_addr learned my addr 172.21.15.7:0/1503046690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98685770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51788/0 (socket says 172.21.15.7:51788) 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 -- 172.21.15.7:0/1503046690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 msgr2=0x55df98685770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98685770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 -- 172.21.15.7:0/1503046690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 msgr2=0x55df98685cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 -- 172.21.15.7:0/1503046690 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9867b860 con 0x55df986985c0 2024-06-13T07:11:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.260+0000 7fbc2b400700 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c068340 tx=0x7fbc2c0681e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49937aebbc8aa4f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.261+0000 7fbc17e00700 1 -- 172.21.15.7:0/1503046690 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c089620 con 0x55df986985c0 2024-06-13T07:11:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.261+0000 7fbc17e00700 1 -- 172.21.15.7:0/1503046690 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c07d8d0 con 0x55df986985c0 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.261+0000 7fbc17e00700 1 -- 172.21.15.7:0/1503046690 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c074840 con 0x55df986985c0 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.261+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1503046690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 msgr2=0x55df986a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.261+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c068340 tx=0x7fbc2c0681e0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1503046690 shutdown_connections 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df986a02e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98685770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/1503046690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98685cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1503046690 >> 172.21.15.7:0/1503046690 conn(0x55df985a2170 msgr2=0x55df98685060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1503046690 shutdown_connections 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/1503046690 wait complete. 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 -- start start 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.262+0000 7fbc3c69bb80 1 Processor -- start 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98691290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df9868bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2414eef0 con 0x55df98688d40 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24150360 con 0x55df98675680 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2414f5c0 con 0x55df986985c0 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc3c69bb80 10 client.0 _ll_register_callbacks cb 0x55df9855d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51796/0 (socket says 172.21.15.7:51796) 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2062481066 learned_addr learned my addr 172.21.15.7:0/2062481066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98691290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2b400700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df9868bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2062481066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 msgr2=0x55df9868bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df9868bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2062481066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 msgr2=0x55df98691290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98691290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2aa00700 1 -- 172.21.15.7:0/2062481066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9868c370 con 0x55df98675680 2024-06-13T07:11:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.263+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98691290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc2aa00700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00f010 tx=0x7fbc1c09b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211cb60443714e6b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc15600700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c09b160 con 0x55df98675680 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fbc1c0ba1e0 con 0x55df98675680 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c07d8d0 con 0x55df98675680 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc1c0a4540 con 0x55df98675680 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc3c69bb80 10 client.27536 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55df98703e80 con 0x55df98675680 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.264+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c07d8d0 con 0x55df98675680 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fbc1c086110 con 0x55df98675680 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 10 client.27536 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 10 client.27536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 20 client.27536 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df987043a0 0x55df987067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 4 client.27536 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df985cf580 con 0x55df987043a0 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.265+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df987043a0 0x55df987067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.266+0000 7fbc15600700 10 client.27536 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.266+0000 7fbc2a000700 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df987043a0 0x55df987067c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71b12c52870f48b server_cookie=817e7161a293fb0e in_seq=0 out_seq=0 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.266+0000 7fbc15600700 1 -- 172.21.15.7:0/2062481066 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df985cf580 con 0x55df987043a0 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc15600700 10 client.27536 handle_command_reply: tid=0 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 client.27536 shutdown 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 msgr2=0x55df98690d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00f010 tx=0x7fbc1c09b990 comp rx=0 tx=0).stop 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 shutdown_connections 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55df987043a0 0x55df987067c0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df986985c0 0x55df9868bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55df98675680 0x55df98690d50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 --2- 172.21.15.7:0/2062481066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df98688d40 0x55df98691290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc15600700 0 client.27536 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.267+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 >> 172.21.15.7:0/2062481066 conn(0x55df985a2170 msgr2=0x55df986858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.268+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 shutdown_connections 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.268+0000 7fbc3c69bb80 1 -- 172.21.15.7:0/2062481066 wait complete. 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.268+0000 7fbc3c69bb80 20 client.27536 trim_cache size 0 max 16384 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-06-13T07:11:01.271 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-06-13T07:11:01.272 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 msgr2=0x7fbc100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 0x7fbc100a5160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbc24092f00 tx=0x7fbc1800b260 comp rx=0 tx=0).stop 2024-06-13T07:11:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 msgr2=0x7fbc2412e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00e9a0 tx=0x7fbc1c00ecb0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 shutdown_connections 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2409be40 0x7fbc24132ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc240047d0 0x7fbc2412e610 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fbc100a2ca0 0x7fbc100a5160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 --2- 172.21.15.7:0/3718226407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc24005190 0x7fbc2412eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.269+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 >> 172.21.15.7:0/3718226407 conn(0x7fbc2400a7a0 msgr2=0x7fbc240a0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.270+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 shutdown_connections 2024-06-13T07:11:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:01.270+0000 7fbc2be00700 1 -- 172.21.15.7:0/3718226407 wait complete. 2024-06-13T07:11:02.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:01 smithi189 ceph-mon[36616]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-06-13T07:11:02.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:01 smithi175 ceph-mon[36886]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-06-13T07:11:02.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:01 smithi007 ceph-mon[32077]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 492 op/s 2024-06-13T07:11:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 900.0 s, 480.6 tps, lat 207.905 ms stddev 236.646 2024-06-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi007.stdout:transaction type: 2024-06-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi007.stdout:scaling factor: 32 2024-06-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi007.stdout:query mode: simple 2024-06-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi007.stdout:number of clients: 100 2024-06-13T07:11:03.171 INFO:teuthology.orchestra.run.smithi007.stdout:number of threads: 4 2024-06-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi007.stdout:duration: 900 s 2024-06-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi007.stdout:number of transactions actually processed: 688891 2024-06-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi007.stdout:latency average = 130.623 ms 2024-06-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi007.stdout:latency stddev = 197.672 ms 2024-06-13T07:11:03.172 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 765.186670 (including connections establishing) 2024-06-13T07:11:03.173 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 765.192724 (excluding connections establishing) 2024-06-13T07:11:03.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-06-13T07:11:03.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-13T07:11:03.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:11:03.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:03 smithi007 ceph-mon[32077]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-06-13T07:11:04.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:03 smithi189 ceph-mon[36616]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-06-13T07:11:04.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:03 smithi175 ceph-mon[36886]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.9 MiB/s wr, 443 op/s 2024-06-13T07:11:04.244 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:06.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:05 smithi189 ceph-mon[36616]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.3 MiB/s wr, 577 op/s 2024-06-13T07:11:06.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:05 smithi175 ceph-mon[36886]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.3 MiB/s wr, 577 op/s 2024-06-13T07:11:06.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:05 smithi007 ceph-mon[32077]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.3 MiB/s wr, 577 op/s 2024-06-13T07:11:07.067 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:06 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:07.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:06 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:07.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:06 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:11:07.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:07 smithi007 ceph-mon[32077]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-06-13T07:11:08.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:07 smithi189 ceph-mon[36616]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-06-13T07:11:08.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:07 smithi175 ceph-mon[36886]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-06-13T07:11:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.730+0000 7f68f7800640 1 -- 172.21.15.7:0/2175979757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f01098c0 msgr2=0x7f68f010bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.730+0000 7f68f7800640 1 --2- 172.21.15.7:0/2175979757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f01098c0 0x7f68f010bd60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009ca0 tx=0x7f68e802a950 comp rx=0 tx=0).stop 2024-06-13T07:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 -- 172.21.15.7:0/2175979757 shutdown_connections 2024-06-13T07:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 --2- 172.21.15.7:0/2175979757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f01098c0 0x7f68f010bd60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 --2- 172.21.15.7:0/2175979757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f00743a0 0x7f68f0074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 --2- 172.21.15.7:0/2175979757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f0075980 0x7f68f0073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 -- 172.21.15.7:0/2175979757 >> 172.21.15.7:0/2175979757 conn(0x7f68f00fcfc0 msgr2=0x7f68f00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 -- 172.21.15.7:0/2175979757 shutdown_connections 2024-06-13T07:11:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.731+0000 7f68f7800640 1 -- 172.21.15.7:0/2175979757 wait complete. 2024-06-13T07:11:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 Processor -- start 2024-06-13T07:11:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 -- start start 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f00743a0 0x7f68f00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 0x7f68f0070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010f8e0 con 0x7f68f00743a0 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010f760 con 0x7f68f01098c0 2024-06-13T07:11:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.733+0000 7f68f7800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68f010fa60 con 0x7f68f0075980 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 0x7f68f0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51812/0 (socket says 172.21.15.7:51812) 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 -- 172.21.15.7:0/1299420596 learned_addr learned my addr 172.21.15.7:0/1299420596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 -- 172.21.15.7:0/1299420596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 msgr2=0x7f68f0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 0x7f68f0070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 -- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f00743a0 msgr2=0x7f68f00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f00743a0 0x7f68f00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.734+0000 7f68f5800640 1 -- 172.21.15.7:0/1299420596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f0073990 con 0x7f68f01098c0 2024-06-13T07:11:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.735+0000 7f68e7e00640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 0x7f68f0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-13T07:11:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.735+0000 7f68f5800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f68e802af40 tx=0x7f68e802fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.739+0000 7f68ee000640 1 -- 172.21.15.7:0/1299420596 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8034040 con 0x7f68f01098c0 2024-06-13T07:11:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.739+0000 7f68ee000640 1 -- 172.21.15.7:0/1299420596 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e8030920 con 0x7f68f01098c0 2024-06-13T07:11:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.739+0000 7f68ee000640 1 -- 172.21.15.7:0/1299420596 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8045480 con 0x7f68f01098c0 2024-06-13T07:11:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.739+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f01aa530 con 0x7f68f01098c0 2024-06-13T07:11:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.739+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01aa9a0 con 0x7f68f01098c0 2024-06-13T07:11:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.743+0000 7f68fa628740 1 Processor -- start 2024-06-13T07:11:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 -- start start 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc2626c00 0x55ccc2627020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc262c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 0x55ccc262ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccc258a420 con 0x55ccc2626c00 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccc2531c50 con 0x55ccc262c760 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.748+0000 7f68fa628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ccc25863e0 con 0x55ccc26275f0 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.749+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc262c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.749+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc262c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48620/0 (socket says 172.21.15.7:48620) 2024-06-13T07:11:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.749+0000 7f68f4e00640 1 -- 172.21.15.7:0/3529859045 learned_addr learned my addr 172.21.15.7:0/3529859045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.749+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 0x55ccc262ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68ee000640 1 -- 172.21.15.7:0/1299420596 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f68e8037020 con 0x7f68f01098c0 2024-06-13T07:11:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc2626c00 0x55ccc2627020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 -- 172.21.15.7:0/3529859045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 msgr2=0x55ccc262c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc262c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 -- 172.21.15.7:0/3529859045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc2626c00 msgr2=0x55ccc2627020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc2626c00 0x55ccc2627020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 -- 172.21.15.7:0/3529859045 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc229e210 con 0x55ccc262c760 2024-06-13T07:11:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.750+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 0x55ccc262ebf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68f00717e0 tx=0x7f68d80096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be10a2c7f41e35f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.751+0000 7f68e6000640 1 -- 172.21.15.7:0/3529859045 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004020 con 0x55ccc262c760 2024-06-13T07:11:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.751+0000 7f68e6000640 1 -- 172.21.15.7:0/3529859045 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d801cd30 con 0x55ccc262c760 2024-06-13T07:11:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.751+0000 7f68e6000640 1 -- 172.21.15.7:0/3529859045 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004020 con 0x55ccc262c760 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.752+0000 7f68ee000640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 0x7f68d40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.752+0000 7f68fa628740 1 -- 172.21.15.7:0/3529859045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 msgr2=0x55ccc262ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.752+0000 7f68fa628740 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 0x55ccc262ebf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68f00717e0 tx=0x7f68d80096c0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.752+0000 7f68ee000640 1 -- 172.21.15.7:0/1299420596 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f68e80f0790 con 0x7f68f01098c0 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.755+0000 7f68fa628740 1 -- 172.21.15.7:0/3529859045 shutdown_connections 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.755+0000 7f68fa628740 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc262c760 0x55ccc262ebf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.755+0000 7f68fa628740 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc262c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.755+0000 7f68fa628740 1 --2- 172.21.15.7:0/3529859045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc2626c00 0x55ccc2627020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.755+0000 7f68fa628740 1 -- 172.21.15.7:0/3529859045 >> 172.21.15.7:0/3529859045 conn(0x55ccc25eefb0 msgr2=0x55ccc2622680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.757+0000 7f68fa628740 1 -- 172.21.15.7:0/3529859045 shutdown_connections 2024-06-13T07:11:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.757+0000 7f68fa628740 1 -- 172.21.15.7:0/3529859045 wait complete. 2024-06-13T07:11:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.758+0000 7f68fa628740 1 -- start start 2024-06-13T07:11:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.758+0000 7f68fa628740 1 Processor -- start 2024-06-13T07:11:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.759+0000 7f68fa628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 0x55ccc2630fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.759+0000 7f68fa628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.759+0000 7f68fa628740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2638520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 0x55ccc2630fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 0x55ccc2630fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51824/0 (socket says 172.21.15.7:51824) 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68fa628740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccc262f710 con 0x55ccc262c760 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48628/0 (socket says 172.21.15.7:48628) 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68fa628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ccc25450b0 con 0x55ccc2626c00 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68e7e00640 1 -- 172.21.15.7:0/3722579851 learned_addr learned my addr 172.21.15.7:0/3722579851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ccc262f890 con 0x55ccc26275f0 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.761+0000 7f68fa628740 10 client.0 _ll_register_callbacks cb 0x55ccc2530f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.760+0000 7f68f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2638520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.761+0000 7f68e7e00640 1 -- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 msgr2=0x55ccc2630fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.761+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 0x55ccc2630fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.761+0000 7f68e7e00640 1 -- 172.21.15.7:0/3722579851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 msgr2=0x55ccc2638520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.761+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2638520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.762+0000 7f68e7e00640 1 -- 172.21.15.7:0/3722579851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc236a620 con 0x55ccc26275f0 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.762+0000 7f68f4e00640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 0x7f68d40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.764+0000 7f68e7e00640 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f68d801cce0 tx=0x7f68d8005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f206b89a666c368e server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.764+0000 7f68dea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.764+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8033060 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.764+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f68d8004230 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.764+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d801cd30 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.765+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f68d8009920 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.765+0000 7f68fa628740 10 client.27433 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.765+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ccc263b870 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.765+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d801cd30 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.765+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f68d802eb40 con 0x55ccc26275f0 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 10 client.27433 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 10 client.27433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 20 client.27433 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc263cc20 0x55ccc263f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 4 client.27433 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc2597b50 con 0x55ccc263cc20 2024-06-13T07:11:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc263cc20 0x55ccc263f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.766+0000 7f68f4e00640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 0x7f68d40a52e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55ccc2623530 tx=0x7f68e0009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.767+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc263cc20 0x55ccc263f040 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2433b5e56d076ba0 server_cookie=c0a63ff5c2cc7390 in_seq=0 out_seq=0 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.767+0000 7f68dea00640 10 client.27433 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.768+0000 7f68dea00640 1 -- 172.21.15.7:0/3722579851 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccc2597b50 con 0x55ccc263cc20 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.768+0000 7f68dea00640 10 client.27433 handle_command_reply: tid=0 2024-06-13T07:11:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.768+0000 7f68fa628740 1 client.27433 shutdown 2024-06-13T07:11:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 msgr2=0x55ccc26314e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f68d801cce0 tx=0x7f68d8005870 comp rx=0 tx=0).stop 2024-06-13T07:11:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 shutdown_connections 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc263cc20 0x55ccc263f040 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2638520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc26275f0 0x55ccc26314e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 --2- 172.21.15.7:0/3722579851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc2626c00 0x55ccc2630fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 >> 172.21.15.7:0/3722579851 conn(0x55ccc25eefb0 msgr2=0x55ccc2621840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68dea00640 0 client.27433 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.770+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 shutdown_connections 2024-06-13T07:11:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.771+0000 7f68fa628740 1 -- 172.21.15.7:0/3722579851 wait complete. 2024-06-13T07:11:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.771+0000 7f68fa628740 20 client.27433 trim_cache size 0 max 16384 2024-06-13T07:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.815+0000 7f68fa628740 1 Processor -- start 2024-06-13T07:11:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.815+0000 7f68fa628740 1 -- start start 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.815+0000 7f68fa628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc261d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.815+0000 7f68fa628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc26384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.816+0000 7f68fa628740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2625650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.816+0000 7f68fa628740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01b32e0 con 0x55ccc262c760 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.816+0000 7f68fa628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01b0b50 con 0x55ccc26275f0 2024-06-13T07:11:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.816+0000 7f68fa628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01ae9a0 con 0x55ccc2626c00 2024-06-13T07:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.818+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc26384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.818+0000 7f68f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2625650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.818+0000 7f68f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2625650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50042/0 (socket says 172.21.15.7:50042) 2024-06-13T07:11:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f5800640 1 -- 172.21.15.7:0/3627470318 learned_addr learned my addr 172.21.15.7:0/3627470318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc261d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 -- 172.21.15.7:0/3627470318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 msgr2=0x55ccc26384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc26384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 -- 172.21.15.7:0/3627470318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 msgr2=0x55ccc2625650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2625650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.819+0000 7f68f4e00640 1 -- 172.21.15.7:0/3627470318 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc2622c20 con 0x55ccc2626c00 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.820+0000 7f68f4e00640 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc261d6d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55ccc263f810 tx=0x7f68e0013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00515bb3b9a5eeb server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.820+0000 7f68e5600640 1 -- 172.21.15.7:0/3627470318 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e00272d0 con 0x55ccc2626c00 2024-06-13T07:11:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.821+0000 7f68e5600640 1 -- 172.21.15.7:0/3627470318 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e002c070 con 0x55ccc2626c00 2024-06-13T07:11:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.821+0000 7f68e5600640 1 -- 172.21.15.7:0/3627470318 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e000f780 con 0x55ccc2626c00 2024-06-13T07:11:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.822+0000 7f68fa628740 1 -- 172.21.15.7:0/3627470318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 msgr2=0x55ccc261d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.822+0000 7f68fa628740 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc261d6d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55ccc263f810 tx=0x7f68e0013070 comp rx=0 tx=0).stop 2024-06-13T07:11:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.824+0000 7f68fa628740 1 -- 172.21.15.7:0/3627470318 shutdown_connections 2024-06-13T07:11:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.824+0000 7f68fa628740 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2625650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.824+0000 7f68fa628740 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc26384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.824+0000 7f68fa628740 1 --2- 172.21.15.7:0/3627470318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc261d6d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.824+0000 7f68fa628740 1 -- 172.21.15.7:0/3627470318 >> 172.21.15.7:0/3627470318 conn(0x55ccc25eefb0 msgr2=0x55ccc2628910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.825+0000 7f68fa628740 1 -- 172.21.15.7:0/3627470318 shutdown_connections 2024-06-13T07:11:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.825+0000 7f68fa628740 1 -- 172.21.15.7:0/3627470318 wait complete. 2024-06-13T07:11:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.825+0000 7f68fa628740 1 -- start start 2024-06-13T07:11:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 Processor -- start 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc2636090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2637030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01af010 con 0x55ccc262c760 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01b0480 con 0x55ccc26275f0 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01af6e0 con 0x55ccc2626c00 2024-06-13T07:11:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.826+0000 7f68fa628740 10 client.0 _ll_register_callbacks cb 0x7f68e0009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.827+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.827+0000 7f68f4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48634/0 (socket says 172.21.15.7:48634) 2024-06-13T07:11:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.827+0000 7f68f4e00640 1 -- 172.21.15.7:0/1832587560 learned_addr learned my addr 172.21.15.7:0/1832587560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.827+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc2636090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.827+0000 7f68f5800640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2637030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.828+0000 7f68e7e00640 1 -- 172.21.15.7:0/1832587560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 msgr2=0x55ccc2635b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.828+0000 7f68e7e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.828+0000 7f68e7e00640 1 -- 172.21.15.7:0/1832587560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 msgr2=0x55ccc2637030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.828+0000 7f68e7e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2637030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.828+0000 7f68e7e00640 1 -- 172.21.15.7:0/1832587560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc2637620 con 0x55ccc26275f0 2024-06-13T07:11:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68f4e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68f5800640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2637030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68dea00640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68e7e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc2636090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ac0 tx=0x7f68d8012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5736eb5ed0530685 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8033070 con 0x55ccc26275f0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.832+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e002c070 con 0x55ccc26275f0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.833+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f68d803d070 con 0x55ccc26275f0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.833+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f68d8046050 con 0x55ccc26275f0 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.833+0000 7f68fa628740 10 client.27551 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.833+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ccc26a78a0 con 0x55ccc26275f0 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.833+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e002c070 con 0x55ccc26275f0 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.834+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f68d8004660 con 0x55ccc26275f0 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.834+0000 7f68fa628740 10 client.27551 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68fa628740 10 client.27551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68fa628740 20 client.27551 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc26a8120 0x55ccc26aa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68fa628740 4 client.27551 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-13T07:11:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccc261ddb0 con 0x55ccc26a8120 2024-06-13T07:11:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.835+0000 7f68f4e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc26a8120 0x55ccc26aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.836+0000 7f68f4e00640 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc26a8120 0x55ccc26aa510 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888abe88779d448f server_cookie=3d8b2b2781507258 in_seq=0 out_seq=0 2024-06-13T07:11:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:08.836+0000 7f68dea00640 10 client.27551 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:09.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 -- 172.21.15.7:0/2404054522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15480746c0 msgr2=0x7f1548074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 --2- 172.21.15.7:0/2404054522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15480746c0 0x7f1548074aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1538009990 tx=0x7f1538009ca0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 -- 172.21.15.7:0/2404054522 shutdown_connections 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 --2- 172.21.15.7:0/2404054522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154810eef0 0x7f15481112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 --2- 172.21.15.7:0/2404054522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15480746c0 0x7f1548074aa0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1538009990 tx=0x7f1538009ca0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 --2- 172.21.15.7:0/2404054522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1548074fe0 0x7f154810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.176+0000 7f154ca00700 1 -- 172.21.15.7:0/2404054522 >> 172.21.15.7:0/2404054522 conn(0x7f15480795f0 msgr2=0x7f1548079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.179+0000 7f154ca00700 1 -- 172.21.15.7:0/2404054522 shutdown_connections 2024-06-13T07:11:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.179+0000 7f154ca00700 1 -- 172.21.15.7:0/2404054522 wait complete. 2024-06-13T07:11:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.182+0000 7f154ca00700 1 Processor -- start 2024-06-13T07:11:09.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.182+0000 7f154ca00700 1 -- start start 2024-06-13T07:11:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f154ca00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 0x7f154819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f154ca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:09.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f154ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154819f9b0 0x7f154819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f1545e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f1545e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:48662/0 (socket says 172.21.15.7:48662) 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f153de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 0x7f154819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.183+0000 7f153de00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 0x7f154819b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:51858/0 (socket says 172.21.15.7:51858) 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f1545e00700 1 -- 172.21.15.7:0/4028129000 learned_addr learned my addr 172.21.15.7:0/4028129000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1548114a20 con 0x7f154819f9b0 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15481148a0 con 0x7f1548074fe0 2024-06-13T07:11:09.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1548114ba0 con 0x7f154810eef0 2024-06-13T07:11:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f1545e00700 1 -- 172.21.15.7:0/4028129000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 msgr2=0x7f154819b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f1545e00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 0x7f154819b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f1545e00700 1 -- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154819f9b0 msgr2=0x7f154819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.184+0000 7f1545e00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154819f9b0 0x7f154819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.186+0000 7f1545e00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15481a04d0 con 0x7f154810eef0 2024-06-13T07:11:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.186+0000 7f1545e00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f153000d950 tx=0x7f153000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.186+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530016070 con 0x7f154810eef0 2024-06-13T07:11:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.186+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15481a07c0 con 0x7f154810eef0 2024-06-13T07:11:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.187+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1530009e20 con 0x7f154810eef0 2024-06-13T07:11:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.187+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15481a7ff0 con 0x7f154810eef0 2024-06-13T07:11:09.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.189+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1548069180 con 0x7f154810eef0 2024-06-13T07:11:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.189+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000f620 con 0x7f154810eef0 2024-06-13T07:11:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.192+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1530019070 con 0x7f154810eef0 2024-06-13T07:11:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.194+0000 7f153f400700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 0x7f152c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.194+0000 7f153de00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 0x7f152c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.195+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1530099900 con 0x7f154810eef0 2024-06-13T07:11:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.195+0000 7f153de00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 0x7f152c0a5160 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:09.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.203+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f153009bf60 con 0x7f154810eef0 2024-06-13T07:11:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.293+0000 7f68dea00640 1 -- 172.21.15.7:0/1832587560 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ccc261ddb0 con 0x55ccc26a8120 2024-06-13T07:11:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.293+0000 7f68dea00640 10 client.27551 handle_command_reply: tid=0 2024-06-13T07:11:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.293+0000 7f68fa628740 1 client.27551 shutdown 2024-06-13T07:11:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 msgr2=0x55ccc2636090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc2636090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009ac0 tx=0x7f68d8012c30 comp rx=0 tx=0).stop 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 shutdown_connections 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x55ccc26a8120 0x55ccc26aa510 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccc262c760 0x55ccc2637030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ccc26275f0 0x55ccc2636090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 --2- 172.21.15.7:0/1832587560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ccc2626c00 0x55ccc2635b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 >> 172.21.15.7:0/1832587560 conn(0x55ccc25eefb0 msgr2=0x55ccc2625650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.294+0000 7f68dea00640 0 client.27551 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.295+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 shutdown_connections 2024-06-13T07:11:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.295+0000 7f68fa628740 1 -- 172.21.15.7:0/1832587560 wait complete. 2024-06-13T07:11:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.295+0000 7f68fa628740 20 client.27551 trim_cache size 0 max 16384 2024-06-13T07:11:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-06-13T07:11:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-06-13T07:11:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-06-13T07:11:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.296+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 msgr2=0x7f68d40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 0x7f68d40a52e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55ccc2623530 tx=0x7f68e0009720 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 msgr2=0x7f68f01a9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f68e802af40 tx=0x7f68e802fce0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 shutdown_connections 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f68d40a2e20 0x7f68d40a52e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68f01098c0 0x7f68f01a9ff0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68f0075980 0x7f68f0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 --2- 172.21.15.7:0/1299420596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f00743a0 0x7f68f00702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 >> 172.21.15.7:0/1299420596 conn(0x7f68f00fcfc0 msgr2=0x7f68f0105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 shutdown_connections 2024-06-13T07:11:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-06-13T07:11:09.297+0000 7f68f7800640 1 -- 172.21.15.7:0/1299420596 wait complete. 2024-06-13T07:11:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.640+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f154802de60 con 0x7f154810eef0 2024-06-13T07:11:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.641+0000 7f153f400700 1 -- 172.21.15.7:0/4028129000 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11871 (secure 0 0 0) 0x7f153009b860 con 0x7f154810eef0 2024-06-13T07:11:09.641 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:10:49.698828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24521,24527,24539]},"id":1}]} 2024-06-13T07:11:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.645+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 msgr2=0x7f152c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.645+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 0x7f152c0a5160 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538005fd0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.645+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 msgr2=0x7f154819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.645+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f153000d950 tx=0x7f153000dc60 comp rx=0 tx=0).stop 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 shutdown_connections 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154819f9b0 0x7f154819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f152c0a2ca0 0x7f152c0a5160 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548074fe0 0x7f154819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 --2- 172.21.15.7:0/4028129000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f154810eef0 0x7f154819b670 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 >> 172.21.15.7:0/4028129000 conn(0x7f15480795f0 msgr2=0x7f15480ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 shutdown_connections 2024-06-13T07:11:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:09.646+0000 7f154ca00700 1 -- 172.21.15.7:0/4028129000 wait complete. 2024-06-13T07:11:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-06-13T07:11:09.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:09 smithi007 ceph-mon[32077]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:11:09.981 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:09.982 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:09.982 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:11:09.982 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:09 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/4028129000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:11:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:09 smithi189 ceph-mon[36616]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:11:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:10.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:10.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:11:10.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:09 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/4028129000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:11:10.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:09 smithi175 ceph-mon[36886]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-06-13T07:11:10.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:11:10.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:11:10.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:11:10.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:09 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/4028129000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:11:10.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-13T07:11:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:11.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:11 smithi007 ceph-mon[32077]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 578 op/s 2024-06-13T07:11:12.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:11 smithi189 ceph-mon[36616]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 578 op/s 2024-06-13T07:11:12.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:11 smithi175 ceph-mon[36886]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 578 op/s 2024-06-13T07:11:13.848 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:13 smithi007 ceph-mon[32077]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-06-13T07:11:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 -- 172.21.15.7:0/3061213484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102620 msgr2=0x7fad50102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 --2- 172.21.15.7:0/3061213484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102620 0x7fad50102a00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad480099c0 tx=0x7fad48009cd0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 -- 172.21.15.7:0/3061213484 shutdown_connections 2024-06-13T07:11:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 --2- 172.21.15.7:0/3061213484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50107700 0x7fad5010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 --2- 172.21.15.7:0/3061213484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102620 0x7fad50102a00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 --2- 172.21.15.7:0/3061213484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50102fd0 0x7fad50107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.988+0000 7fad56e00700 1 -- 172.21.15.7:0/3061213484 >> 172.21.15.7:0/3061213484 conn(0x7fad500fdf10 msgr2=0x7fad50100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.990+0000 7fad56e00700 1 -- 172.21.15.7:0/3061213484 shutdown_connections 2024-06-13T07:11:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.990+0000 7fad56e00700 1 -- 172.21.15.7:0/3061213484 wait complete. 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.993+0000 7fad56e00700 1 Processor -- start 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.993+0000 7fad56e00700 1 -- start start 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.993+0000 7fad56e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50102620 0x7fad5019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.993+0000 7fad56e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102fd0 0x7fad5019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.993+0000 7fad56e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.994+0000 7fad56e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad50112850 con 0x7fad50102620 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.994+0000 7fad56e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad501126d0 con 0x7fad50107700 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.994+0000 7fad56e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad501129d0 con 0x7fad50102fd0 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.994+0000 7fad55a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50102620 0x7fad5019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.995+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.995+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50234/0 (socket says 172.21.15.7:50234) 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.995+0000 7fad56400700 1 -- 172.21.15.7:0/3476312657 learned_addr learned my addr 172.21.15.7:0/3476312657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.995+0000 7fad55000700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102fd0 0x7fad5019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.997+0000 7fad56400700 1 -- 172.21.15.7:0/3476312657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102fd0 msgr2=0x7fad5019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.997+0000 7fad56400700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102fd0 0x7fad5019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.997+0000 7fad56400700 1 -- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50102620 msgr2=0x7fad5019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.997+0000 7fad56400700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50102620 0x7fad5019b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.997+0000 7fad56400700 1 -- 172.21.15.7:0/3476312657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5010eb70 con 0x7fad50107700 2024-06-13T07:11:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:13.999+0000 7fad56400700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b970 tx=0x7fad4400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.000+0000 7fad4e000700 1 -- 172.21.15.7:0/3476312657 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44019070 con 0x7fad50107700 2024-06-13T07:11:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.000+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5010ee60 con 0x7fad50107700 2024-06-13T07:11:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.000+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5019f540 con 0x7fad50107700 2024-06-13T07:11:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.001+0000 7fad4e000700 1 -- 172.21.15.7:0/3476312657 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4400ce10 con 0x7fad50107700 2024-06-13T07:11:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.001+0000 7fad4e000700 1 -- 172.21.15.7:0/3476312657 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44007370 con 0x7fad50107700 2024-06-13T07:11:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.015+0000 7fad4e000700 1 -- 172.21.15.7:0/3476312657 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fad4401c070 con 0x7fad50107700 2024-06-13T07:11:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.018+0000 7fad62c7eb80 1 Processor -- start 2024-06-13T07:11:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.020+0000 7fad4e000700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 0x7fad380a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.020+0000 7fad4e000700 1 -- 172.21.15.7:0/3476312657 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fad440cf760 con 0x7fad50107700 2024-06-13T07:11:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.022+0000 7fad55000700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 0x7fad380a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.022+0000 7fad62c7eb80 1 -- start start 2024-06-13T07:11:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.022+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e17140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e24920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c684d26a10 con 0x55c684e17680 2024-06-13T07:11:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c684d69b00 con 0x55c684e22530 2024-06-13T07:11:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c684d248f0 con 0x55c684e16d60 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad55a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e17140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad55a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e17140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:60458/0 (socket says 172.21.15.7:60458) 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad55a00700 1 -- 172.21.15.7:0/134632665 learned_addr learned my addr 172.21.15.7:0/134632665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.023+0000 7fad56400700 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e24920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.024+0000 7fad56400700 1 -- 172.21.15.7:0/134632665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 msgr2=0x55c684e17140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.024+0000 7fad56400700 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e17140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.024+0000 7fad56400700 1 -- 172.21.15.7:0/134632665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 msgr2=0x55c684e21ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.024+0000 7fad56400700 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e21ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.024+0000 7fad56400700 1 -- 172.21.15.7:0/134632665 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c684d11f90 con 0x55c684e22530 2024-06-13T07:11:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.026+0000 7fad55000700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 0x7fad380a51e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.026+0000 7fad56400700 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e24920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad4409b920 tx=0x7fad4409c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817d7346a042b63f server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.027+0000 7fad42a00700 1 -- 172.21.15.7:0/134632665 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad440a4890 con 0x55c684e22530 2024-06-13T07:11:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.029+0000 7fad42a00700 1 -- 172.21.15.7:0/134632665 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad440ccfd0 con 0x55c684e22530 2024-06-13T07:11:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.030+0000 7fad62c7eb80 1 -- 172.21.15.7:0/134632665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 msgr2=0x55c684e24920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.030+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e24920 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fad4409b920 tx=0x7fad4409c3a0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.031+0000 7fad62c7eb80 1 -- 172.21.15.7:0/134632665 shutdown_connections 2024-06-13T07:11:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.031+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e24920 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.031+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e17140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.031+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/134632665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e21ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.031+0000 7fad62c7eb80 1 -- 172.21.15.7:0/134632665 >> 172.21.15.7:0/134632665 conn(0x55c684d44170 msgr2=0x55c684e13280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.032+0000 7fad62c7eb80 1 -- 172.21.15.7:0/134632665 shutdown_connections 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.032+0000 7fad62c7eb80 1 -- 172.21.15.7:0/134632665 wait complete. 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.032+0000 7fad62c7eb80 1 -- start start 2024-06-13T07:11:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.033+0000 7fad62c7eb80 1 Processor -- start 2024-06-13T07:11:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.033+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e21c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e1ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c684d05f60 con 0x55c684e16d60 2024-06-13T07:11:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c684d65ac0 con 0x55c684e22530 2024-06-13T07:11:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c684e25020 con 0x55c684e17680 2024-06-13T07:11:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.034+0000 7fad62c7eb80 10 client.0 _ll_register_callbacks cb 0x55c684cff380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad55a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50260/0 (socket says 172.21.15.7:50260) 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 -- 172.21.15.7:0/939039544 learned_addr learned my addr 172.21.15.7:0/939039544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 -- 172.21.15.7:0/939039544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 msgr2=0x55c684e1ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e1ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 -- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 msgr2=0x55c684e21c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e21c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.035+0000 7fad56400700 1 -- 172.21.15.7:0/939039544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c684cb88d0 con 0x55c684e22530 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.036+0000 7fad56400700 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad4409c280 tx=0x7fad440a4760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e64576369b95cd97 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.036+0000 7fad37400700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.037+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad440b9070 con 0x55c684e22530 2024-06-13T07:11:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.037+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad440ccfd0 con 0x55c684e22530 2024-06-13T07:11:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.038+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fad44099f20 con 0x55c684e22530 2024-06-13T07:11:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.039+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fad440b9070 con 0x55c684e22530 2024-06-13T07:11:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.039+0000 7fad62c7eb80 10 client.27566 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.040+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c684e2f720 con 0x55c684e22530 2024-06-13T07:11:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.041+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad440ccfd0 con 0x55c684e22530 2024-06-13T07:11:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fad4407aa90 con 0x55c684e22530 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 10 client.27566 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 10 client.27566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 20 client.27566 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684e31bf0 0x55c684e33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 4 client.27566 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.045+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c684d77000 con 0x55c684e31bf0 2024-06-13T07:11:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.046+0000 7fad55a00700 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684e31bf0 0x55c684e33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.047+0000 7fad55a00700 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684e31bf0 0x55c684e33fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1e81f8ab540c7ce server_cookie=9619bc1b8eb4d7f5 in_seq=0 out_seq=0 2024-06-13T07:11:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.047+0000 7fad37400700 10 client.27566 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:11:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad37400700 1 -- 172.21.15.7:0/939039544 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c684d77000 con 0x55c684e31bf0 2024-06-13T07:11:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad37400700 10 client.27566 handle_command_reply: tid=0 2024-06-13T07:11:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 client.27566 shutdown 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 msgr2=0x55c684e1f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fad4409c280 tx=0x7fad440a4760 comp rx=0 tx=0).stop 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 shutdown_connections 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684e31bf0 0x55c684e33fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e1f440 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/939039544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e1ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad37400700 0 client.27566 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:11:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.050+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 >> 172.21.15.7:0/939039544 conn(0x55c684d44170 msgr2=0x55c684e12e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.051+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 shutdown_connections 2024-06-13T07:11:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.051+0000 7fad62c7eb80 1 -- 172.21.15.7:0/939039544 wait complete. 2024-06-13T07:11:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.051+0000 7fad62c7eb80 20 client.27566 trim_cache size 0 max 16384 2024-06-13T07:11:14.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:13 smithi189 ceph-mon[36616]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-06-13T07:11:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.091+0000 7fad62c7eb80 1 Processor -- start 2024-06-13T07:11:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.091+0000 7fad62c7eb80 1 -- start start 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad30011130 con 0x55c684e17680 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad3000e960 con 0x55c684e22530 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad3000c860 con 0x55c684e16d60 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:50276/0 (socket says 172.21.15.7:50276) 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad56400700 1 -- 172.21.15.7:0/3622346296 learned_addr learned my addr 172.21.15.7:0/3622346296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.092+0000 7fad55000700 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.093+0000 7fad55a00700 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e13100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.093+0000 7fad56400700 1 -- 172.21.15.7:0/3622346296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 msgr2=0x55c684e13100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.093+0000 7fad56400700 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e13100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.093+0000 7fad56400700 1 -- 172.21.15.7:0/3622346296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 msgr2=0x55c684e20e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.094+0000 7fad56400700 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e20e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.094+0000 7fad56400700 1 -- 172.21.15.7:0/3622346296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c684e12a20 con 0x55c684e22530 2024-06-13T07:11:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.094+0000 7fad56400700 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad4409bbb0 tx=0x7fad4409b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72122135403db738 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.095+0000 7fad42000700 1 -- 172.21.15.7:0/3622346296 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad440a1500 con 0x55c684e22530 2024-06-13T07:11:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.095+0000 7fad42000700 1 -- 172.21.15.7:0/3622346296 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad440a4890 con 0x55c684e22530 2024-06-13T07:11:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.095+0000 7fad42000700 1 -- 172.21.15.7:0/3622346296 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4407aa90 con 0x55c684e22530 2024-06-13T07:11:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 -- 172.21.15.7:0/3622346296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 msgr2=0x55c684e39580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fad4409bbb0 tx=0x7fad4409b4a0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 -- 172.21.15.7:0/3622346296 shutdown_connections 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e39580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e16d60 0x55c684e13100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/3622346296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e17680 0x55c684e20e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.098+0000 7fad62c7eb80 1 -- 172.21.15.7:0/3622346296 >> 172.21.15.7:0/3622346296 conn(0x55c684d44170 msgr2=0x55c684e20730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.099+0000 7fad62c7eb80 1 -- 172.21.15.7:0/3622346296 shutdown_connections 2024-06-13T07:11:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.099+0000 7fad62c7eb80 1 -- 172.21.15.7:0/3622346296 wait complete. 2024-06-13T07:11:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.100+0000 7fad62c7eb80 1 -- start start 2024-06-13T07:11:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.100+0000 7fad62c7eb80 1 Processor -- start 2024-06-13T07:11:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.101+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.101+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.101+0000 7fad62c7eb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e29500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.102+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad3000ce20 con 0x55c684e16d60 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.102+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad3000e290 con 0x55c684e22530 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.102+0000 7fad62c7eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad3000d4f0 con 0x55c684e17680 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.102+0000 7fad62c7eb80 10 client.0 _ll_register_callbacks cb 0x55c684cff380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.102+0000 7fad56400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e29500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.103+0000 7fad55a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.103+0000 7fad55a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45500/0 (socket says 172.21.15.7:45500) 2024-06-13T07:11:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.103+0000 7fad55a00700 1 -- 172.21.15.7:0/99262331 learned_addr learned my addr 172.21.15.7:0/99262331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad55a00700 1 -- 172.21.15.7:0/99262331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 msgr2=0x55c684e28fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-13T07:11:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad55a00700 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e28fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad55a00700 1 -- 172.21.15.7:0/99262331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 msgr2=0x55c684e29500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad55a00700 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e29500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad55a00700 1 -- 172.21.15.7:0/99262331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c684e19f50 con 0x55c684e16d60 2024-06-13T07:11:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.104+0000 7fad37400700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-06-13T07:11:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.105+0000 7fad55a00700 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad480099c0 tx=0x7fad48005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78f3fdcbd1a0ae5 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.105+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48003930 con 0x55c684e16d60 2024-06-13T07:11:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.106+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad440a4890 con 0x55c684e16d60 2024-06-13T07:11:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.106+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fad4800bd80 con 0x55c684e16d60 2024-06-13T07:11:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.108+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fad48003930 con 0x55c684e16d60 2024-06-13T07:11:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.108+0000 7fad62c7eb80 10 client.16722 fetch_fsmap learned FSMap version 44 2024-06-13T07:11:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.108+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c684eadad0 con 0x55c684e16d60 2024-06-13T07:11:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.111+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad440a4890 con 0x55c684e16d60 2024-06-13T07:11:14.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.111+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fad4805a470 con 0x55c684e16d60 2024-06-13T07:11:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 10 client.16722 fetch_fsmap finished waiting for FSMap version 44 2024-06-13T07:11:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 10 client.16722 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-13T07:11:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 20 client.16722 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684eae1a0 0x55c684eb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:14.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 4 client.16722 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.112+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 --> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c684d71580 con 0x55c684eae1a0 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.113+0000 7fad55000700 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684eae1a0 0x55c684eb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.115+0000 7fad55000700 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684eae1a0 0x55c684eb05c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af8a68e811b3cc50 server_cookie=d1c8fad47c76a69a in_seq=0 out_seq=0 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.115+0000 7fad37400700 10 client.16722 ms_handle_connect on v2:172.21.15.7:6838/827561506 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.116+0000 7fad37400700 1 -- 172.21.15.7:0/99262331 <== mds.2 v2:172.21.15.7:6838/827561506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c684d71580 con 0x55c684eae1a0 2024-06-13T07:11:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.116+0000 7fad37400700 10 client.16722 handle_command_reply: tid=0 2024-06-13T07:11:14.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.116+0000 7fad62c7eb80 1 client.16722 shutdown 2024-06-13T07:11:14.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 msgr2=0x55c684e28a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad480099c0 tx=0x7fad48005fd0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 shutdown_connections 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:6838/827561506,v1:172.21.15.7:6839/827561506] conn(0x55c684eae1a0 0x55c684eb05c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad37400700 0 client.16722 ms_handle_reset on v2:172.21.15.7:6838/827561506 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c684e22530 0x55c684e29500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c684e16d60 0x55c684e28a80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 --2- 172.21.15.7:0/99262331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c684e17680 0x55c684e28fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.118+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 >> 172.21.15.7:0/99262331 conn(0x55c684d44170 msgr2=0x55c684e20fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:14.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:13 smithi175 ceph-mon[36886]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.9 MiB/s wr, 503 op/s 2024-06-13T07:11:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.119+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 shutdown_connections 2024-06-13T07:11:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.119+0000 7fad62c7eb80 1 -- 172.21.15.7:0/99262331 wait complete. 2024-06-13T07:11:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.120+0000 7fad62c7eb80 20 client.16722 trim_cache size 0 max 16384 2024-06-13T07:11:14.121 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:11:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 msgr2=0x7fad380a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 0x7fad380a51e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 msgr2=0x7fad5010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b970 tx=0x7fad4400bc80 comp rx=0 tx=0).stop 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 shutdown_connections 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fad380a2d20 0x7fad380a51e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad50107700 0x7fad5010e2c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50102620 0x7fad5019b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 --2- 172.21.15.7:0/3476312657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad50102fd0 0x7fad5019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 >> 172.21.15.7:0/3476312657 conn(0x7fad500fdf10 msgr2=0x7fad500ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.121+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 shutdown_connections 2024-06-13T07:11:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:14.122+0000 7fad56e00700 1 -- 172.21.15.7:0/3476312657 wait complete. 2024-06-13T07:11:16.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:15 smithi189 ceph-mon[36616]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-06-13T07:11:16.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:15 smithi175 ceph-mon[36886]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-06-13T07:11:16.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:15 smithi007 ceph-mon[32077]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-06-13T07:11:16.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-13T07:11:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:17.915 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:17 smithi007 ceph-mon[32077]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-06-13T07:11:18.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:17 smithi189 ceph-mon[36616]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-06-13T07:11:18.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:17 smithi175 ceph-mon[36886]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.0 MiB/s wr, 562 op/s 2024-06-13T07:11:18.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 data 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:total 36 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 . 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 .. 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 base 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jun 13 06:55 current_logfiles 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jun 13 06:55 global 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 log 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_commit_ts 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_dynshmem 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Jun 13 06:55 pg_hba.conf 2024-06-13T07:11:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 13 06:55 pg_ident.conf 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 pg_logical 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 pg_multixact 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_notify 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_replslot 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_serial 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_snapshots 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 pg_stat 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 07:11 pg_stat_tmp 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 07:11 pg_subtrans 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_tblspc 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 pg_twophase 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Jun 13 07:11 pg_wal 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 pg_xact 2024-06-13T07:11:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 13 06:55 postgresql.auto.conf 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Jun 13 06:55 postgresql.conf 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jun 13 06:55 postmaster.opts 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 . 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 1 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 13493 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Jun 13 07:02 13494 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:56 pgsql_tmp 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-06-13T07:11:19.184 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 . 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 .. 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 1247 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1247_vm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:55 1249 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1249_fsm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1249_vm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T07:11:19.185 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 06:55 1259 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1259_vm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T07:11:19.186 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T07:11:19.187 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2579 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T07:11:19.188 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606_vm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:55 2608 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2608_vm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T07:11:19.189 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2610 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2610_vm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T07:11:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 13 06:55 2619 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2619_fsm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2619_vm 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T07:11:19.191 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:55 2658 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 2659 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2662 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2663 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2664 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2665 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2666 2024-06-13T07:11:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2667 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:55 2673 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:55 2674 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2678 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2679 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T07:11:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2696 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2703 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2704 2024-06-13T07:11:19.194 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T07:11:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T07:11:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T07:11:19.197 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3455 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T07:11:19.198 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T07:11:19.199 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T07:11:19.200 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T07:11:19.201 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T07:11:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T07:11:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T07:11:19.229 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T07:11:19.230 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T07:11:19.231 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Jun 13 06:55 . 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 .. 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 1247 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1247_vm 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:55 1249 2024-06-13T07:11:19.232 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1249_fsm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1249_vm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 06:55 1259 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1259_vm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T07:11:19.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T07:11:19.234 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2579 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T07:11:19.235 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606 2024-06-13T07:11:19.236 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2606_vm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:55 2608 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2608_vm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2610 2024-06-13T07:11:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2610_vm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T07:11:19.238 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Jun 13 06:55 2619 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2619_fsm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2619_vm 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T07:11:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:55 2658 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:55 2659 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2662 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2663 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2664 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2665 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2666 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2667 2024-06-13T07:11:19.240 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:55 2673 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:55 2674 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2678 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2679 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T07:11:19.241 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T07:11:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2696 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2703 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2704 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T07:11:19.243 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T07:11:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T07:11:19.245 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T07:11:19.246 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3455 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T07:11:19.247 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T07:11:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T07:11:19.249 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T07:11:19.250 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T07:11:19.251 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T07:11:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T07:11:19.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T07:11:19.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T07:11:19.253 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T07:11:19.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T07:11:19.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T07:11:19.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:total 544126 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Jun 13 07:02 . 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 .. 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 112 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 113 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:56 1247 2024-06-13T07:11:19.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1247_fsm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 1247_vm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Jun 13 06:56 1249 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:56 1249_fsm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 1249_vm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Jun 13 06:55 1255 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1255_fsm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1255_vm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 13 07:11 1259 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1259_fsm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 1259_vm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jun 13 06:55 13344 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13344_fsm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13344_vm 2024-06-13T07:11:19.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13346 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13348 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13349_fsm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13349_vm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13351 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13353 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13354_fsm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13354_vm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13356 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13358 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 13359_fsm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13359_vm 2024-06-13T07:11:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 13361 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 13363 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1417 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 1418 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439861248 Jun 13 07:11 16396 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 13 07:11 16396_fsm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 07:10 16396_vm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 401408 Jun 13 07:11 16397 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 07:11 16397_fsm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 07:11 16397_vm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 13 07:11 16399 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 07:11 16399_fsm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 07:11 16399_vm 2024-06-13T07:11:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 07:11 16400 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 07:11 16402 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Jun 13 07:11 16404 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 36126720 Jun 13 07:11 16406 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 07:11 16406_fsm 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 07:07 16406_vm 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 174 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 175 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2187 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2224 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2328 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2336 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2337 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2579 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2600 2024-06-13T07:11:19.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2600_fsm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2600_vm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2601_fsm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2601_vm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2602 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2602_fsm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2602_vm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2603 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2603_fsm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2603_vm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2604 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2605 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2605_fsm 2024-06-13T07:11:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2605_vm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 2606 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2606_fsm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 2606_vm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2607 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2607_fsm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2607_vm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 13 06:56 2608 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2608_fsm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 2608_vm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Jun 13 06:55 2609 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2609_fsm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2609_vm 2024-06-13T07:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:56 2610 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2610_fsm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 2610_vm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2611 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2612_fsm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2612_vm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2613 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2615_fsm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2615_vm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2616 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2616_fsm 2024-06-13T07:11:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2616_vm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 13 06:55 2617 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2617_fsm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2617_vm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 13 06:55 2618 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2618_fsm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2618_vm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 13 07:11 2619 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 07:01 2619_fsm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:56 2619_vm 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2620 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2650 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2651 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2652 2024-06-13T07:11:19.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2653 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2654 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2655 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2656 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2657 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 13 06:56 2658 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 13 06:56 2659 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2660 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2661 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:56 2662 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:56 2663 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2664 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2665 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2666 2024-06-13T07:11:19.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2667 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2668 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2669 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2670 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Jun 13 06:56 2673 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Jun 13 06:56 2674 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jun 13 06:55 2675 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2678 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:56 2679 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2680 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2681 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2682 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2683 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2684 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2685 2024-06-13T07:11:19.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2686 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2687 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 2688 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 13 06:55 2689 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 13 06:55 2690 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Jun 13 06:55 2691 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2692 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2693 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 07:03 2696 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2699 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2701 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2702 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:56 2703 2024-06-13T07:11:19.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:56 2704 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2753 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2753_fsm 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2753_vm 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2754 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2755 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2756 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 2757 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2830 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2831 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2832 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2833 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2834 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2835 2024-06-13T07:11:19.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2836 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2837 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Jun 13 06:55 2838 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2838_fsm 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2838_vm 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2839 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2840_fsm 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2840_vm 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2841 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2995 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2996 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3079_fsm 2024-06-13T07:11:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3079_vm 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3080 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3081 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:55 3085 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3118 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3119 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 13 06:55 3164 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3256 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3257 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3258 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3350 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3351 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3379 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3380 2024-06-13T07:11:19.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3381 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3394 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3394_fsm 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3394_vm 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3395 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3429 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3430 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3431 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3433 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3439 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3440 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 13 06:56 3455 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Jun 13 06:55 3456 2024-06-13T07:11:19.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3456_fsm 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3456_vm 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3466 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3467 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3468 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3501 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3502 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3503 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3534 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3541_fsm 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3541_vm 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3542 2024-06-13T07:11:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3574 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3575 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3576 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3596 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3597 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3598 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3599 2024-06-13T07:11:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3600_fsm 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3600_vm 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3601_fsm 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3601_vm 2024-06-13T07:11:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3602_fsm 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3602_vm 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3603_fsm 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3603_vm 2024-06-13T07:11:19.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3604 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3605 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3606 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3607 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3608 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 13 06:55 3609 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3712 2024-06-13T07:11:19.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 3764_fsm 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3764_vm 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3766 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 3767 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3997 2024-06-13T07:11:19.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4143 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4144 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4145 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4146 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4147 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4148 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4149 2024-06-13T07:11:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4150 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4151 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4152 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4153 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4154 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4155 2024-06-13T07:11:19.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4156 2024-06-13T07:11:19.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4157 2024-06-13T07:11:19.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4158 2024-06-13T07:11:19.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4159 2024-06-13T07:11:19.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4160 2024-06-13T07:11:19.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4161 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4162 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4163 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4164 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4165 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4166 2024-06-13T07:11:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4167 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4168 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4169 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4170 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4171 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4172 2024-06-13T07:11:19.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4173 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4174 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 5002 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 548 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 549 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6102 2024-06-13T07:11:19.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6104 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6106 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6110 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6111 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6112 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6113 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6117 2024-06-13T07:11:19.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 826 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 827 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 828 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Jun 13 06:55 pg_internal.init 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 06:55 PG_VERSION 2024-06-13T07:11:19.321 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:56 . 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Jun 13 06:56 .. 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.322 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-06-13T07:11:19.323 INFO:teuthology.orchestra.run.smithi007.stdout:total 560 2024-06-13T07:11:19.323 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Jun 13 06:55 . 2024-06-13T07:11:19.323 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1213 2024-06-13T07:11:19.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1213_fsm 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1213_vm 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1214 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1214_fsm 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1214_vm 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 1232 2024-06-13T07:11:19.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 1233 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1260 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1260_fsm 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1260_vm 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1261 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1261_fsm 2024-06-13T07:11:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1261_vm 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1262 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 1262_fsm 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 1262_vm 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2396 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 13 06:55 2396_fsm 2024-06-13T07:11:19.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2396_vm 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2397 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2671 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2672 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2676 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2677 2024-06-13T07:11:19.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2694 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2695 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2697 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 13 06:55 2698 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2846 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2847 2024-06-13T07:11:19.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2964 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2965 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 2966 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 2967 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 3592 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 3593 2024-06-13T07:11:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4060 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4061 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4175 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4176 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4177 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4178 2024-06-13T07:11:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4181 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4182 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4183 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4184 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 4185 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 4186 2024-06-13T07:11:19.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6000 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6001 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6002 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 6100 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6114 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 6115 2024-06-13T07:11:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 07:11 pg_control 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 13 06:55 pg_filenode.map 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Jun 13 06:55 pg_internal.init 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout:total 2 2024-06-13T07:11:19.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Jun 13 07:11 postgresql-Thu.log 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 . 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 mappings 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 13 07:11 replorigin_checkpoint 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 snapshots 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 .. 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 .. 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 . 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.339 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 members 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 offsets 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T07:11:19.340 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 .. 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 06:55 0000 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T07:11:19.341 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Jun 13 06:55 .. 2024-06-13T07:11:19.342 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 13 07:00 0000 2024-06-13T07:11:19.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.342 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-06-13T07:11:19.342 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.342 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.343 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.343 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.343 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-06-13T07:11:19.343 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.343 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.345 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-06-13T07:11:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.347 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:total 15 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 13 07:11 . 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Jun 13 07:11 db_0.stat 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Jun 13 07:11 db_13494.stat 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Jun 13 07:11 global.stat 2024-06-13T07:11:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 07:11 . 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:total 136 2024-06-13T07:11:19.349 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 07:11 . 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Jun 13 07:11 000A 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:total 884736 2024-06-13T07:11:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Jun 13 07:11 . 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:11 000000010000000000000062 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000063 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:04 000000010000000000000064 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000065 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:03 000000010000000000000066 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 000000010000000000000067 2024-06-13T07:11:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000068 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:05 000000010000000000000069 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 00000001000000000000006A 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 00000001000000000000006B 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:05 00000001000000000000006C 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:03 00000001000000000000006D 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:04 00000001000000000000006E 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:00 00000001000000000000006F 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000070 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000071 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000072 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000073 2024-06-13T07:11:19.353 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000074 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 000000010000000000000075 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 000000010000000000000076 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 000000010000000000000077 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:03 000000010000000000000078 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:02 000000010000000000000079 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 00000001000000000000007A 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:01 00000001000000000000007B 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 00000001000000000000007C 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 00000001000000000000007D 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 00000001000000000000007E 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 00000001000000000000007F 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:08 000000010000000000000080 2024-06-13T07:11:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:08 000000010000000000000081 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000082 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 000000010000000000000083 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:05 000000010000000000000084 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:09 000000010000000000000085 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 000000010000000000000086 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 000000010000000000000087 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:09 000000010000000000000088 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000089 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:08 00000001000000000000008A 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 00000001000000000000008B 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:08 00000001000000000000008C 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:09 00000001000000000000008D 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:05 00000001000000000000008E 2024-06-13T07:11:19.355 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 00000001000000000000008F 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000090 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 000000010000000000000091 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000092 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:07 000000010000000000000093 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000094 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000095 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:06 000000010000000000000096 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 13 07:10 000000010000000000000097 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 archive_status 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 13 06:55 . 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Jun 13 07:11 .. 2024-06-13T07:11:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:19.357 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-06-13T07:11:19.357 INFO:teuthology.orchestra.run.smithi007.stdout:total 176 2024-06-13T07:11:19.357 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Jun 13 06:55 . 2024-06-13T07:11:19.357 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Jun 13 07:11 .. 2024-06-13T07:11:19.357 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 13 07:11 0000 2024-06-13T07:11:19.357 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-13T07:11:19.368 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T07:11:19.393 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-13T07:11:19.418 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-13T07:11:20.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:19 smithi189 ceph-mon[36616]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 610 op/s 2024-06-13T07:11:20.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:19 smithi175 ceph-mon[36886]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 610 op/s 2024-06-13T07:11:20.164 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:19 smithi007 ceph-mon[32077]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 610 op/s 2024-06-13T07:11:20.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:20 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:21.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:20 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:21.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:20 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 -- 172.21.15.7:0/2585100236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc02010eec0 msgr2=0x7fc0201112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 --2- 172.21.15.7:0/2585100236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc02010eec0 0x7fc0201112b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b0a0 tx=0x7fc01800b3b0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 -- 172.21.15.7:0/2585100236 shutdown_connections 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 --2- 172.21.15.7:0/2585100236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc02010eec0 0x7fc0201112b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 --2- 172.21.15.7:0/2585100236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc020074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 --2- 172.21.15.7:0/2585100236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc020074fe0 0x7fc02010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.887+0000 7fc026e00700 1 -- 172.21.15.7:0/2585100236 >> 172.21.15.7:0/2585100236 conn(0x7fc020079640 msgr2=0x7fc020079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.888+0000 7fc026e00700 1 -- 172.21.15.7:0/2585100236 shutdown_connections 2024-06-13T07:11:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.888+0000 7fc026e00700 1 -- 172.21.15.7:0/2585100236 wait complete. 2024-06-13T07:11:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.889+0000 7fc026e00700 1 Processor -- start 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.889+0000 7fc026e00700 1 -- start start 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.889+0000 7fc026e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc026e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 0x7fc02019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc026e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc02010eec0 0x7fc02019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc026e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc020114a40 con 0x7fc0200746c0 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc026e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201148c0 con 0x7fc020074fe0 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc026e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc020114bc0 con 0x7fc02010eec0 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47868/0 (socket says 172.21.15.7:47868) 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01fe00700 1 -- 172.21.15.7:0/371908128 learned_addr learned my addr 172.21.15.7:0/371908128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 0x7fc02019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc024e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc02010eec0 0x7fc02019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 -- 172.21.15.7:0/371908128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc02010eec0 msgr2=0x7fc02019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc02010eec0 0x7fc02019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 -- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 msgr2=0x7fc02019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01f400700 1 -- 172.21.15.7:0/371908128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02019fe70 con 0x7fc020074fe0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.890+0000 7fc01fe00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc01f400700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 0x7fc02019b3f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc01cc00700 1 -- 172.21.15.7:0/371908128 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c004220 con 0x7fc020074fe0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc01cc00700 1 -- 172.21.15.7:0/371908128 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c004d10 con 0x7fc020074fe0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc01cc00700 1 -- 172.21.15.7:0/371908128 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c005020 con 0x7fc020074fe0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0201a0160 con 0x7fc020074fe0 2024-06-13T07:11:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.891+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a79e0 con 0x7fc020074fe0 2024-06-13T07:11:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.896+0000 7fc01cc00700 1 -- 172.21.15.7:0/371908128 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fc00c004750 con 0x7fc020074fe0 2024-06-13T07:11:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.897+0000 7fc0316ddb80 1 Processor -- start 2024-06-13T07:11:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.897+0000 7fc0316ddb80 1 -- start start 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b62f4e40 0x5639b6378070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b6383300 0x5639b63856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5639b6287a10 con 0x5639b6383300 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5639b62cab00 con 0x5639b62f4e40 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639b62858f0 con 0x5639b63785b0 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc01cc00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 0x7fc0040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37882/0 (socket says 172.21.15.7:37882) 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.898+0000 7fc01fe00700 1 -- 172.21.15.7:0/4073418847 learned_addr learned my addr 172.21.15.7:0/4073418847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01cc00700 1 -- 172.21.15.7:0/371908128 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc00c0cf1b0 con 0x7fc020074fe0 2024-06-13T07:11:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc024e00700 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b62f4e40 0x5639b6378070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01f400700 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b6383300 0x5639b63856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc024e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 0x7fc0040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01fe00700 1 -- 172.21.15.7:0/4073418847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b62f4e40 msgr2=0x5639b6378070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01fe00700 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b62f4e40 0x5639b6378070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01fe00700 1 -- 172.21.15.7:0/4073418847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b6383300 msgr2=0x5639b63856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01fe00700 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b6383300 0x5639b63856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.899+0000 7fc01fe00700 1 -- 172.21.15.7:0/4073418847 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b6272f90 con 0x5639b63785b0 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.900+0000 7fc01fe00700 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0100117e0 tx=0x7fc010011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8976a97d750a8f53 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.900+0000 7fc015600700 1 -- 172.21.15.7:0/4073418847 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01001e070 con 0x5639b63785b0 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.900+0000 7fc015600700 1 -- 172.21.15.7:0/4073418847 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01000f960 con 0x5639b63785b0 2024-06-13T07:11:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.900+0000 7fc015600700 1 -- 172.21.15.7:0/4073418847 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100199f0 con 0x5639b63785b0 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 -- 172.21.15.7:0/4073418847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 msgr2=0x5639b6378a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0100117e0 tx=0x7fc010011af0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc024e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 0x7fc0040a5160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b780 tx=0x7fc018006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 -- 172.21.15.7:0/4073418847 shutdown_connections 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b6383300 0x5639b63856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b62f4e40 0x5639b6378070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/4073418847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6378a30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 -- 172.21.15.7:0/4073418847 >> 172.21.15.7:0/4073418847 conn(0x5639b62a5170 msgr2=0x5639b63744b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 -- 172.21.15.7:0/4073418847 shutdown_connections 2024-06-13T07:11:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.901+0000 7fc0316ddb80 1 -- 172.21.15.7:0/4073418847 wait complete. 2024-06-13T07:11:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.902+0000 7fc0316ddb80 1 -- start start 2024-06-13T07:11:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.902+0000 7fc0316ddb80 1 Processor -- start 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b6382ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b637fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6380300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5639b6266f60 con 0x5639b62f4e40 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5639b62c6ac0 con 0x5639b6383300 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639b6385fc0 con 0x5639b63785b0 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc0316ddb80 10 client.0 _ll_register_callbacks cb 0x5639b6264720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b6382ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.903+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b6382ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47886/0 (socket says 172.21.15.7:47886) 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc01fe00700 1 -- 172.21.15.7:0/1734704355 learned_addr learned my addr 172.21.15.7:0/1734704355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6380300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc01f400700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b637fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 -- 172.21.15.7:0/1734704355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 msgr2=0x5639b637fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b637fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 -- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 msgr2=0x5639b6382ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b6382ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.904+0000 7fc024e00700 1 -- 172.21.15.7:0/1734704355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b62198d0 con 0x5639b6383300 2024-06-13T07:11:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc024e00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6380300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc0180062a0 tx=0x7fc0180099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9b44a12302f8ce server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc00a000700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180094a0 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc018028070 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01000f960 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc018038050 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc0316ddb80 10 client.27584 fetch_fsmap learned FSMap version 45 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.905+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5639b6390740 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01000f960 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fc018011070 con 0x5639b6383300 2024-06-13T07:11:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 10 client.27584 fetch_fsmap finished waiting for FSMap version 45 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 10 client.27584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 20 client.27584 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b63930a0 0x5639b6395490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 4 client.27584 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b62d8000 con 0x5639b63930a0 2024-06-13T07:11:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.906+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b63930a0 0x5639b6395490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.907+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b63930a0 0x5639b6395490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a42b985c4393d42e server_cookie=45d28827ea8921c8 in_seq=0 out_seq=0 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.907+0000 7fc00a000700 10 client.27584 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.908+0000 7fc00a000700 1 -- 172.21.15.7:0/1734704355 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639b62d8000 con 0x5639b63930a0 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.908+0000 7fc00a000700 10 client.27584 handle_command_reply: tid=0 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.908+0000 7fc0316ddb80 1 client.27584 shutdown 2024-06-13T07:11:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.908+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 msgr2=0x5639b6380300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.908+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6380300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc0180062a0 tx=0x7fc0180099e0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 shutdown_connections 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6380300 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b6382ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b63930a0 0x5639b6395490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1734704355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b637fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 >> 172.21.15.7:0/1734704355 conn(0x5639b62a5170 msgr2=0x5639b6373740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc00a000700 0 client.27584 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 shutdown_connections 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1734704355 wait complete. 2024-06-13T07:11:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.909+0000 7fc0316ddb80 20 client.27584 trim_cache size 0 max 16384 2024-06-13T07:11:21.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 Processor -- start 2024-06-13T07:11:21.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 -- start start 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b63785b0 0x5639b63846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6384c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b2e10 con 0x5639b63785b0 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b0610 con 0x5639b6383300 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.927+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201ae460 con 0x5639b62f4e40 2024-06-13T07:11:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b63785b0 0x5639b63846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37890/0 (socket says 172.21.15.7:37890) 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 -- 172.21.15.7:0/1158074300 learned_addr learned my addr 172.21.15.7:0/1158074300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc024e00700 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6384c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 -- 172.21.15.7:0/1158074300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 msgr2=0x5639b6384c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6384c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 -- 172.21.15.7:0/1158074300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b63785b0 msgr2=0x5639b63846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b63785b0 0x5639b63846f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 -- 172.21.15.7:0/1158074300 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b637fb90 con 0x5639b62f4e40 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.928+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc010011730 tx=0x7fc010009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18031efa89c17e31 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.929+0000 7fc014c00700 1 -- 172.21.15.7:0/1158074300 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010028070 con 0x5639b62f4e40 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.929+0000 7fc014c00700 1 -- 172.21.15.7:0/1158074300 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01000fd80 con 0x5639b62f4e40 2024-06-13T07:11:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.929+0000 7fc014c00700 1 -- 172.21.15.7:0/1158074300 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01001e040 con 0x5639b62f4e40 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1158074300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 msgr2=0x5639b6382600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc010011730 tx=0x7fc010009260 comp rx=0 tx=0).stop 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1158074300 shutdown_connections 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b6384c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b62f4e40 0x5639b6382600 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1158074300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b63785b0 0x5639b63846f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1158074300 >> 172.21.15.7:0/1158074300 conn(0x5639b62a5170 msgr2=0x5639b6373fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1158074300 shutdown_connections 2024-06-13T07:11:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.930+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1158074300 wait complete. 2024-06-13T07:11:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 -- start start 2024-06-13T07:11:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 Processor -- start 2024-06-13T07:11:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b639b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b639a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201aead0 con 0x5639b62f4e40 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201aff40 con 0x5639b6383300 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.931+0000 7fc0316ddb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201af1a0 con 0x5639b63785b0 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc0316ddb80 10 client.0 _ll_register_callbacks cb 0x5639b6264720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01fe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b639b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37904/0 (socket says 172.21.15.7:37904) 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 -- 172.21.15.7:0/1668250490 learned_addr learned my addr 172.21.15.7:0/1668250490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc024e00700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b639a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 -- 172.21.15.7:0/1668250490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 msgr2=0x5639b639a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b639a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 -- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 msgr2=0x5639b639b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b639b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01f400700 1 -- 172.21.15.7:0/1668250490 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b639a820 con 0x5639b63785b0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b639b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.932+0000 7fc024e00700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b639a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc01f400700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c5b0 tx=0x7fc00c09b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4026cfa467b7c57 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc00a000700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0b9060 con 0x5639b63785b0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01000fd80 con 0x5639b63785b0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fc00c0cca20 con 0x5639b63785b0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc00c01c040 con 0x5639b63785b0 2024-06-13T07:11:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc0316ddb80 10 client.27475 fetch_fsmap learned FSMap version 45 2024-06-13T07:11:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.933+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5639b638b340 con 0x5639b63785b0 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01000fd80 con 0x5639b63785b0 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 45) ==== 8358+0+0 (secure 0 0 0) 0x7fc00c0a3e50 con 0x5639b63785b0 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 10 client.27475 fetch_fsmap finished waiting for FSMap version 45 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 10 client.27475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 20 client.27475 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b638ba30 0x5639b63ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 4 client.27475 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:11:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 --> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639b62d2580 con 0x5639b638ba30 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.934+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b638ba30 0x5639b63ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc01fe00700 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b638ba30 0x5639b63ffa70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb07b2dc20c3a857 server_cookie=edb9bafac26201e4 in_seq=0 out_seq=0 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc00a000700 10 client.27475 ms_handle_connect on v2:172.21.15.7:6840/1761725822 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc00a000700 1 -- 172.21.15.7:0/1668250490 <== mds.1 v2:172.21.15.7:6840/1761725822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639b62d2580 con 0x5639b638ba30 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc00a000700 10 client.27475 handle_command_reply: tid=0 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 client.27475 shutdown 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 msgr2=0x5639b6399a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c5b0 tx=0x7fc00c09b2c0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 shutdown_connections 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5639b6383300 0x5639b639a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b62f4e40 0x5639b639b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.7:6840/1761725822,v1:172.21.15.7:6841/1761725822] conn(0x5639b638ba30 0x5639b63ffa70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 --2- 172.21.15.7:0/1668250490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639b63785b0 0x5639b6399a80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 >> 172.21.15.7:0/1668250490 conn(0x5639b62a5170 msgr2=0x5639b637fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.935+0000 7fc00a000700 0 client.27475 ms_handle_reset on v2:172.21.15.7:6840/1761725822 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.936+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 shutdown_connections 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.936+0000 7fc0316ddb80 1 -- 172.21.15.7:0/1668250490 wait complete. 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.936+0000 7fc0316ddb80 20 client.27475 trim_cache size 0 max 16384 2024-06-13T07:11:21.939 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 msgr2=0x7fc0040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 0x7fc0040a5160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b780 tx=0x7fc018006210 comp rx=0 tx=0).stop 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 msgr2=0x7fc02019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 0x7fc02019b3f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00ed00 comp rx=0 tx=0).stop 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 shutdown_connections 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc02010eec0 0x7fc02019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0200746c0 0x7fc02019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fc0040a2ca0 0x7fc0040a5160 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 --2- 172.21.15.7:0/371908128 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc020074fe0 0x7fc02019b3f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 >> 172.21.15.7:0/371908128 conn(0x7fc020079640 msgr2=0x7fc0200ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 shutdown_connections 2024-06-13T07:11:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:21.937+0000 7fc026e00700 1 -- 172.21.15.7:0/371908128 wait complete. 2024-06-13T07:11:22.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:21 smithi189 ceph-mon[36616]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 623 op/s 2024-06-13T07:11:22.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:21 smithi175 ceph-mon[36886]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 623 op/s 2024-06-13T07:11:22.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:21 smithi007 ceph-mon[32077]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.3 MiB/s wr, 623 op/s 2024-06-13T07:11:22.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-13T07:11:23.432 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:23.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:23 smithi007 ceph-mon[32077]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.9 MiB/s wr, 465 op/s 2024-06-13T07:11:24.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:23 smithi189 ceph-mon[36616]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.9 MiB/s wr, 465 op/s 2024-06-13T07:11:24.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:23 smithi175 ceph-mon[36886]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.9 MiB/s wr, 465 op/s 2024-06-13T07:11:26.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:25 smithi007 ceph-mon[32077]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.8 MiB/s wr, 465 op/s 2024-06-13T07:11:26.082 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:25 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:25 smithi189 ceph-mon[36616]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.8 MiB/s wr, 465 op/s 2024-06-13T07:11:26.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:25 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:25 smithi175 ceph-mon[36886]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.8 MiB/s wr, 465 op/s 2024-06-13T07:11:26.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:25 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 -- 172.21.15.7:0/2892713627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae81046b0 msgr2=0x7f1ae8104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 --2- 172.21.15.7:0/2892713627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae81046b0 0x7f1ae8104b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009990 tx=0x7f1ad4009ca0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 -- 172.21.15.7:0/2892713627 shutdown_connections 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 --2- 172.21.15.7:0/2892713627 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae8105070 0x7f1ae8111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 --2- 172.21.15.7:0/2892713627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae810a6b0 0x7f1ae810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 --2- 172.21.15.7:0/2892713627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae81046b0 0x7f1ae8104b30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 -- 172.21.15.7:0/2892713627 >> 172.21.15.7:0/2892713627 conn(0x7f1ae8100160 msgr2=0x7f1ae8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 -- 172.21.15.7:0/2892713627 shutdown_connections 2024-06-13T07:11:26.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.404+0000 7f1aed400700 1 -- 172.21.15.7:0/2892713627 wait complete. 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.405+0000 7f1aed400700 1 Processor -- start 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.405+0000 7f1aed400700 1 -- start start 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 0x7f1ae819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 0x7f1ae819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 0x7f1ae819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8114a90 con 0x7f1ae810a6b0 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8114910 con 0x7f1ae81046b0 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1aed400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8114c10 con 0x7f1ae8105070 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 0x7f1ae819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 0x7f1ae819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 0x7f1ae819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49708/0 (socket says 172.21.15.7:49708) 2024-06-13T07:11:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1ae6800700 1 -- 172.21.15.7:0/2319356017 learned_addr learned my addr 172.21.15.7:0/2319356017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.406+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 0x7f1ae819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 -- 172.21.15.7:0/2319356017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 msgr2=0x7f1ae819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 0x7f1ae819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 -- 172.21.15.7:0/2319356017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 msgr2=0x7f1ae819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 0x7f1ae819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 -- 172.21.15.7:0/2319356017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae819feb0 con 0x7f1ae810a6b0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 0x7f1ae819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1ae7200700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 0x7f1ae819f770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000efc0 tx=0x7f1ae000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1adf400700 1 -- 172.21.15.7:0/2319356017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00092e0 con 0x7f1ae810a6b0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.407+0000 7f1adf400700 1 -- 172.21.15.7:0/2319356017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae0004bb0 con 0x7f1ae810a6b0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.408+0000 7f1adf400700 1 -- 172.21.15.7:0/2319356017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0004280 con 0x7f1ae810a6b0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.408+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae81a0140 con 0x7f1ae810a6b0 2024-06-13T07:11:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.408+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81a7a20 con 0x7f1ae810a6b0 2024-06-13T07:11:26.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.411+0000 7f1adf400700 1 -- 172.21.15.7:0/2319356017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1ae001b030 con 0x7f1ae810a6b0 2024-06-13T07:11:26.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.412+0000 7f1adf400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 0x7f1acc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.412+0000 7f1af7d99b80 1 Processor -- start 2024-06-13T07:11:26.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.412+0000 7f1adf400700 1 -- 172.21.15.7:0/2319356017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1ae00ceeb0 con 0x7f1ae810a6b0 2024-06-13T07:11:26.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 -- start start 2024-06-13T07:11:26.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 0x7f1acc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 0x5616e0bec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 0x5616e0bf96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5616e0afba10 con 0x5616e0bec5b0 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616e0b3eb00 con 0x5616e0b68e40 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616e0af98f0 con 0x5616e0bf7300 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.413+0000 7f1ae5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47926/0 (socket says 172.21.15.7:47926) 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 -- 172.21.15.7:0/1198563051 learned_addr learned my addr 172.21.15.7:0/1198563051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae7200700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 0x5616e0bf96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae6800700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 0x5616e0bec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 -- 172.21.15.7:0/1198563051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 msgr2=0x5616e0bf96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 0x5616e0bf96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 -- 172.21.15.7:0/1198563051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 msgr2=0x5616e0bec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 0x5616e0bec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae5e00700 1 -- 172.21.15.7:0/1198563051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e0ae6f90 con 0x5616e0bec5b0 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae7200700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 0x5616e0bf96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae6800700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 0x5616e0bec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.414+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 0x7f1acc0a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009cf0 tx=0x7f1ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.415+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4015000 tx=0x7f1ad40096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370979f142249010 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.415+0000 7f1ad3e00700 1 -- 172.21.15.7:0/1198563051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400b880 con 0x5616e0bec5b0 2024-06-13T07:11:26.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.415+0000 7f1ad3e00700 1 -- 172.21.15.7:0/1198563051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad4027070 con 0x5616e0bec5b0 2024-06-13T07:11:26.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.415+0000 7f1ad3e00700 1 -- 172.21.15.7:0/1198563051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400f460 con 0x5616e0bec5b0 2024-06-13T07:11:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 -- 172.21.15.7:0/1198563051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 msgr2=0x5616e0beca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4015000 tx=0x7f1ad40096e0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 -- 172.21.15.7:0/1198563051 shutdown_connections 2024-06-13T07:11:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0bf7300 0x5616e0bf96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0b68e40 0x5616e0bec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/1198563051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bec5b0 0x5616e0beca30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 -- 172.21.15.7:0/1198563051 >> 172.21.15.7:0/1198563051 conn(0x5616e0b19170 msgr2=0x5616e0be84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 -- 172.21.15.7:0/1198563051 shutdown_connections 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.416+0000 7f1af7d99b80 1 -- 172.21.15.7:0/1198563051 wait complete. 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.417+0000 7f1af7d99b80 1 -- start start 2024-06-13T07:11:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.417+0000 7f1af7d99b80 1 Processor -- start 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bf3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bf4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5616e0adaf60 con 0x5616e0bf7300 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5616e0b3aac0 con 0x5616e0bec5b0 2024-06-13T07:11:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616e0bf9f60 con 0x5616e0b68e40 2024-06-13T07:11:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1af7d99b80 10 client.0 _ll_register_callbacks cb 0x5616e0ad8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1ae5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bf3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37940/0 (socket says 172.21.15.7:37940) 2024-06-13T07:11:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bf4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.418+0000 7f1ae6800700 1 -- 172.21.15.7:0/2169513613 learned_addr learned my addr 172.21.15.7:0/2169513613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 -- 172.21.15.7:0/2169513613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 msgr2=0x5616e0bf6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 -- 172.21.15.7:0/2169513613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 msgr2=0x5616e0bf4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bf4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 -- 172.21.15.7:0/2169513613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e0a8d8d0 con 0x5616e0bec5b0 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae7200700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bf4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bf3d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009ce0 tx=0x7f1ad4003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efbcba10dfe4baf server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ad0c00700 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400bc90 con 0x5616e0bec5b0 2024-06-13T07:11:26.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad4027070 con 0x5616e0bec5b0 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.419+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mon.2 v2:172.21.15.175:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1ad400be30 con 0x5616e0bec5b0 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1ad403a050 con 0x5616e0bec5b0 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1af7d99b80 10 client.27602 fetch_fsmap learned FSMap version 46 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5616e0c04640 con 0x5616e0bec5b0 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mon.2 v2:172.21.15.175:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad4027070 con 0x5616e0bec5b0 2024-06-13T07:11:26.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mon.2 v2:172.21.15.175:3300/0 5 ==== fsmap(e 46) ==== 8362+0+0 (secure 0 0 0) 0x7f1ad402b4b0 con 0x5616e0bec5b0 2024-06-13T07:11:26.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1af7d99b80 10 client.27602 fetch_fsmap finished waiting for FSMap version 46 2024-06-13T07:11:26.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1af7d99b80 10 client.27602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:26.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.420+0000 7f1af7d99b80 20 client.27602 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:26.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c06b10 0x5616e0c08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1af7d99b80 4 client.27602 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616e0b4c000 con 0x5616e0c06b10 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c06b10 0x5616e0c08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c06b10 0x5616e0c08f00 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d98e62b0c0380b server_cookie=a46afbc145c33ea1 in_seq=0 out_seq=0 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.421+0000 7f1ad0c00700 10 client.27602 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.422+0000 7f1ad0c00700 1 -- 172.21.15.7:0/2169513613 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5616e0b4c000 con 0x5616e0c06b10 2024-06-13T07:11:26.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.422+0000 7f1ad0c00700 10 client.27602 handle_command_reply: tid=0 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 client.27602 shutdown 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 msgr2=0x5616e0bf3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bf3d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009ce0 tx=0x7f1ad4003910 comp rx=0 tx=0).stop 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 shutdown_connections 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c06b10 0x5616e0c08f00 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bf4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1ad0c00700 0 client.27602 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:26.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2169513613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bf3d30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 >> 172.21.15.7:0/2169513613 conn(0x5616e0b19170 msgr2=0x5616e0be7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 shutdown_connections 2024-06-13T07:11:26.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2169513613 wait complete. 2024-06-13T07:11:26.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.423+0000 7f1af7d99b80 20 client.27602 trim_cache size 0 max 16384 2024-06-13T07:11:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.440+0000 7f1af7d99b80 1 Processor -- start 2024-06-13T07:11:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.440+0000 7f1af7d99b80 1 -- start start 2024-06-13T07:11:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81b2cd0 con 0x5616e0bf7300 2024-06-13T07:11:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81b04a0 con 0x5616e0bec5b0 2024-06-13T07:11:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ae2f0 con 0x5616e0b68e40 2024-06-13T07:11:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bee5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47936/0 (socket says 172.21.15.7:47936) 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37950/0 (socket says 172.21.15.7:37950) 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae7200700 1 -- 172.21.15.7:0/2150323497 learned_addr learned my addr 172.21.15.7:0/2150323497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.441+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 -- 172.21.15.7:0/2150323497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 msgr2=0x5616e0bee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 -- 172.21.15.7:0/2150323497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 msgr2=0x5616e0bee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 -- 172.21.15.7:0/2150323497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e0bf45f0 con 0x5616e0b68e40 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ae6800700 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5616e0b3eb00 tx=0x7f1ad801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c3497e810f0414b server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ad3400700 1 -- 172.21.15.7:0/2150323497 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8022040 con 0x5616e0b68e40 2024-06-13T07:11:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ad3400700 1 -- 172.21.15.7:0/2150323497 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8004d10 con 0x5616e0b68e40 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.442+0000 7f1ad3400700 1 -- 172.21.15.7:0/2150323497 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8029690 con 0x5616e0b68e40 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2150323497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 msgr2=0x5616e0bf9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5616e0b3eb00 tx=0x7f1ad801c000 comp rx=0 tx=0).stop 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2150323497 shutdown_connections 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0bee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0bf9480 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/2150323497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0bee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2150323497 >> 172.21.15.7:0/2150323497 conn(0x5616e0b19170 msgr2=0x5616e0be7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2150323497 shutdown_connections 2024-06-13T07:11:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.443+0000 7f1af7d99b80 1 -- 172.21.15.7:0/2150323497 wait complete. 2024-06-13T07:11:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.444+0000 7f1af7d99b80 1 -- start start 2024-06-13T07:11:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.444+0000 7f1af7d99b80 1 Processor -- start 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0c06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0c074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ae960 con 0x5616e0bf7300 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81afdd0 con 0x5616e0bec5b0 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81af030 con 0x5616e0b68e40 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1af7d99b80 10 client.0 _ll_register_callbacks cb 0x5616e0ad8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0c06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae5e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0c074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47942/0 (socket says 172.21.15.7:47942) 2024-06-13T07:11:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae7200700 1 -- 172.21.15.7:0/516692370 learned_addr learned my addr 172.21.15.7:0/516692370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 -- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 msgr2=0x5616e0c074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0c074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 -- 172.21.15.7:0/516692370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 msgr2=0x5616e0c6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae6800700 1 -- 172.21.15.7:0/516692370 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e0c07d90 con 0x5616e0b68e40 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.445+0000 7f1ae7200700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1ae6800700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0c06f90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5616e0b3eb00 tx=0x7f1ad8009150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4161be8beafc6cd server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1ad0c00700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8004950 con 0x5616e0b68e40 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ad8004d10 con 0x5616e0b68e40 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f1ad8004af0 con 0x5616e0b68e40 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1ad8047050 con 0x5616e0b68e40 2024-06-13T07:11:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.446+0000 7f1af7d99b80 10 client.27493 fetch_fsmap learned FSMap version 46 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5616e0c70ac0 con 0x5616e0b68e40 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8004d10 con 0x5616e0b68e40 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 46) ==== 8362+0+0 (secure 0 0 0) 0x7f1ad802f070 con 0x5616e0b68e40 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 10 client.27493 fetch_fsmap finished waiting for FSMap version 46 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 10 client.27493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 20 client.27493 populate_metadata read hostname 'smithi007' 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c71170 0x5616e0c73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 4 client.27493 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-13T07:11:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 --> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616e0bed040 con 0x5616e0c71170 2024-06-13T07:11:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.447+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c71170 0x5616e0c73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1ae5e00700 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c71170 0x5616e0c73590 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f137d59ecc77e687 server_cookie=95e6f1c7827d5f79 in_seq=0 out_seq=0 2024-06-13T07:11:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1ad0c00700 10 client.27493 ms_handle_connect on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1ad0c00700 1 -- 172.21.15.7:0/516692370 <== mds.0 v2:172.21.15.175:6838/4167274347 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616e0bed040 con 0x5616e0c71170 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1ad0c00700 10 client.27493 handle_command_reply: tid=0 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1af7d99b80 1 client.27493 shutdown 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.448+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 msgr2=0x5616e0c06f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0c06f90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5616e0b3eb00 tx=0x7f1ad8009150 comp rx=0 tx=0).stop 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 shutdown_connections 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:6838/4167274347,v1:172.21.15.175:6839/4167274347] conn(0x5616e0c71170 0x5616e0c73590 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616e0bf7300 0x5616e0c6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e0b68e40 0x5616e0c06f90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 --2- 172.21.15.7:0/516692370 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5616e0bec5b0 0x5616e0c074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1ad0c00700 0 client.27493 ms_handle_reset on v2:172.21.15.175:6838/4167274347 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 >> 172.21.15.7:0/516692370 conn(0x5616e0b19170 msgr2=0x5616e0be7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 shutdown_connections 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 1 -- 172.21.15.7:0/516692370 wait complete. 2024-06-13T07:11:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.449+0000 7f1af7d99b80 20 client.27493 trim_cache size 0 max 16384 2024-06-13T07:11:26.453 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-06-13T07:11:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 msgr2=0x7f1acc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 0x7f1acc0a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009cf0 tx=0x7f1ad8009260 comp rx=0 tx=0).stop 2024-06-13T07:11:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 msgr2=0x7f1ae819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 0x7f1ae819f770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000efc0 tx=0x7f1ae000c3a0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 shutdown_connections 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ae810a6b0 0x7f1ae819f770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f1acc0a2ca0 0x7f1acc0a5160 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ae81046b0 0x7f1ae819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 --2- 172.21.15.7:0/2319356017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ae8105070 0x7f1ae819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 >> 172.21.15.7:0/2319356017 conn(0x7f1ae8100160 msgr2=0x7f1ae8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 shutdown_connections 2024-06-13T07:11:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:26.450+0000 7f1aed400700 1 -- 172.21.15.7:0/2319356017 wait complete. 2024-06-13T07:11:27.258 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_1373963c202323287783ccb099bbae1ec8f14a5d/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-13T07:11:27.259 INFO:tasks.fwd_scrub:done joining 2024-06-13T07:11:27.259 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-13T07:11:27.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-13T07:11:27.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 -- 172.21.15.7:0/3051622666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 msgr2=0x7f5290110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 --2- 172.21.15.7:0/3051622666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f5290110250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5288009ca0 tx=0x7f528802a8c0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 -- 172.21.15.7:0/3051622666 shutdown_connections 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 --2- 172.21.15.7:0/3051622666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f5290110250 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 --2- 172.21.15.7:0/3051622666 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290108f70 0x7f52901093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 --2- 172.21.15.7:0/3051622666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290107e20 0x7f5290108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 -- 172.21.15.7:0/3051622666 >> 172.21.15.7:0/3051622666 conn(0x7f5290076de0 msgr2=0x7f5290079200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 -- 172.21.15.7:0/3051622666 shutdown_connections 2024-06-13T07:11:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.523+0000 7f5294c00640 1 -- 172.21.15.7:0/3051622666 wait complete. 2024-06-13T07:11:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 Processor -- start 2024-06-13T07:11:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 -- start start 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290107e20 0x7f529019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 0x7f529019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113af0 con 0x7f5290109930 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113970 con 0x7f5290107e20 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.524+0000 7f5294c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5290113c70 con 0x7f5290108f70 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 0x7f529019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47966/0 (socket says 172.21.15.7:47966) 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528fe00640 1 -- 172.21.15.7:0/3159719432 learned_addr learned my addr 172.21.15.7:0/3159719432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528f400640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290107e20 0x7f529019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 -- 172.21.15.7:0/3159719432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290107e20 msgr2=0x7f529019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290107e20 0x7f529019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 -- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 msgr2=0x7f52901a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528ea00640 1 -- 172.21.15.7:0/3159719432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901a5cf0 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.525+0000 7f528fe00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f528ea00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 0x7f529019e610 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e9c0 tx=0x7f527c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00ccc0 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c0035f0 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52901a5fe0 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00ce60 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.526+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a64a0 con 0x7f5290108f70 2024-06-13T07:11:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.527+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529010f130 con 0x7f5290108f70 2024-06-13T07:11:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.530+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f527c0037a0 con 0x7f5290108f70 2024-06-13T07:11:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.531+0000 7f5287e00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 0x7f52740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.531+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f527c0cf7e0 con 0x7f5290108f70 2024-06-13T07:11:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.532+0000 7f528f400640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 0x7f52740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.532+0000 7f528f400640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 0x7f52740a5050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f52800095f0 tx=0x7f5280005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:27.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.535+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f527c09c3a0 con 0x7f5290108f70 2024-06-13T07:11:27.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.743+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5290062640 con 0x7f5290108f70 2024-06-13T07:11:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.746+0000 7f5287e00640 1 -- 172.21.15.7:0/3159719432 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f527c09bca0 con 0x7f5290108f70 2024-06-13T07:11:27.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:27.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:11:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 msgr2=0x7f52740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 0x7f52740a5050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f52800095f0 tx=0x7f5280005f90 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 msgr2=0x7f529019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 0x7f529019e610 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e9c0 tx=0x7f527c00ee80 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 shutdown_connections 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f52740a2b90 0x7f52740a5050 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5290109930 0x7f52901a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5290108f70 0x7f529019e610 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 --2- 172.21.15.7:0/3159719432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5290107e20 0x7f529019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 >> 172.21.15.7:0/3159719432 conn(0x7f5290076de0 msgr2=0x7f529010c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 shutdown_connections 2024-06-13T07:11:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.749+0000 7f5294c00640 1 -- 172.21.15.7:0/3159719432 wait complete. 2024-06-13T07:11:27.768 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-13T06:44:44.175120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-13T06:50:10.636472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-13T06:50:15.402440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-13T06:50:28.238859+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-13T07:11:27.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-13T07:11:27.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 -- 172.21.15.7:0/3774400432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 msgr2=0x7f3280076780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 --2- 172.21.15.7:0/3774400432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f3280076780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3270009900 tx=0x7f327002a410 comp rx=0 tx=0).stop 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 -- 172.21.15.7:0/3774400432 shutdown_connections 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 --2- 172.21.15.7:0/3774400432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3280077f40 0x7f32800787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 --2- 172.21.15.7:0/3774400432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32800775a0 0x7f3280077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 --2- 172.21.15.7:0/3774400432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f3280076780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 -- 172.21.15.7:0/3774400432 >> 172.21.15.7:0/3774400432 conn(0x7f32800ff050 msgr2=0x7f3280101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 -- 172.21.15.7:0/3774400432 shutdown_connections 2024-06-13T07:11:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.989+0000 7f327f200640 1 -- 172.21.15.7:0/3774400432 wait complete. 2024-06-13T07:11:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 Processor -- start 2024-06-13T07:11:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 -- start start 2024-06-13T07:11:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f328019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3280077f40 0x7f32801a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3280113b50 con 0x7f3280076360 2024-06-13T07:11:27.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32801139d0 con 0x7f32800775a0 2024-06-13T07:11:27.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.991+0000 7f327f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3280113cd0 con 0x7f3280077f40 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f327de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f328019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49794/0 (socket says 172.21.15.7:49794) 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 -- 172.21.15.7:0/960287550 learned_addr learned my addr 172.21.15.7:0/960287550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f327e800640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3280077f40 0x7f32801a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 -- 172.21.15.7:0/960287550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3280077f40 msgr2=0x7f32801a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3280077f40 0x7f32801a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 -- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 msgr2=0x7f328019e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f328019e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 -- 172.21.15.7:0/960287550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801a5e10 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.992+0000 7f3277e00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f326800d950 tx=0x7f326800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.993+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268017030 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.993+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3268004020 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.993+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801a6100 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.993+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680047a0 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.993+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801a6650 con 0x7f32800775a0 2024-06-13T07:11:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.994+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3280110430 con 0x7f32800775a0 2024-06-13T07:11:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.996+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f32680041d0 con 0x7f32800775a0 2024-06-13T07:11:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.997+0000 7f3276a00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:27.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.998+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f32680cedd0 con 0x7f32800775a0 2024-06-13T07:11:27.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.998+0000 7f327de00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:27.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:27.998+0000 7f327de00640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 0x7f32640a5050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3270031440 tx=0x7f32700315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.002+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f326809b990 con 0x7f32800775a0 2024-06-13T07:11:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:27 smithi189 ceph-mon[36616]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-06-13T07:11:28.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:27 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3159719432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:11:28.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:27 smithi175 ceph-mon[36886]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-06-13T07:11:28.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:27 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3159719432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:11:28.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:27 smithi007 ceph-mon[32077]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.3 MiB/s wr, 279 op/s 2024-06-13T07:11:28.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:27 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3159719432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:11:28.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.207+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32800625a0 con 0x7f32800775a0 2024-06-13T07:11:28.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.208+0000 7f3276a00640 1 -- 172.21.15.7:0/960287550 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f326809b290 con 0x7f32800775a0 2024-06-13T07:11:28.208 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-06-13T07:11:28.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.210+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 msgr2=0x7f32640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.210+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 0x7f32640a5050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3270031440 tx=0x7f32700315b0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.210+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 msgr2=0x7f328019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.210+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f326800d950 tx=0x7f326800de10 comp rx=0 tx=0).stop 2024-06-13T07:11:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 shutdown_connections 2024-06-13T07:11:28.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f32640a2b90 0x7f32640a5050 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3280077f40 0x7f32801a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32800775a0 0x7f328019e730 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 --2- 172.21.15.7:0/960287550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3280076360 0x7f328019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 >> 172.21.15.7:0/960287550 conn(0x7f32800ff050 msgr2=0x7f3280100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 shutdown_connections 2024-06-13T07:11:28.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.211+0000 7f327f200640 1 -- 172.21.15.7:0/960287550 wait complete. 2024-06-13T07:11:28.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-13T07:11:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.448+0000 7fde9fc00640 1 -- 172.21.15.7:0/2942846905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980775a0 msgr2=0x7fde98077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.448+0000 7fde9fc00640 1 --2- 172.21.15.7:0/2942846905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980775a0 0x7fde98077a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fde84009900 tx=0x7fde8402a3f0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 -- 172.21.15.7:0/2942846905 shutdown_connections 2024-06-13T07:11:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 --2- 172.21.15.7:0/2942846905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98077f40 0x7fde980787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 --2- 172.21.15.7:0/2942846905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980775a0 0x7fde98077a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 --2- 172.21.15.7:0/2942846905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98076360 0x7fde98076780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 -- 172.21.15.7:0/2942846905 >> 172.21.15.7:0/2942846905 conn(0x7fde980ff190 msgr2=0x7fde981015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 -- 172.21.15.7:0/2942846905 shutdown_connections 2024-06-13T07:11:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.449+0000 7fde9fc00640 1 -- 172.21.15.7:0/2942846905 wait complete. 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 Processor -- start 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 -- start start 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 0x7fde9819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 0x7fde9819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 0x7fde981a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113cd0 con 0x7fde98076360 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113b50 con 0x7fde98077f40 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113e50 con 0x7fde980775a0 2024-06-13T07:11:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 0x7fde9819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.450+0000 7fde9dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 0x7fde981a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 0x7fde9819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48004/0 (socket says 172.21.15.7:48004) 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9d200640 1 -- 172.21.15.7:0/1146636201 learned_addr learned my addr 172.21.15.7:0/1146636201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde97e00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 0x7fde9819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 -- 172.21.15.7:0/1146636201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 msgr2=0x7fde9819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 0x7fde9819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 -- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 msgr2=0x7fde9819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 0x7fde9819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 -- 172.21.15.7:0/1146636201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde981a5f40 con 0x7fde98077f40 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde97e00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 0x7fde9819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9dc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 0x7fde981a58d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fde90009fd0 tx=0x7fde9000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000ce20 con 0x7fde98077f40 2024-06-13T07:11:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde981a6230 con 0x7fde98077f40 2024-06-13T07:11:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90005020 con 0x7fde98077f40 2024-06-13T07:11:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.451+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90010720 con 0x7fde98077f40 2024-06-13T07:11:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.452+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a6750 con 0x7fde98077f40 2024-06-13T07:11:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.453+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde98110570 con 0x7fde98077f40 2024-06-13T07:11:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.455+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fde90005540 con 0x7fde98077f40 2024-06-13T07:11:28.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.456+0000 7fde95600640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 0x7fde800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.456+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fde900cfaa0 con 0x7fde98077f40 2024-06-13T07:11:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.457+0000 7fde9d200640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 0x7fde800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.457+0000 7fde9d200640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 0x7fde800a5050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009c90 tx=0x7fde8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.459+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde9009c660 con 0x7fde98077f40 2024-06-13T07:11:28.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.663+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde980625a0 con 0x7fde98077f40 2024-06-13T07:11:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.664+0000 7fde95600640 1 -- 172.21.15.7:0/1146636201 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fde9009bf60 con 0x7fde98077f40 2024-06-13T07:11:28.665 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 msgr2=0x7fde800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 0x7fde800a5050 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009c90 tx=0x7fde8c005f90 comp rx=0 tx=0).stop 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 msgr2=0x7fde981a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 0x7fde981a58d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fde90009fd0 tx=0x7fde9000ef10 comp rx=0 tx=0).stop 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 shutdown_connections 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fde800a2b90 0x7fde800a5050 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde98077f40 0x7fde981a58d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde980775a0 0x7fde9819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 --2- 172.21.15.7:0/1146636201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde98076360 0x7fde9819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.667+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 >> 172.21.15.7:0/1146636201 conn(0x7fde980ff190 msgr2=0x7fde98100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.668+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 shutdown_connections 2024-06-13T07:11:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.668+0000 7fde9fc00640 1 -- 172.21.15.7:0/1146636201 wait complete. 2024-06-13T07:11:28.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 -- 172.21.15.7:0/3072055200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac107e90 msgr2=0x7f04ac1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 --2- 172.21.15.7:0/3072055200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac107e90 0x7f04ac1082b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f049c009900 tx=0x7f049c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 -- 172.21.15.7:0/3072055200 shutdown_connections 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 --2- 172.21.15.7:0/3072055200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 0x7f04ac110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 --2- 172.21.15.7:0/3072055200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac109070 0x7f04ac1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 --2- 172.21.15.7:0/3072055200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac107e90 0x7f04ac1082b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 -- 172.21.15.7:0/3072055200 >> 172.21.15.7:0/3072055200 conn(0x7f04ac076e70 msgr2=0x7f04ac0792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.891+0000 7f04b1800640 1 -- 172.21.15.7:0/3072055200 shutdown_connections 2024-06-13T07:11:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.892+0000 7f04b1800640 1 -- 172.21.15.7:0/3072055200 wait complete. 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.892+0000 7f04b1800640 1 Processor -- start 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 -- start start 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac109070 0x7f04ac19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 0x7f04ac1a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac113bb0 con 0x7f04ac109a30 2024-06-13T07:11:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac113a30 con 0x7f04ac107e90 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b1800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac113d30 con 0x7f04ac109070 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04abe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04abe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49836/0 (socket says 172.21.15.7:49836) 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04abe00640 1 -- 172.21.15.7:0/1901100057 learned_addr learned my addr 172.21.15.7:0/1901100057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04a3e00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac109070 0x7f04ac19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.893+0000 7f04b0e00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 0x7f04ac1a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 -- 172.21.15.7:0/1901100057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac109070 msgr2=0x7f04ac19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac109070 0x7f04ac19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 -- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 msgr2=0x7f04ac1a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 0x7f04ac1a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 -- 172.21.15.7:0/1901100057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac1a5e00 con 0x7f04ac107e90 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04abe00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f049c031440 tx=0x7f049c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c03f030 con 0x7f04ac107e90 2024-06-13T07:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f049c003ce0 con 0x7f04ac107e90 2024-06-13T07:11:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac1a6090 con 0x7f04ac107e90 2024-06-13T07:11:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.894+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c035070 con 0x7f04ac107e90 2024-06-13T07:11:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.895+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1a6580 con 0x7f04ac107e90 2024-06-13T07:11:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.896+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac10f230 con 0x7f04ac107e90 2024-06-13T07:11:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.897+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f049c042030 con 0x7f04ac107e90 2024-06-13T07:11:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.899+0000 7f04a9600640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 0x7f04900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.899+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f049c0ef7c0 con 0x7f04ac107e90 2024-06-13T07:11:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.899+0000 7f04a3e00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 0x7f04900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.900+0000 7f04a3e00640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 0x7f04900a5050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f04ac19f6f0 tx=0x7f0494005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:28.901+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f049c0bc380 con 0x7f04ac107e90 2024-06-13T07:11:29.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.102+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04ac02d470 con 0x7f04ac107e90 2024-06-13T07:11:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.103+0000 7f04a9600640 1 -- 172.21.15.7:0/1901100057 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f049c0bbc80 con 0x7f04ac107e90 2024-06-13T07:11:29.104 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 msgr2=0x7f04900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 0x7f04900a5050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f04ac19f6f0 tx=0x7f0494005f90 comp rx=0 tx=0).stop 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 msgr2=0x7f04ac19e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f049c031440 tx=0x7f049c032ce0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 shutdown_connections 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f04900a2b90 0x7f04900a5050 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac109a30 0x7f04ac1a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04ac109070 0x7f04ac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 --2- 172.21.15.7:0/1901100057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04ac107e90 0x7f04ac19e280 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.106+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 >> 172.21.15.7:0/1901100057 conn(0x7f04ac076e70 msgr2=0x7f04ac10c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.107+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 shutdown_connections 2024-06-13T07:11:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.107+0000 7f04b1800640 1 -- 172.21.15.7:0/1901100057 wait complete. 2024-06-13T07:11:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:28 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/960287550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:28 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1146636201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:28 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/960287550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:28 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1146636201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-13T07:11:29.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.331+0000 7f6763e00640 1 -- 172.21.15.7:0/1662659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109070 msgr2=0x7f67641094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.331+0000 7f6763e00640 1 --2- 172.21.15.7:0/1662659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109070 0x7f67641094f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c02a3f0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 -- 172.21.15.7:0/1662659964 shutdown_connections 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 --2- 172.21.15.7:0/1662659964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764109a30 0x7f6764110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 --2- 172.21.15.7:0/1662659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109070 0x7f67641094f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 --2- 172.21.15.7:0/1662659964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764107e90 0x7f67641082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 -- 172.21.15.7:0/1662659964 >> 172.21.15.7:0/1662659964 conn(0x7f6764076e70 msgr2=0x7f67640792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 -- 172.21.15.7:0/1662659964 shutdown_connections 2024-06-13T07:11:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.332+0000 7f6763e00640 1 -- 172.21.15.7:0/1662659964 wait complete. 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.333+0000 7f6763e00640 1 Processor -- start 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.333+0000 7f6763e00640 1 -- start start 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.333+0000 7f6763e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764109070 0x7f676419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 0x7f67641a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6764113be0 con 0x7f6764109070 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6764113a60 con 0x7f6764107e90 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6764113d60 con 0x7f6764109a30 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f675a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764109070 0x7f676419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:49854/0 (socket says 172.21.15.7:49854) 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 -- 172.21.15.7:0/962983984 learned_addr learned my addr 172.21.15.7:0/962983984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763400640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 0x7f67641a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 -- 172.21.15.7:0/962983984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 msgr2=0x7f67641a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 0x7f67641a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 -- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764109070 msgr2=0x7f676419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764109070 0x7f676419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6762a00640 1 -- 172.21.15.7:0/962983984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67641a5db0 con 0x7f6764107e90 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.334+0000 7f6763400640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 0x7f67641a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:11:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f6762a00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f675400e9c0 tx=0x7f675400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675400cd80 con 0x7f6764107e90 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67641a60a0 con 0x7f6764107e90 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67540035f0 con 0x7f6764107e90 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754010690 con 0x7f6764107e90 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.335+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a6610 con 0x7f6764107e90 2024-06-13T07:11:29.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:28 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/960287550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:28 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1146636201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-13T07:11:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.336+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676410f1d0 con 0x7f6764107e90 2024-06-13T07:11:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.339+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f67540037a0 con 0x7f6764107e90 2024-06-13T07:11:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.340+0000 7f675be00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 0x7f67480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.340+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f67540cf860 con 0x7f6764107e90 2024-06-13T07:11:29.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.341+0000 7f675a000640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 0x7f67480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.341+0000 7f675a000640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 0x7f67480a5050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f674c031440 tx=0x7f674c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.344+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f675409c420 con 0x7f6764107e90 2024-06-13T07:11:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.551+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67640625a0 con 0x7f6764107e90 2024-06-13T07:11:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.552+0000 7f675be00640 1 -- 172.21.15.7:0/962983984 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f675409bd20 con 0x7f6764107e90 2024-06-13T07:11:29.552 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.554+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 msgr2=0x7f67480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.554+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 0x7f67480a5050 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f674c031440 tx=0x7f674c0315b0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.554+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 msgr2=0x7f676419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.554+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f675400e9c0 tx=0x7f675400ee80 comp rx=0 tx=0).stop 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 shutdown_connections 2024-06-13T07:11:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f67480a2b90 0x7f67480a5050 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6764109a30 0x7f67641a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6764109070 0x7f676419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 --2- 172.21.15.7:0/962983984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6764107e90 0x7f676419e190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 >> 172.21.15.7:0/962983984 conn(0x7f6764076e70 msgr2=0x7f676410c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 shutdown_connections 2024-06-13T07:11:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.555+0000 7f6763e00640 1 -- 172.21.15.7:0/962983984 wait complete. 2024-06-13T07:11:29.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 -- 172.21.15.7:0/1959145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a30104c10 msgr2=0x7f8a30105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 --2- 172.21.15.7:0/1959145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a30104c10 0x7f8a30105090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009860 tx=0x7f8a2002a340 comp rx=0 tx=0).stop 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 -- 172.21.15.7:0/1959145037 shutdown_connections 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 --2- 172.21.15.7:0/1959145037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a301055d0 0x7f8a3010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 --2- 172.21.15.7:0/1959145037 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a30104c10 0x7f8a30105090 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 --2- 172.21.15.7:0/1959145037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30103a20 0x7f8a30103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 -- 172.21.15.7:0/1959145037 >> 172.21.15.7:0/1959145037 conn(0x7f8a300fefb0 msgr2=0x7f8a301013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 -- 172.21.15.7:0/1959145037 shutdown_connections 2024-06-13T07:11:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.791+0000 7f8a37000640 1 -- 172.21.15.7:0/1959145037 wait complete. 2024-06-13T07:11:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 Processor -- start 2024-06-13T07:11:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 -- start start 2024-06-13T07:11:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a30103a20 0x7f8a30199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a301055d0 0x7f8a301a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010f680 con 0x7f8a30103a20 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010f500 con 0x7f8a301055d0 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.793+0000 7f8a37000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3010f800 con 0x7f8a30104c10 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a30103a20 0x7f8a30199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:38062/0 (socket says 172.21.15.7:38062) 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 -- 172.21.15.7:0/710196003 learned_addr learned my addr 172.21.15.7:0/710196003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a36600640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a301055d0 0x7f8a301a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 -- 172.21.15.7:0/710196003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a301055d0 msgr2=0x7f8a301a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a301055d0 0x7f8a301a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 -- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a30103a20 msgr2=0x7f8a30199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a30103a20 0x7f8a30199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 -- 172.21.15.7:0/710196003 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a301a1920 con 0x7f8a30104c10 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.794+0000 7f8a35200640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005940 tx=0x7f8a20033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.795+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2003f020 con 0x7f8a30104c10 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.795+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a20030850 con 0x7f8a30104c10 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.795+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301a1bb0 con 0x7f8a30104c10 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.795+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20037040 con 0x7f8a30104c10 2024-06-13T07:11:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.795+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a20a0 con 0x7f8a30104c10 2024-06-13T07:11:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.796+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3010adc0 con 0x7f8a30104c10 2024-06-13T07:11:29.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.798+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8a20042020 con 0x7f8a30104c10 2024-06-13T07:11:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.800+0000 7f8a2e000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 0x7f8a180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:11:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.800+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8a200ef7c0 con 0x7f8a30104c10 2024-06-13T07:11:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.800+0000 7f8a35c00640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 0x7f8a180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:11:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.801+0000 7f8a35c00640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 0x7f8a180a5050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a3019b1e0 tx=0x7f8a24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:11:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:29.804+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a20049020 con 0x7f8a30104c10 2024-06-13T07:11:30.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.067+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8a300625d0 con 0x7f8a30104c10 2024-06-13T07:11:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.078+0000 7f8a2e000640 1 -- 172.21.15.7:0/710196003 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740363 (secure 0 0 0) 0x7f8a200bc380 con 0x7f8a30104c10 2024-06-13T07:11:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:11:30.090 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:30.090 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 18, 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2901233239 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-06-13T07:11:30.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2901233239 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/2901233239", 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-06-13T06:41:59.456160+0000", 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24112, 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-06-13T07:11:30.092 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-13T07:11:30.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-13T07:11:30.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-13T07:11:30.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.102 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.103 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-13T07:11:30.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.105 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-13T07:11:30.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-13T07:11:30.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-13T07:11:30.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.117 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-13T07:11:30.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-13T07:11:30.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-13T07:11:30.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.132 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:11:30.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-13T07:11:30.136 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-13T07:11:30.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-13T07:11:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.145 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-13T07:11:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-13T07:11:30.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-13T07:11:30.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-13T07:11:30.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-13T07:11:30.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-13T07:11:30.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.172 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:11:30.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:11:30.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-13T07:11:30.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-13T07:11:30.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.180 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:11:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:11:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:11:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-13T07:11:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-13T07:11:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:11:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:11:30.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:11:30.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.214 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:11:30.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:11:30.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.225 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:11:30.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:11:30.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:11:30.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.233 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-13T07:11:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-13T07:11:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-13T07:11:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:11:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-13T07:11:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-13T07:11:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-13T07:11:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-13T07:11:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-13T07:11:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-13T07:11:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-13T07:11:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-13T07:11:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:11:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-13T07:11:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-13T07:11:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-13T07:11:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-13T07:11:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.353 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.354 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.357 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:11:30.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-13T07:11:30.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.365 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.366 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.369 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.371 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.372 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.377 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.380 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.381 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.388 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:30.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.393 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.399 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.402 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.405 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.406 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.417 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.419 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-13T07:11:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-13T07:11:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-13T07:11:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-13T07:11:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-13T07:11:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-13T07:11:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.529 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.540 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.543 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:11:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-13T07:11:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-13T07:11:30.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-13T07:11:30.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24116, 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.566 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.568 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.570 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.571 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-13T07:11:30.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-13T07:11:30.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-13T07:11:30.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.582 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.583 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.584 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-13T07:11:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.586 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.588 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-13T07:11:30.594 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.599 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:11:30.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:11:30.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:11:30.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:11:30.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.606 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-13T07:11:30.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:11:30.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:11:30.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:11:30.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-13T07:11:30.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-13T07:11:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-13T07:11:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-13T07:11:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:11:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:11:30.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:11:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.669 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-13T07:11:30.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:11:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:11:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:11:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:11:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:11:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:11:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:11:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:11:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.695 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.725 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.732 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.734 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.741 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.748 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-13T07:11:30.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.751 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.754 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-13T07:11:30.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.760 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.764 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.778 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.784 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-13T07:11:30.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-13T07:11:30.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.791 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:11:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-13T07:11:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:11:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-13T07:11:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:11:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-13T07:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-13T07:11:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-13T07:11:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-13T07:11:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-13T07:11:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-13T07:11:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-13T07:11:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-13T07:11:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-13T07:11:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:11:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:11:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-06-13T07:11:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-06-13T07:11:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-06-13T07:11:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-06-13T07:11:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-06-13T07:11:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-06-13T07:11:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-06-13T07:11:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-06-13T07:11:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-06-13T07:11:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-06-13T07:11:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-13T07:11:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-13T07:11:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-13T07:11:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-06-13T07:11:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-13T07:11:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-13T07:11:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-13T07:11:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-13T07:11:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-13T07:11:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-13T07:11:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-06-13T07:11:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-06-13T07:11:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-06-13T07:11:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-13T07:11:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-06-13T07:11:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-06-13T07:11:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-06-13T07:11:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-06-13T07:11:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-06-13T07:11:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-13T07:11:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-13T07:11:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-13T07:11:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-13T07:11:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-06-13T07:11:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-06-13T07:11:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-06-13T07:11:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-06-13T07:11:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-06-13T07:11:31.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-06-13T07:11:31.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.158 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.159 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.160 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.161 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.162 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.170 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-06-13T07:11:31.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-06-13T07:11:31.179 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.180 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-13T07:11:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-06-13T07:11:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-06-13T07:11:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-06-13T07:11:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.323 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.328 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-06-13T07:11:31.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.332 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-06-13T07:11:31.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-06-13T07:11:31.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.353 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.354 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.359 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-06-13T07:11:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.362 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.365 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-13T07:11:31.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.384 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-06-13T07:11:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.388 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.391 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.399 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.401 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.402 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.405 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.407 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.408 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-06-13T07:11:31.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.414 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-06-13T07:11:31.421 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-13T07:11:31.422 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-13T07:11:31.423 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-13T07:11:31.424 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-13T07:11:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-06-13T07:11:31.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1594541014 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4126457919 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-13T07:11:31.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1042620941 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1504089036 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-06-13T07:11:31.429 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.088+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 msgr2=0x7f8a180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.088+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 0x7f8a180a5050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a3019b1e0 tx=0x7f8a24009260 comp rx=0 tx=0).stop 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.088+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 msgr2=0x7f8a3019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.088+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005940 tx=0x7f8a20033d90 comp rx=0 tx=0).stop 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 shutdown_connections 2024-06-13T07:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8a180a2b90 0x7f8a180a5050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a301055d0 0x7f8a301a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a30104c10 0x7f8a3019a250 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 --2- 172.21.15.7:0/710196003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a30103a20 0x7f8a30199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 >> 172.21.15.7:0/710196003 conn(0x7f8a300fefb0 msgr2=0x7f8a30107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 shutdown_connections 2024-06-13T07:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:11:30.089+0000 7f8a37000640 1 -- 172.21.15.7:0/710196003 wait complete. 2024-06-13T07:11:31.431 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:29 smithi007 ceph-mon[32077]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-13T07:11:31.432 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:29 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1901100057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.432 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:29 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/962983984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:29 smithi189 ceph-mon[36616]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-13T07:11:31.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:29 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1901100057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:29 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/962983984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:30 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/710196003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:29 smithi175 ceph-mon[36886]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:29 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1901100057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:29 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/962983984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-13T07:11:31.433 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:30 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/710196003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:11:31.441 DEBUG:tasks.check_counter:Getting stats from a 2024-06-13T07:11:31.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-13T07:11:31.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:30 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/710196003' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-13T07:11:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:32.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:31 smithi175 ceph-mon[36886]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-13T07:11:32.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:31 smithi007 ceph-mon[32077]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-13T07:11:32.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:31 smithi189 ceph-mon[36616]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-13T07:11:34.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:33 smithi007 ceph-mon[32077]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-13T07:11:34.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:33 smithi175 ceph-mon[36886]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-13T07:11:34.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:33 smithi189 ceph-mon[36616]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2037, 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2034, 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 998234, 2024-06-13T07:11:34.388 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 4209991, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 36, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 5, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.459285624, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.135312713, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.138287641, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.131882474, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 2032, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.648512651, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000319149 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 223, 2024-06-13T07:11:34.389 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000141723, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000635 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3296, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 3328, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1529765, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 1923746, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 10, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 5, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.567333134, 2024-06-13T07:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.246575429, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.342909239, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.037597406, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3328, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.086090776, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000326349 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 5149, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.003944797, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000766 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 518544 2024-06-13T07:11:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4537, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 5261, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2366425, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 2515832, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 12, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 7, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 1.680568231, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 1.145146620, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.474031724, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.100442103, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 5259, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.785687400, 2024-06-13T07:11:34.392 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000339548 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4466, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.002196434, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000491 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 649856, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 232752 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 1, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-MDSRank": { 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-06-13T07:11:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3744, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 2.583503861, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000690038 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-PurgeQueue": { 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 48, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.004649622, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000096867 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "request": 1009, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "reply": 1008, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "reply_latency": { 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1008, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 4.480947914, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.004445384 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_reply": 0, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "forward": 1, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_complete": 2, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_keys": 1, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_commit": 11, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_split": 0, 2024-06-13T07:11:34.395 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_merge": 0, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes": 314, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_top": 0, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_bottom": 0, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pin_tail": 314, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pinned": 314, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_expired": 475, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_with_caps": 57, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": 57, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "subtrees": 3, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse": 3122, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_hit": 2355, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_forward": 0, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_discover": 6, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:11:34.396 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_remote_ino": 0, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_lock": 191, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "load_cent": 7300, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "q": 0, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "exported": 11, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "exported_inodes": 470, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "imported": 12, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "imported_inodes": 506, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_dir_fetch": 0, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_backtrace_fetch": 3135, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_peer_discover": 1, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rfiles": 1019, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rbytes": 1528665134, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rsnaps": 0, 2024-06-13T07:11:34.397 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_fetch": 3135, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_set_tag": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_inodes": 13, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dirfrag_rstats": 13, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_file_inodes": 3122, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_revoke": 170, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_grant": 425, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:11:34.398 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps": 975, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps_dirty": 705, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_cap_release": 36, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "process_request_cap_release": 554 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache": { 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update_receipt": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_try_discover": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_send_discover": 6, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_handle_discover": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_delayed": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:11:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_created": 131, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_enqueued": 5, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_reintegrated": 126, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_migrated": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_processing": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_started": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_completed": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_exportdir": 11, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_flush": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:11:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragstats": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_inodestats": 0 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log": { 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evlrg": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evadd": 1854, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evex": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evtrm": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ev": 1854, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evexg": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "evexd": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "segadd": 3, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "segex": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "segtrm": 0, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "seg": 3, 2024-06-13T07:11:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: "segmjr": 2, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "segexg": 0, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "segexd": 0, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "expos": 4194304, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "wrpos": 7283033, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "rdpos": 0, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "jlat": { 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 768, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 5.018229127, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.006534152 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "replayed": 0 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mem": { 2024-06-13T07:11:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 78, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "ino+": 546, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "ino-": 468, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dir": 25, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dir+": 33, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dir-": 8, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dn": 314, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dn+": 799, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "dn-": 485, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cap": 57, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cap+": 486, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cap-": 429, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "rss": 58408, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "heap": 223612 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.403 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server": { 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_client_request": 1939, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_server_request": 0, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_request": 1009, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_session": 269, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_peer_request": 421, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "req_create_latency": { 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 32, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.088921449, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002778795 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getattr_latency": { 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 13, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.018144626, 2024-06-13T07:11:34.404 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001395740 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getfilelock_latency": { 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "req_link_latency": { 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 126, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.592481191, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.004702231 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookup_latency": { 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 524, 2024-06-13T07:11:34.405 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.077695469, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002056670 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookuphash_latency": { 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupino_latency": { 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.406 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupname_latency": { 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupparent_latency": { 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupsnap_latency": { 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.407 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lssnap_latency": { 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mkdir_latency": { 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.408 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mknod_latency": { 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mksnap_latency": { 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "req_open_latency": { 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 48, 2024-06-13T07:11:34.409 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.045423585, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000946324 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "req_readdir_latency": { 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 8, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.017718883, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.002214860 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rename_latency": { 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 126, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 2.133690626, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.016934052 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.410 INFO:teuthology.orchestra.run.smithi007.stdout: "req_renamesnap_latency": { 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "req_snapdiff_latency": { 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmdir_latency": { 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.411 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmsnap_latency": { 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmxattr_latency": { 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setattr_latency": { 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.412 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setdirlayout_latency": { 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setfilelock_latency": { 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setlayout_latency": { 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.413 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setxattr_latency": { 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "req_symlink_latency": { 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "req_unlink_latency": { 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 131, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.506872085, 2024-06-13T07:11:34.414 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.003869252 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getvxattr_latency": { 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessions": { 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "session_count": 1, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "session_add": 1, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "session_remove": 0, 2024-06-13T07:11:34.415 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_open": 1, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_stale": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "total_load": 200, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "average_load": 200, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "avg_session_uptime": 955, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "mdthresh_evicted": 0 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:34.416 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 296032, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 166, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 8600, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 164, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 490705, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 15022, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter": { 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_active": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_laggy": 0, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send": 4141, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send_bytes": 3163610, 2024-06-13T07:11:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "op_resend": 33, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_reply": 4141, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_latency": { 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4141, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 29.701536931, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.007172551 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_inflight": 0, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "oplen_avg": { 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4141, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 4282 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op": 4141, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_r": 3143, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_w": 998, 2024-06-13T07:11:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "op_rmw": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "op_pg": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_stat": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_create": 10, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_read": 3, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_write": 771, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writefull": 67, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writesame": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_append": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_zero": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_truncate": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_delete": 36, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_mapext": 0, 2024-06-13T07:11:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_sparse_read": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_clonerange": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_getxattr": 3138, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_setxattr": 20, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_rmxattr": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_call": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_watch": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_notify": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_other": 0, 2024-06-13T07:11:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_active": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_send": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_resend": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_ping": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_active": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_send": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_resend": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_active": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_send": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_resend": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_active": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_send": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_resend": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "command_active": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "command_send": 0, 2024-06-13T07:11:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "command_resend": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "map_epoch": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "map_full": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "map_inc": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_sessions": 12, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_open": 12, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_close": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_laggy": 0, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_wr": 184, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_rd": 8, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_del": 45 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "oft": { 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_objs": 1, 2024-06-13T07:11:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_updates": 0, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_removes": 0 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd-slow-ops": { 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_ops_count": 0 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "purge_queue": { 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops": 0, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops_high_water": 20, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing": 0, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_high_water": 3, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed_ops": 24, 2024-06-13T07:11:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed": 5, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_item_in_journal": 0 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 9866, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 3981367, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 9866, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 9866, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 3981367, 2024-06-13T07:11:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_bytes": { 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 4141, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 3164029, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 4141, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 3164029, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_ops": { 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 1024, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 4141, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 4141, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 4141, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4141, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 5, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 545, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 5, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 3, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 545, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle-0x55b911abc500": { 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 1854, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 3088729, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 1854, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 768, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 3088729, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:34.435 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:35.432 INFO:tasks.check_counter:Daemon mds.a mds.exported=11 2024-06-13T07:11:35.432 INFO:tasks.check_counter:Daemon mds.a mds.imported=12 2024-06-13T07:11:35.432 DEBUG:tasks.check_counter:Getting stats from d 2024-06-13T07:11:35.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-13T07:11:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:36.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:35 smithi007 ceph-mon[32077]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-06-13T07:11:36.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:35 smithi189 ceph-mon[36616]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-06-13T07:11:36.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:35 smithi175 ceph-mon[36886]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-06-13T07:11:38.059 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:37 smithi007 ceph-mon[32077]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-13T07:11:38.278 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 360, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 355, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 520528, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 218928, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.104036568, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.029218105, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.056876929, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.014564542, 2024-06-13T07:11:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 355, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.069309414, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000195237 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 520528, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 218928 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 270, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 203392, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.046828160, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.039112209, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003676932, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 268, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.048910803, 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000182502 2024-06-13T07:11:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 203392 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:11:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001092645, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000912730, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149735, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 358, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 467200, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 358, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 358, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 467200, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:38.290 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:38.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:37 smithi189 ceph-mon[36616]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-13T07:11:38.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:37 smithi175 ceph-mon[36886]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-13T07:11:39.201 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-13T07:11:39.201 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-13T07:11:39.201 DEBUG:tasks.check_counter:Getting stats from g 2024-06-13T07:11:39.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-13T07:11:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:40.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:39 smithi007 ceph-mon[32077]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:40.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:39 smithi189 ceph-mon[36616]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:40.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:39 smithi175 ceph-mon[36886]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 5781, 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 6960, 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2980766, 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 7255464, 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 40, 2024-06-13T07:11:42.132 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 5, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 2.353967278, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 1.400460422, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.563244783, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.347069255, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 6959, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 3.496346947, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000502420 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4947, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.002849897, 2024-06-13T07:11:42.133 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000576 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 645104, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 245696 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2173, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2443, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1269751, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 1602440, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 9, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 6, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.566694628, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.165720265, 2024-06-13T07:11:42.134 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.080592993, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.277343434, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 2441, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.712987166, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000292088 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 193, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000132766, 2024-06-13T07:11:42.135 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000687 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 609808 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4760, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4388, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2410272, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 1732606, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 11, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 5, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.749469870, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.258365022, 2024-06-13T07:11:42.136 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.452872596, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.083389212, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4387, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.732905503, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000395009 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 5362, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.005079205, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000947 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:11:42.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 1, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-MDSRank": { 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 4799, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 3.483729695, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000725928 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "finisher-PurgeQueue": { 2024-06-13T07:11:42.138 INFO:teuthology.orchestra.run.smithi007.stdout: "queue_len": 0, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "complete_latency": { 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 600, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.088202219, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000147003 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "request": 1108, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "reply": 1106, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "reply_latency": { 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1106, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 7.005540295, 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.006334123 2024-06-13T07:11:42.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_reply": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "forward": 2, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_complete": 2, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_fetch_keys": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_commit": 11, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_split": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_merge": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes": 198, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_top": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_bottom": 0, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pin_tail": 198, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_pinned": 198, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_expired": 371, 2024-06-13T07:11:42.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inodes_with_caps": 179, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": 179, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "subtrees": 8, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse": 2590, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_hit": 2256, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_forward": 1, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_discover": 10, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_remote_ino": 0, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "traverse_lock": 71, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "load_cent": 5853, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "q": 0, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "exported": 11, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "exported_inodes": 360, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "imported": 17, 2024-06-13T07:11:42.141 INFO:teuthology.orchestra.run.smithi007.stdout: "imported_inodes": 540, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_dir_fetch": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_backtrace_fetch": 5460, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "openino_peer_discover": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rfiles": 1019, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rbytes": 1528665134, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "root_rsnaps": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_fetch": 5460, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_set_tag": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_inodes": 12, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_dirfrag_rstats": 12, 2024-06-13T07:11:42.142 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_file_inodes": 5448, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_revoke": 36, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_grant": 655, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps": 948, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_caps_dirty": 728, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_cap_release": 175, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "process_request_cap_release": 829 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache": { 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_update_receipt": 0, 2024-06-13T07:11:42.143 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_try_discover": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_send_discover": 10, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "dir_handle_discover": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_delayed": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_created": 204, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_enqueued": 204, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_reintegrated": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "strays_migrated": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_processing": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_started": 0, 2024-06-13T07:11:42.144 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_completed": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_exportdir": 11, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_flush": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_fragstats": 0, 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ireq_inodestats": 0 2024-06-13T07:11:42.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log": { 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evlrg": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evadd": 2550, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evex": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evtrm": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "ev": 2550, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evexg": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "evexd": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "segadd": 4, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "segex": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "segtrm": 0, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "seg": 4, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "segmjr": 2, 2024-06-13T07:11:42.146 INFO:teuthology.orchestra.run.smithi007.stdout: "segexg": 0, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "segexd": 0, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "expos": 4194304, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "wrpos": 8041177, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "rdpos": 0, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "jlat": { 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 439, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 4.926634365, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.011222401 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "replayed": 0 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mem": { 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 200, 2024-06-13T07:11:42.147 INFO:teuthology.orchestra.run.smithi007.stdout: "ino+": 753, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "ino-": 553, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dir": 30, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dir+": 37, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dir-": 7, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dn": 201, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dn+": 777, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "dn-": 576, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cap": 179, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cap+": 881, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cap-": 702, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "rss": 61020, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "heap": 223612 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server": { 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_client_request": 1669, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "dispatch_server_request": 0, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_request": 1108, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_client_session": 21, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "handle_peer_request": 409, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "req_create_latency": { 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 200, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.642742837, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.003213714 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getattr_latency": { 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.001450141, 2024-06-13T07:11:42.149 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001450141 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getfilelock_latency": { 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "req_link_latency": { 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookup_latency": { 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 519, 2024-06-13T07:11:42.150 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 4.746841208, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.009146129 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookuphash_latency": { 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupino_latency": { 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 1, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000271252, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000271252 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupname_latency": { 2024-06-13T07:11:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupparent_latency": { 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lookupsnap_latency": { 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.152 INFO:teuthology.orchestra.run.smithi007.stdout: "req_lssnap_latency": { 2024-06-13T07:11:42.153 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:41 smithi007 ceph-mon[32077]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mkdir_latency": { 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mknod_latency": { 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.154 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "req_mksnap_latency": { 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "req_open_latency": { 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 176, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.253384813, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001439686 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "req_readdir_latency": { 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 5, 2024-06-13T07:11:42.155 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.008989508, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001797901 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rename_latency": { 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 198, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 1.331587505, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.006725189 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "req_renamesnap_latency": { 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "req_snapdiff_latency": { 2024-06-13T07:11:42.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmdir_latency": { 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmsnap_latency": { 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.157 INFO:teuthology.orchestra.run.smithi007.stdout: "req_rmxattr_latency": { 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setattr_latency": { 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setdirlayout_latency": { 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setfilelock_latency": { 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setlayout_latency": { 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "req_setxattr_latency": { 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "req_symlink_latency": { 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "req_unlink_latency": { 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 6, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.020273031, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.003378838 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:11:42.160 INFO:teuthology.orchestra.run.smithi007.stdout: "req_getvxattr_latency": { 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessions": { 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "session_count": 1, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "session_add": 2, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "session_remove": 1, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_open": 1, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "sessions_stale": 0, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "total_load": 102, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "average_load": 102, 2024-06-13T07:11:42.161 INFO:teuthology.orchestra.run.smithi007.stdout: "avg_session_uptime": 983, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "mdthresh_evicted": 0 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 120, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 120, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:42.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:42.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:42.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 316324, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 309, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 440, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 5, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 8600, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 164, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:42.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 722020, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 7486, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter": { 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "op_active": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "op_laggy": 0, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send": 6723, 2024-06-13T07:11:42.166 INFO:teuthology.orchestra.run.smithi007.stdout: "op_send_bytes": 3986709, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op_resend": 33, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op_reply": 6723, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op_latency": { 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 6723, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 54.527128087, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.008110535 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op_inflight": 0, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "oplen_avg": { 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 6723, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 6866 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op": 6723, 2024-06-13T07:11:42.167 INFO:teuthology.orchestra.run.smithi007.stdout: "op_r": 5542, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "op_w": 1181, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "op_rmw": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "op_pg": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_stat": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_create": 10, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_read": 74, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_write": 513, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writefull": 120, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_writesame": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_append": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_zero": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_truncate": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_delete": 420, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_mapext": 0, 2024-06-13T07:11:42.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_sparse_read": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_clonerange": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_getxattr": 5460, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_setxattr": 20, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_rmxattr": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_call": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_watch": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_notify": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdop_other": 0, 2024-06-13T07:11:42.169 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_active": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_send": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_resend": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "linger_ping": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_active": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_send": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolop_resend": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_active": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_send": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolstat_resend": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_active": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_send": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "statfs_resend": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "command_active": 0, 2024-06-13T07:11:42.170 INFO:teuthology.orchestra.run.smithi007.stdout: "command_send": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "command_resend": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "map_epoch": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "map_full": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "map_inc": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_sessions": 12, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_open": 12, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_session_close": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_laggy": 0, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_wr": 182, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_rd": 10, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_del": 57 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "oft": { 2024-06-13T07:11:42.171 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_objs": 1, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_updates": 0, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "omap_total_removes": 0 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "osd-slow-ops": { 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "slow_ops_count": 0 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "purge_queue": { 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops": 0, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_ops_high_water": 16, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing": 0, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executing_high_water": 8, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed_ops": 408, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_executed": 204, 2024-06-13T07:11:42.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pq_item_in_journal": 0 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 12710, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 5491364, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 12710, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 12710, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 5491364, 2024-06-13T07:11:42.173 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_bytes": { 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:42.174 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 6723, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 4008756, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 6723, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 4008756, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-objecter_ops": { 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 1024, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 0, 2024-06-13T07:11:42.175 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 0, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 6723, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 6723, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 6723, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 6723, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.176 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 204, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 22236, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 204, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 74, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 22236, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.177 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-write_buf_throttle-0x563da6330500": { 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 3758096384, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 2550, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 3846873, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 2550, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 439, 2024-06-13T07:11:42.178 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 3846873, 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:42.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:42.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:41 smithi189 ceph-mon[36616]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:42.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:41 smithi175 ceph-mon[36886]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:43.135 INFO:tasks.check_counter:Daemon mds.g mds.exported=11 2024-06-13T07:11:43.135 INFO:tasks.check_counter:Daemon mds.g mds.imported=17 2024-06-13T07:11:43.135 DEBUG:tasks.check_counter:Getting stats from j 2024-06-13T07:11:43.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-13T07:11:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:11:44.146 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:43 smithi007 ceph-mon[32077]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-13T07:11:44.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:43 smithi189 ceph-mon[36616]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-13T07:11:44.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:43 smithi175 ceph-mon[36886]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-13T07:11:46.052 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.003913358, 2024-06-13T07:11:46.053 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000856329, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.054 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 6, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 275, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3312, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 204368, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.053622520, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.044137710, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.005177288, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:46.055 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 274, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.070525964, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000257394 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 204368 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 359, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 356, 2024-06-13T07:11:46.056 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 518352, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 220400, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.127976608, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.069495391, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.041310876, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.014726213, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 355, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.076804602, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000216350 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 518352, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 220400 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:46.058 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:46.060 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 150766, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 31, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-06-13T07:11:46.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-06-13T07:11:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 361, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 467590, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 361, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 361, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 467590, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-06-13T07:11:46.063 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-06-13T07:11:46.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:45 smithi007 ceph-mon[32077]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-13T07:11:46.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:45 smithi189 ceph-mon[36616]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-13T07:11:46.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:45 smithi175 ceph-mon[36886]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-13T07:11:47.023 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-13T07:11:47.023 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-13T07:11:47.023 DEBUG:tasks.check_counter:Getting stats from b 2024-06-13T07:11:47.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-13T07:11:47.502 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:11:48.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:47 smithi007 ceph-mon[32077]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-06-13T07:11:48.213 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:47 smithi175 ceph-mon[36886]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-06-13T07:11:48.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:47 smithi189 ceph-mon[36616]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 9176, 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 11212, 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3262427, 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 17082557, 2024-06-13T07:11:50.036 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 262, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 7, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.736250096, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.226802294, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.385069290, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.153110335, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 11207, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 3.359433589, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000299762 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10356, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001775435, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000171 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 60368, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 30288 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 6167, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 6504, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3038121, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 6524669, 2024-06-13T07:11:50.038 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.427122749, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.141162692, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.198662233, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.100374543, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6507, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.058895121, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000316412 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 5024, 2024-06-13T07:11:50.039 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000989062, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000196 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 663808, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 240864 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 6422, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 6915, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3117265, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 4147497, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 11, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 6, 2024-06-13T07:11:50.040 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.416313670, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.140551077, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.179248900, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.101665108, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6913, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1.133458149, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000163960 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 4920, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001063108, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000216 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.041 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 556320 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 8402, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 2.987662547, 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000355589 2024-06-13T07:11:50.042 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 167, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.019419686, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000116285 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 2038, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 1840, 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-13T07:11:50.043 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1840, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 8.917306500, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004846362 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 198, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 7, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 2, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 771, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 3, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 846, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 2, 2024-06-13T07:11:50.044 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 822, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 22, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 843, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 875, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 814, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 814, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 9, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 2437, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 1711, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 197, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-13T07:11:50.045 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 49, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 3386, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 25, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 860, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 18, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 644, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 5062, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 1015, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 1480025516, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 5062, 2024-06-13T07:11:50.046 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 452, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 14, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 438, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 4610, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 5, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 116, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 1377, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:11:50.047 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 1429, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 1238, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 36, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 222 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 16, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:11:50.048 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 64, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 39, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 25, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 14, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 25, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-13T07:11:50.049 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 3, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 3604, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 3526, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 3526, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 78, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 24, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 74, 2024-06-13T07:11:50.050 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 73, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 73, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 1, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 1, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 1, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 15095167, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 15369031, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1822, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 10.653196816, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.005846979 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.051 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 834, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 1717, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 883, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 66, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 75, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 9, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 846, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 2054, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 1208, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 814, 2024-06-13T07:11:50.052 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 1863, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 1049, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 71244, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 231804 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 2188, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 830, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 2038, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 228, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 1660, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1029, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 7.485420286, 2024-06-13T07:11:50.053 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.007274460 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 36, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.038715780, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001075438 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-13T07:11:50.054 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 25, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.045313218, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001812528 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 116, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.116390379, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001003365 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.055 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000169835, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000169835 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.057 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 37, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.552169453, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.014923498 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.058 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 418, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.145984846, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000349246 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-13T07:11:50.059 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 65, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.150546009, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002316092 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 55, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.240811889, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.004378397 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.060 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.011014196, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.005507098 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.061 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001904179, 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000952089 2024-06-13T07:11:50.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.063 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 8, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.023826818, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002978352 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 46, 2024-06-13T07:11:50.064 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.105039612, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002283469 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-06-13T07:11:50.065 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 55, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 55, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 991, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:50.066 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:50.067 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:50.068 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 328182, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 932, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 440, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 5, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8600, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 164, 2024-06-13T07:11:50.069 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 2953256, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 30234, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 10914, 2024-06-13T07:11:50.070 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 14187696, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 32, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 10914, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10914, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 354.500757181, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.032481286 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 10914, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 15701 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 10914, 2024-06-13T07:11:50.071 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 5124, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 5790, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 698, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 2106, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 33, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 1841, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 247, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 3, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 606, 2024-06-13T07:11:50.072 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 5083, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 3196, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:11:50.073 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-13T07:11:50.074 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 98, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 9, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 12, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 12, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 1788, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 24, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 76 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-06-13T07:11:50.075 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 18, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 8, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 90, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 39, 2024-06-13T07:11:50.076 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 21761, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 7426779, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 21761, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 21761, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 7426779, 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:50.077 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:50.078 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 10914, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 14191654, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 10914, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 14191654, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:50.079 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 10914, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 10914, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 10914, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 10914, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 39, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 4235, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 39, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 19, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 4235, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.081 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55d84156a500": { 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 3604, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 11174727, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 3604, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:50.082 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1822, 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 11174727, 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:50.083 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-13T07:11:50.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:49 smithi007 ceph-mon[32077]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-06-13T07:11:50.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:49 smithi189 ceph-mon[36616]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-06-13T07:11:50.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:49 smithi175 ceph-mon[36886]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-06-13T07:11:51.079 INFO:tasks.check_counter:Daemon mds.b mds.exported=25 2024-06-13T07:11:51.079 INFO:tasks.check_counter:Daemon mds.b mds.imported=18 2024-06-13T07:11:51.079 DEBUG:tasks.check_counter:Getting stats from e 2024-06-13T07:11:51.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-13T07:11:51.327 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:11:52.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:51 smithi175 ceph-mon[36886]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:52.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:51 smithi007 ceph-mon[32077]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:52.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:51 smithi189 ceph-mon[36616]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-06-13T07:11:53.413 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-13T07:11:53.413 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:53.413 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 4, 2024-06-13T07:11:53.413 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 4, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 2992, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 448, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.001385947, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000032686, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000559786, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000681056, 2024-06-13T07:11:53.414 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000227018 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 2, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 273, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 320, 2024-06-13T07:11:53.415 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 203920, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.014536438, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.011113716, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000741238, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 271, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.023549296, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000086897 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:53.416 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 203920 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 359, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 356, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 518352, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 220416, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2024-06-13T07:11:53.417 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.048827096, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.018950303, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.018935637, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.008670246, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 355, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.022665531, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000063846 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:53.418 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 518352, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 220416 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:53.419 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:53.420 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:53.421 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 148721, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 30, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:53.422 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 928, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 361, 2024-06-13T07:11:53.423 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 467590, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 361, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 361, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 467590, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:53.424 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-13T07:11:54.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:53 smithi175 ceph-mon[36886]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:54.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:53 smithi007 ceph-mon[32077]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:54.262 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-06-13T07:11:54.262 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-06-13T07:11:54.262 DEBUG:tasks.check_counter:Getting stats from h 2024-06-13T07:11:54.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-13T07:11:54.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:53 smithi189 ceph-mon[36616]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:54.505 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:11:56.065 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:55 smithi175 ceph-mon[36886]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:56.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:55 smithi189 ceph-mon[36616]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 3608, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 3608, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 4106764, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1146527, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.261723791, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.074021646, 2024-06-13T07:11:56.344 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.066508630, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.103297449, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3607, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.134705172, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000037345 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-13T07:11:56.345 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 5, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 275, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 1036, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 692362, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.014696203, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.010740922, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.001568251, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000070966, 2024-06-13T07:11:56.346 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 273, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.021757548, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000079697 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 691664 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.347 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 385, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 360, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 628060, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 220286, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.048715649, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.018709946, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.018985848, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.008627368, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 359, 2024-06-13T07:11:56.348 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.029264763, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000081517 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 627792, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 219968 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-13T07:11:56.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6138, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.136863700, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000022297 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-06-13T07:11:56.350 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-06-13T07:11:56.351 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 309, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 1, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 308, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 307, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-06-13T07:11:56.352 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-06-13T07:11:56.353 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:11:56.354 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-06-13T07:11:56.355 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-06-13T07:11:56.356 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-06-13T07:11:56.357 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 1854, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 3, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 7283033, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 7282459, 2024-06-13T07:11:56.358 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 1854 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 67, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 537, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 470, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 12, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 24, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 12, 2024-06-13T07:11:56.359 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 309, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 790, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 481, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 53648, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215420 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-06-13T07:11:56.360 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.362 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.363 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.364 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-06-13T07:11:56.366 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-06-13T07:11:56.367 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.369 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-06-13T07:11:56.370 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.371 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-06-13T07:11:56.372 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 2845, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:56.373 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:56.374 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:56.375 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:56.376 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:56.377 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 184251, 2024-06-13T07:11:56.377 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 107, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8600, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 164, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:56.378 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 398358, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 11938, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 3607, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 3607, 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-06-13T07:11:56.379 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3607, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 5.898914641, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001635407 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3607, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 3608 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 3607, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 3607, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 2183, 2024-06-13T07:11:56.380 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 1422, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:11:56.381 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-06-13T07:11:56.382 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 0, 2024-06-13T07:11:56.383 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 4, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 5, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 1, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.384 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:56.385 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 3994, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 4353609, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 3994, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3994, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 4353609, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.386 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 3607, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 3088734, 2024-06-13T07:11:56.387 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3607, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 3088734, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:56.388 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 3607, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 3607, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3607, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 3607, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:11:56.389 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.390 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x564e7b6106e0": { 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-06-13T07:11:56.391 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:56.392 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-13T07:11:56.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:55 smithi007 ceph-mon[32077]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-06-13T07:11:57.212 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-13T07:11:57.212 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-13T07:11:57.213 DEBUG:tasks.check_counter:Getting stats from k 2024-06-13T07:11:57.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-13T07:11:57.450 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.b/config 2024-06-13T07:11:58.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:57 smithi175 ceph-mon[36886]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-13T07:11:58.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:57 smithi189 ceph-mon[36616]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-13T07:11:58.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:57 smithi007 ceph-mon[32077]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-13T07:11:59.183 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 0, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 2, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 0, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 0, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.001063489, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.000277402, 2024-06-13T07:11:59.184 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.185 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 7, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 279, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3472, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 283184, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.014597040, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.010594876, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.001519760, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 278, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.018791340, 2024-06-13T07:11:59.186 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000067594 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 283184 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 364, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 361, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 519712, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 223616, 2024-06-13T07:11:59.187 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 2, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 1, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.049669503, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.019286621, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.019610563, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.008637317, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 360, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.021634077, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000060094 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.188 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 519712, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 223616 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:11:59.189 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:11:59.190 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:11:59.191 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 148682, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 29, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-06-13T07:11:59.192 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 928, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 2, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-06-13T07:11:59.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 367, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 468267, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 367, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 367, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 468267, 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-06-13T07:11:59.194 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-06-13T07:11:59.195 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-06-13T07:11:59.195 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-06-13T07:11:59.195 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:59.195 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-06-13T07:11:59.195 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-06-13T07:11:59.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:11:59 smithi175 ceph-mon[36886]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T07:11:59.990 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-13T07:11:59.990 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-13T07:11:59.990 DEBUG:tasks.check_counter:Getting stats from c 2024-06-13T07:11:59.991 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-13T07:12:00.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:11:59 smithi189 ceph-mon[36616]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T07:12:00.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:11:59 smithi007 ceph-mon[32077]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-06-13T07:12:00.438 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:12:02.200 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:01 smithi189 ceph-mon[36616]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T07:12:02.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:01 smithi175 ceph-mon[36886]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T07:12:02.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:01 smithi007 ceph-mon[32077]: pgmap v988: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-06-13T07:12:02.929 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-06-13T07:12:02.929 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:12:02.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1439, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1441, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 4250021, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 457666, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.121828303, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.033293790, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.036252228, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.040169106, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1439, 2024-06-13T07:12:02.930 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.060053264, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000041732 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1481, 2024-06-13T07:12:02.931 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1456, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 897728, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 569463, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.150610078, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.041408710, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.045506159, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.053205230, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1455, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.069612905, 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000047843 2024-06-13T07:12:02.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 628336, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 221248 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1202, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1471, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 405344, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 908694, 2024-06-13T07:12:02.933 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.143786026, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.046867177, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.039381323, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.045017096, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1470, 2024-06-13T07:12:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.092139320, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000062679 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-06-13T07:12:02.938 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 528048 2024-06-13T07:12:02.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.939 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-06-13T07:12:02.939 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6318, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.189559293, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000030003 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-06-13T07:12:02.940 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 400, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 13, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 387, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-06-13T07:12:02.942 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 381, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 8, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-06-13T07:12:02.943 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-06-13T07:12:02.944 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:12:02.945 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-06-13T07:12:02.946 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:12:02.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 2568, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-06-13T07:12:02.948 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 4, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 8070492, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 8068768, 2024-06-13T07:12:02.949 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 2568 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 192, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 744, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 552, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 19, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 30, 2024-06-13T07:12:02.950 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 11, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 400, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 767, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 367, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 53060, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215420 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-06-13T07:12:02.951 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-06-13T07:12:02.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.956 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-06-13T07:12:02.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.957 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.957 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.958 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-06-13T07:12:02.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.962 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-06-13T07:12:02.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2837, 2024-06-13T07:12:02.966 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:12:02.967 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:12:02.968 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-06-13T07:12:02.969 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 197574, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 233, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:12:02.970 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8600, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 164, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 748346, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 5950, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-06-13T07:12:02.971 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 3731, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 3731, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3731, 2024-06-13T07:12:02.972 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 21.547599890, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005775288 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3731, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3732 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 3731, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 3731, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-06-13T07:12:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 2290, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1439, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-06-13T07:12:02.974 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:12:02.975 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-06-13T07:12:02.976 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 5, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 7, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 2, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-06-13T07:12:02.977 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-06-13T07:12:02.978 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:12:02.979 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 4118, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 5159664, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 4118, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 4118, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 5159664, 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:02.980 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:02.981 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 3731, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 3876193, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3731, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 3876193, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 3731, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 3731, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3731, 2024-06-13T07:12:02.983 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 3731, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:02.984 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x56394368e6e0": { 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-06-13T07:12:02.986 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:02.987 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-06-13T07:12:03.740 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-13T07:12:03.740 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-13T07:12:03.740 DEBUG:tasks.check_counter:Getting stats from f 2024-06-13T07:12:03.740 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-13T07:12:04.015 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:12:04.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:03 smithi189 ceph-mon[36616]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:04.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:03 smithi175 ceph-mon[36886]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:04.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:03 smithi007 ceph-mon[32077]: pgmap v989: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:06.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:05 smithi189 ceph-mon[36616]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-06-13T07:12:06.107 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.001069517, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000222117, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:06.108 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 361, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 358, 2024-06-13T07:12:06.109 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 518896, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 221696, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.049045954, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.019561876, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.017908776, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.008828973, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 357, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.025729718, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000072072 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.110 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 518896, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 221696 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 6, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 276, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 3312, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 204544, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:06.111 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.014531958, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.010813845, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.001208385, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 275, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.029444285, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000107070 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-13T07:12:06.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 204544 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:12:06.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:12:06.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 148699, 2024-06-13T07:12:06.115 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 30, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:12:06.116 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:06.117 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 363, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 467850, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 363, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 363, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 467850, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:06.118 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-06-13T07:12:06.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:05 smithi175 ceph-mon[36886]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-06-13T07:12:06.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:05 smithi007 ceph-mon[32077]: pgmap v990: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-06-13T07:12:06.826 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-06-13T07:12:06.826 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-06-13T07:12:06.826 DEBUG:tasks.check_counter:Getting stats from i 2024-06-13T07:12:06.826 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-13T07:12:07.071 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:12:08.252 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:07 smithi189 ceph-mon[36616]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:08.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:07 smithi175 ceph-mon[36886]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:08.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:07 smithi007 ceph-mon[32077]: pgmap v991: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:08.842 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1456, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1458, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 4663201, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 463018, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.144321579, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.043887101, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.040566636, 2024-06-13T07:12:08.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.046981896, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1456, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.072992806, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000050132 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:12:08.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1984, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1959, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 3923953, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 729320, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.181769316, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.053099192, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.054790582, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.062360435, 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:08.845 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1958, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.098951282, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000050536 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 628496, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 221408 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:12:08.846 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 857, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1126, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 4414265, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 887851, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.084945605, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.034892744, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.019539425, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.022151519, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1125, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.051364767, 2024-06-13T07:12:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000045657 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 617232 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.848 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6550, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.197382273, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000030134 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.849 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-06-13T07:12:08.850 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 922, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 527, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 395, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 177, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 9, 2024-06-13T07:12:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-06-13T07:12:08.852 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 1015, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 1480037717, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-06-13T07:12:08.853 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-06-13T07:12:08.854 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-06-13T07:12:08.855 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-06-13T07:12:08.856 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 3634, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 71, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 15095167, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 15476979, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 15469614, 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-06-13T07:12:08.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 3634 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 828, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 1702, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 874, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 61, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 68, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 7, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 922, 2024-06-13T07:12:08.858 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 1775, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 853, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 58864, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215420 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-06-13T07:12:08.859 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-06-13T07:12:08.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.867 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.868 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.870 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 3, 2024-06-13T07:12:08.871 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2843, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:12:08.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-06-13T07:12:08.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-06-13T07:12:08.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 220235, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 864, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-06-13T07:12:08.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8600, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 164, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 2635707, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 23166, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-06-13T07:12:08.876 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 3904, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 26, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 4, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 3904, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3904, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 48.210129172, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.012348906 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3904, 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3905 2024-06-13T07:12:08.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 3904, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 3904, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 2356, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1545, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-06-13T07:12:08.878 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 2, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-06-13T07:12:08.879 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-06-13T07:12:08.880 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 5, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 5, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-06-13T07:12:08.881 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-06-13T07:12:08.882 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 4293, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 12592151, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 4293, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 4293, 2024-06-13T07:12:08.883 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 12592151, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:08.884 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 3904, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 11282685, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3904, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 11282685, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:08.885 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 3904, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 3904, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3904, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 3904, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:08.887 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55e0059286e0": { 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:08.888 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:08.889 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-06-13T07:12:09.581 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-13T07:12:09.581 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-06-13T07:12:09.581 DEBUG:tasks.check_counter:Getting stats from l 2024-06-13T07:12:09.581 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-13T07:12:09.829 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.c/config 2024-06-13T07:12:10.266 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:09 smithi189 ceph-mon[36616]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2024-06-13T07:12:10.266 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:09 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:10.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:09 smithi175 ceph-mon[36886]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2024-06-13T07:12:10.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:09 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:09 smithi007 ceph-mon[32077]: pgmap v992: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2024-06-13T07:12:10.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:09 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 367, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 364, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 520528, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 225536, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-06-13T07:12:11.626 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.049360974, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.019183768, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.018770126, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.008568489, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 363, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.019935919, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000054919 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:11.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 520528, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 225536 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 6, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 282, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 3312, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 205600, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.013906682, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.010434224, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.001120925, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:12:11.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 281, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.014718969, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000052380 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 205600 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-06-13T07:12:11.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.000277679, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000210865, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-06-13T07:12:11.631 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-06-13T07:12:11.632 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-06-13T07:12:11.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 148704, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 29, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-06-13T07:12:11.634 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-06-13T07:12:11.635 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 369, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 468630, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 369, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 369, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 468630, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-06-13T07:12:11.636 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-06-13T07:12:11.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-06-13T07:12:11.637 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:11.637 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-06-13T07:12:11.637 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-06-13T07:12:12.298 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:12.298 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.298 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.298 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:12.299 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:12.299 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:11 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.327 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-13T07:12:12.327 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-13T07:12:12.327 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-13T07:12:12.342 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-13T07:12:12.342 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T07:12:12.342 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:12.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:11 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.410 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-06-13T07:12:12.410 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-13T07:12:12.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: pgmap v993: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:12.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-13T07:12:12.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-13T07:12:12.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:11 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' 2024-06-13T07:12:12.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd blocklist ls 2024-06-13T07:12:12.940 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:14.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:13 smithi189 ceph-mon[36616]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-06-13T07:12:14.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:13 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:13 smithi175 ceph-mon[36886]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-06-13T07:12:14.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:13 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:14.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:13 smithi007 ceph-mon[32077]: pgmap v994: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-06-13T07:12:14.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:13 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 -- 172.21.15.7:0/1358719205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de8108500 msgr2=0x7f7de81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 --2- 172.21.15.7:0/1358719205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de8108500 0x7f7de81088e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009990 tx=0x7f7dd8009ca0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 -- 172.21.15.7:0/1358719205 shutdown_connections 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 --2- 172.21.15.7:0/1358719205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de806ab30 0x7f7de810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 --2- 172.21.15.7:0/1358719205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de8108500 0x7f7de81088e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 --2- 172.21.15.7:0/1358719205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 -- 172.21.15.7:0/1358719205 >> 172.21.15.7:0/1358719205 conn(0x7f7de8078340 msgr2=0x7f7de8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:15.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 -- 172.21.15.7:0/1358719205 shutdown_connections 2024-06-13T07:12:15.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.427+0000 7f7def200700 1 -- 172.21.15.7:0/1358719205 wait complete. 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.428+0000 7f7def200700 1 Processor -- start 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.428+0000 7f7def200700 1 -- start start 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de806ab30 0x7f7de81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de8108500 0x7f7de8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81127d0 con 0x7f7de806ab30 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8112650 con 0x7f7de8108500 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7def200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8112950 con 0x7f7de806a170 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7de7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7de7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:35264/0 (socket says 172.21.15.7:35264) 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7ddf400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de806ab30 0x7f7de81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7de7e00700 1 -- 172.21.15.7:0/189257512 learned_addr learned my addr 172.21.15.7:0/189257512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.429+0000 7f7ded200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de8108500 0x7f7de8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 -- 172.21.15.7:0/189257512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de8108500 msgr2=0x7f7de8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de8108500 0x7f7de8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 -- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de806ab30 msgr2=0x7f7de81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de806ab30 0x7f7de81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 -- 172.21.15.7:0/189257512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de81a7230 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de7e00700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005910 tx=0x7f7dd800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8027070 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd8004650 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800f460 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.430+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81a74c0 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.431+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81a7910 con 0x7f7de806a170 2024-06-13T07:12:15.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.432+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de810a9e0 con 0x7f7de806a170 2024-06-13T07:12:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.434+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7dd802a070 con 0x7f7de806a170 2024-06-13T07:12:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.435+0000 7f7de5600700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 0x7f7dcc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.435+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f7dd80d0250 con 0x7f7de806a170 2024-06-13T07:12:15.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.435+0000 7f7ddf400700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 0x7f7dcc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.436+0000 7f7ddf400700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 0x7f7dcc0a50e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7de8078910 tx=0x7f7dd000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.439+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7dd805e850 con 0x7f7de806a170 2024-06-13T07:12:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.689+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7de8102990 con 0x7f7de806a170 2024-06-13T07:12:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.690+0000 7f7de5600700 1 -- 172.21.15.7:0/189257512 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+581 (secure 0 0 0) 0x7f7dd8013070 con 0x7f7de806a170 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3381938819 2024-06-14T06:41:26.246814+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1735370722 2024-06-14T06:41:59.455768+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1576245213 2024-06-14T06:41:59.455768+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/239861889 2024-06-14T06:41:26.246814+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3124714037 2024-06-14T06:41:59.455768+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1109324950 2024-06-14T06:41:26.246814+0000 2024-06-13T07:12:15.690 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4091601586 2024-06-14T06:41:26.246814+0000 2024-06-13T07:12:15.691 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1476683041 2024-06-14T06:41:59.455768+0000 2024-06-13T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.694+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 msgr2=0x7f7dcc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.694+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 0x7f7dcc0a50e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7de8078910 tx=0x7f7dd000b040 comp rx=0 tx=0).stop 2024-06-13T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.694+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 msgr2=0x7f7de8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.694+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005910 tx=0x7f7dd800bab0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 shutdown_connections 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f7dcc0a2c20 0x7f7dcc0a50e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7de8108500 0x7f7de8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de806a170 0x7f7de8101460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 --2- 172.21.15.7:0/189257512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7de806ab30 0x7f7de81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 >> 172.21.15.7:0/189257512 conn(0x7f7de8078340 msgr2=0x7f7de80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 shutdown_connections 2024-06-13T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:15.695+0000 7f7def200700 1 -- 172.21.15.7:0/189257512 wait complete. 2024-06-13T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-06-13T07:12:15.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:15 smithi007 ceph-mon[32077]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:15.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:15 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/189257512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:12:16.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:15 smithi189 ceph-mon[36616]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:16.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:15 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/189257512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:12:16.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:15 smithi175 ceph-mon[36886]: pgmap v995: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:16.368 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:15 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/189257512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-13T07:12:16.583 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-13T07:12:16.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:12:16.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /tmp/cephfs 2024-06-13T07:12:16.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T07:12:16.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:12:16.741 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-06-13T07:12:16.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:12:16.835 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-06-13T07:12:16.900 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-06-13T07:12:16.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:12:16.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-06-13T07:12:16.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-13T07:12:16.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-06-13T07:12:17.071 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-13T07:12:17.085 INFO:tasks.cephadm:Teardown begin 2024-06-13T07:12:17.085 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-13T07:12:17.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:12:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:18.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:17 smithi189 ceph-mon[36616]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:18.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:17 smithi175 ceph-mon[36886]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:18.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:17 smithi007 ceph-mon[32077]: pgmap v996: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-06-13T07:12:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:19 smithi007 ceph-mon[32077]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:19.163 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:19 smithi007 ceph-mon[32077]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:19.336 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:19 smithi189 ceph-mon[36616]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:19.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:19 smithi189 ceph-mon[36616]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:19 smithi175 ceph-mon[36886]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-13T07:12:19.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:19 smithi175 ceph-mon[36886]: pgmap v997: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:19.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 -- 172.21.15.7:0/2047334357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101d50 msgr2=0x7ff43810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 --2- 172.21.15.7:0/2047334357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101d50 0x7ff43810e240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b0a0 tx=0x7ff43000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 -- 172.21.15.7:0/2047334357 shutdown_connections 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 --2- 172.21.15.7:0/2047334357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101d50 0x7ff43810e240 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 --2- 172.21.15.7:0/2047334357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff4381077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 --2- 172.21.15.7:0/2047334357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101390 0x7ff438101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.773+0000 7ff43d400700 1 -- 172.21.15.7:0/2047334357 >> 172.21.15.7:0/2047334357 conn(0x7ff438078340 msgr2=0x7ff438078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.774+0000 7ff43d400700 1 -- 172.21.15.7:0/2047334357 shutdown_connections 2024-06-13T07:12:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.774+0000 7ff43d400700 1 -- 172.21.15.7:0/2047334357 wait complete. 2024-06-13T07:12:19.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 Processor -- start 2024-06-13T07:12:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 -- start start 2024-06-13T07:12:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101390 0x7ff43819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 0x7ff43819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff43819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381127c0 con 0x7ff438101d50 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff438112640 con 0x7ff438107400 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.775+0000 7ff43d400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff438112940 con 0x7ff438101390 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff435e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 0x7ff43819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff43819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff435e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 0x7ff43819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38042/0 (socket says 172.21.15.7:38042) 2024-06-13T07:12:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff435e00700 1 -- 172.21.15.7:0/322740256 learned_addr learned my addr 172.21.15.7:0/322740256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff436800700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101390 0x7ff43819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 -- 172.21.15.7:0/322740256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101390 msgr2=0x7ff43819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101390 0x7ff43819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 -- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 msgr2=0x7ff43819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 0x7ff43819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.776+0000 7ff437200700 1 -- 172.21.15.7:0/322740256 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43819fdf0 con 0x7ff438107400 2024-06-13T07:12:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff437200700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff43819f6b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b070 tx=0x7ff430004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430021070 con 0x7ff438107400 2024-06-13T07:12:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4381a0080 con 0x7ff438107400 2024-06-13T07:12:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff430007b80 con 0x7ff438107400 2024-06-13T07:12:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430005040 con 0x7ff438107400 2024-06-13T07:12:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.777+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381a7960 con 0x7ff438107400 2024-06-13T07:12:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.778+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff438069180 con 0x7ff438107400 2024-06-13T07:12:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.780+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff430024070 con 0x7ff438107400 2024-06-13T07:12:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.782+0000 7ff42f400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 0x7ff41c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.782+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7ff4300d0190 con 0x7ff438107400 2024-06-13T07:12:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.782+0000 7ff436800700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 0x7ff41c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.783+0000 7ff436800700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 0x7ff41c0a5160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff428009be0 tx=0x7ff428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:19.784+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff430007d30 con 0x7ff438107400 2024-06-13T07:12:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.068+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4381a7b30 con 0x7ff438107400 2024-06-13T07:12:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.069+0000 7ff42f400700 1 -- 172.21.15.7:0/322740256 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11877 (secure 0 0 0) 0x7ff43009cde0 con 0x7ff438107400 2024-06-13T07:12:20.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:12:20.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:12:17.657012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24521,24527]},"id":1}]} 2024-06-13T07:12:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.073+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 msgr2=0x7ff41c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.073+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 0x7ff41c0a5160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff428009be0 tx=0x7ff428009260 comp rx=0 tx=0).stop 2024-06-13T07:12:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.073+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 msgr2=0x7ff43819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.073+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff43819f6b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b070 tx=0x7ff430004c80 comp rx=0 tx=0).stop 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 shutdown_connections 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ff41c0a2ca0 0x7ff41c0a5160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff438107400 0x7ff43819f6b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff438101390 0x7ff43819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 --2- 172.21.15.7:0/322740256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff438101d50 0x7ff43819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 >> 172.21.15.7:0/322740256 conn(0x7ff438078340 msgr2=0x7ff43810c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 shutdown_connections 2024-06-13T07:12:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:20.074+0000 7ff43d400700 1 -- 172.21.15.7:0/322740256 wait complete. 2024-06-13T07:12:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 48 2024-06-13T07:12:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:20 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/322740256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:20.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:20 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/322740256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:20.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:20 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/322740256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:21.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs dump --format=json 2024-06-13T07:12:21.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:21 smithi007 ceph-mon[32077]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-06-13T07:12:21.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:21 smithi189 ceph-mon[36616]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-06-13T07:12:21.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:21 smithi175 ceph-mon[36886]: pgmap v998: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-06-13T07:12:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:23.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:23 smithi175 ceph-mon[36886]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-06-13T07:12:23.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:23 smithi007 ceph-mon[32077]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-06-13T07:12:23.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:23 smithi189 ceph-mon[36616]: pgmap v999: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 -- 172.21.15.7:0/1440478236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 msgr2=0x7f8af010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 --2- 172.21.15.7:0/1440478236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af010eea0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00098f0 tx=0x7f8ae0009c00 comp rx=0 tx=0).stop 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 -- 172.21.15.7:0/1440478236 shutdown_connections 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 --2- 172.21.15.7:0/1440478236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0109480 0x7f8af0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 --2- 172.21.15.7:0/1440478236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af010eea0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 --2- 172.21.15.7:0/1440478236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0108ac0 0x7f8af0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 -- 172.21.15.7:0/1440478236 >> 172.21.15.7:0/1440478236 conn(0x7f8af0075c60 msgr2=0x7f8af01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 -- 172.21.15.7:0/1440478236 shutdown_connections 2024-06-13T07:12:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.154+0000 7f8af7000700 1 -- 172.21.15.7:0/1440478236 wait complete. 2024-06-13T07:12:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.155+0000 7f8af7000700 1 Processor -- start 2024-06-13T07:12:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 -- start start 2024-06-13T07:12:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0109480 0x7f8af019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af01a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118e60 con 0x7f8af0108ac0 2024-06-13T07:12:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118ce0 con 0x7f8af010eac0 2024-06-13T07:12:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.156+0000 7f8af7000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0118fe0 con 0x7f8af0109480 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47278/0 (socket says 172.21.15.7:47278) 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 -- 172.21.15.7:0/2397670800 learned_addr learned my addr 172.21.15.7:0/2397670800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8ae7e00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0109480 0x7f8af019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8af5000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af01a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 -- 172.21.15.7:0/2397670800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0109480 msgr2=0x7f8af019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0109480 0x7f8af019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 -- 172.21.15.7:0/2397670800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 msgr2=0x7f8af01a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af01a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.157+0000 7f8aefe00700 1 -- 172.21.15.7:0/2397670800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af01a43c0 con 0x7f8af0108ac0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8aefe00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0005950 tx=0x7f8ae000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0005140 con 0x7f8af0108ac0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae000b560 con 0x7f8af0108ac0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af01a4650 con 0x7f8af0108ac0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0003ae0 con 0x7f8af0108ac0 2024-06-13T07:12:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.158+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af01abf30 con 0x7f8af0108ac0 2024-06-13T07:12:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.160+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af0069180 con 0x7f8af0108ac0 2024-06-13T07:12:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.162+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8ae000ba80 con 0x7f8af0108ac0 2024-06-13T07:12:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.163+0000 7f8aed600700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 0x7f8ad40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.163+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8ae00d0260 con 0x7f8af0108ac0 2024-06-13T07:12:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.163+0000 7f8ae7e00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 0x7f8ad40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.164+0000 7f8ae7e00700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 0x7f8ad40a5160 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8af0077c40 tx=0x7f8ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:24.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.168+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ae0013030 con 0x7f8af0108ac0 2024-06-13T07:12:24.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.455+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8af0111070 con 0x7f8af0108ac0 2024-06-13T07:12:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.456+0000 7f8aed600700 1 -- 172.21.15.7:0/2397670800 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11877 (secure 0 0 0) 0x7f8ae009ceb0 con 0x7f8af0108ac0 2024-06-13T07:12:24.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:12:24.457 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14361,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3083618837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3083618837},{"type":"v1","addr":"172.21.15.7:6835","nonce":3083618837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1184948673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1184948673},{"type":"v1","addr":"172.21.15.189:6833","nonce":1184948673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/587777641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":587777641},{"type":"v1","addr":"172.21.15.175:6833","nonce":587777641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/3062011119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":3062011119},{"type":"v1","addr":"172.21.15.7:6837","nonce":3062011119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1289787148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1289787148},{"type":"v1","addr":"172.21.15.175:6835","nonce":1289787148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/4153094559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4153094559},{"type":"v1","addr":"172.21.15.189:6835","nonce":4153094559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-13T06:50:19.413292+0000","modified":"2024-06-13T07:12:17.657012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24539,"mds_1":24527,"mds_2":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/4202454996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4202454996},{"type":"v1","addr":"172.21.15.189:6837","nonce":4202454996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6837/2460722721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2460722721},{"type":"v1","addr":"172.21.15.175:6837","nonce":2460722721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6839/3291148051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3291148051},{"type":"v1","addr":"172.21.15.189:6839","nonce":3291148051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"g","rank":2,"incarnation":21,"state":"up:active","state_seq":29,"addr":"172.21.15.7:6839/827561506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":827561506},{"type":"v1","addr":"172.21.15.7:6839","nonce":827561506}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":28,"addr":"172.21.15.7:6841/1761725822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1761725822},{"type":"v1","addr":"172.21.15.7:6841","nonce":1761725822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6839/4167274347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4167274347},{"type":"v1","addr":"172.21.15.175:6839","nonce":4167274347}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24521,24527]},"id":1}]} 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 msgr2=0x7f8ad40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 0x7f8ad40a5160 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8af0077c40 tx=0x7f8ad8009260 comp rx=0 tx=0).stop 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 msgr2=0x7f8af019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0005950 tx=0x7f8ae000fb20 comp rx=0 tx=0).stop 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 shutdown_connections 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8ad40a2ca0 0x7f8ad40a5160 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8af010eac0 0x7f8af01a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af0108ac0 0x7f8af019f3b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 --2- 172.21.15.7:0/2397670800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8af0109480 0x7f8af019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.459+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 >> 172.21.15.7:0/2397670800 conn(0x7f8af0075c60 msgr2=0x7f8af0076320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.460+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 shutdown_connections 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:24.460+0000 7f8af7000700 1 -- 172.21.15.7:0/2397670800 wait complete. 2024-06-13T07:12:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 48 2024-06-13T07:12:25.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd dump --format=json 2024-06-13T07:12:25.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:25 smithi175 ceph-mon[36886]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:25.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:25 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2397670800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:25.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:25 smithi007 ceph-mon[32077]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:25.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:25 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2397670800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:25 smithi189 ceph-mon[36616]: pgmap v1000: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:25.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:25 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2397670800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-13T07:12:27.527 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:27 smithi007 ceph-mon[32077]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:27.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:27 smithi175 ceph-mon[36886]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:27.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:27 smithi189 ceph-mon[36616]: pgmap v1001: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-06-13T07:12:28.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1332691955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb4108aa0 msgr2=0x7f8cb4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:28.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1332691955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb4108aa0 0x7f8cb4108f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009990 tx=0x7f8ca0009ca0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1332691955 shutdown_connections 2024-06-13T07:12:28.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1332691955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb4115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1332691955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb410eaa0 0x7f8cb410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1332691955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb4108aa0 0x7f8cb4108f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.079+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1332691955 >> 172.21.15.7:0/1332691955 conn(0x7f8cb4075c60 msgr2=0x7f8cb41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.080+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1332691955 shutdown_connections 2024-06-13T07:12:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.080+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1332691955 wait complete. 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 Processor -- start 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 -- start start 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb4108aa0 0x7f8cb419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb410eaa0 0x7f8cb41a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4118df0 con 0x7f8cb410eaa0 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4118c70 con 0x7f8cb4109460 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.081+0000 7f8cb9c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb4118f70 con 0x7f8cb4108aa0 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:40002/0 (socket says 172.21.15.7:40002) 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb4108aa0 0x7f8cb419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 -- 172.21.15.7:0/1000790113 learned_addr learned my addr 172.21.15.7:0/1000790113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb3400700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb410eaa0 0x7f8cb41a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 -- 172.21.15.7:0/1000790113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb4108aa0 msgr2=0x7f8cb419f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb4108aa0 0x7f8cb419f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 -- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb410eaa0 msgr2=0x7f8cb41a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb410eaa0 0x7f8cb41a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 -- 172.21.15.7:0/1000790113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb41a4250 con 0x7f8cb4109460 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.082+0000 7f8cb2000700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000bae0 tx=0x7f8ca000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.083+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0026070 con 0x7f8cb4109460 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.083+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0003d70 con 0x7f8cb4109460 2024-06-13T07:12:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.083+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000fe50 con 0x7f8cb4109460 2024-06-13T07:12:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.083+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb41a44e0 con 0x7f8cb4109460 2024-06-13T07:12:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.083+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb41abdc0 con 0x7f8cb4109460 2024-06-13T07:12:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.084+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb4069180 con 0x7f8cb4109460 2024-06-13T07:12:28.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.086+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8ca0029070 con 0x7f8cb4109460 2024-06-13T07:12:28.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.087+0000 7f8cab400700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 0x7f8c980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:28.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.087+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7f8ca00cfff0 con 0x7f8cb4109460 2024-06-13T07:12:28.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.088+0000 7f8cb2a00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 0x7f8c980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:28.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.088+0000 7f8cb2a00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 0x7f8c980a5300 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009be0 tx=0x7f8ca4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:28.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.092+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ca005d850 con 0x7f8cb4109460 2024-06-13T07:12:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.323+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cb402dca0 con 0x7f8cb4109460 2024-06-13T07:12:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.325+0000 7f8cab400700 1 -- 172.21.15.7:0/1000790113 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24494 (secure 0 0 0) 0x7f8ca0013070 con 0x7f8cb4109460 2024-06-13T07:12:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-06-13T07:12:28.330 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"b9295806-294f-11ef-bc9c-c7b262605968","created":"2024-06-13T06:40:52.731225+0000","modified":"2024-06-13T06:51:00.875426+0000","last_up_change":"2024-06-13T06:49:00.585817+0000","last_in_change":"2024-06-13T06:48:40.067260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-13T06:44:44.175120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-13T06:50:10.636472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-13T06:50:15.402440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-13T06:50:28.238859+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e144dfd9-116a-4342-8fe5-e40f94ed6071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6803","nonce":625935948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6805","nonce":625935948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6809","nonce":625935948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":625935948},{"type":"v1","addr":"172.21.15.7:6807","nonce":625935948}]},"public_addr":"172.21.15.7:6803/625935948","cluster_addr":"172.21.15.7:6805/625935948","heartbeat_back_addr":"172.21.15.7:6809/625935948","heartbeat_front_addr":"172.21.15.7:6807/625935948","state":["exists","up"]},{"osd":1,"uuid":"c21f9125-8329-4858-a611-5aeb5e3d6da0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6801","nonce":2094167604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6803","nonce":2094167604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6807","nonce":2094167604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":2094167604},{"type":"v1","addr":"172.21.15.175:6805","nonce":2094167604}]},"public_addr":"172.21.15.175:6801/2094167604","cluster_addr":"172.21.15.175:6803/2094167604","heartbeat_back_addr":"172.21.15.175:6807/2094167604","heartbeat_front_addr":"172.21.15.175:6805/2094167604","state":["exists","up"]},{"osd":2,"uuid":"073fad02-1136-4f2e-bd65-8a2a7ceb5a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6801","nonce":2765218208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6803","nonce":2765218208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6807","nonce":2765218208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2765218208},{"type":"v1","addr":"172.21.15.189:6805","nonce":2765218208}]},"public_addr":"172.21.15.189:6801/2765218208","cluster_addr":"172.21.15.189:6803/2765218208","heartbeat_back_addr":"172.21.15.189:6807/2765218208","heartbeat_front_addr":"172.21.15.189:6805/2765218208","state":["exists","up"]},{"osd":3,"uuid":"60aad01f-267a-43ad-b0d3-acb77db40434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6811","nonce":2396372699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6813","nonce":2396372699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6817","nonce":2396372699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2396372699},{"type":"v1","addr":"172.21.15.7:6815","nonce":2396372699}]},"public_addr":"172.21.15.7:6811/2396372699","cluster_addr":"172.21.15.7:6813/2396372699","heartbeat_back_addr":"172.21.15.7:6817/2396372699","heartbeat_front_addr":"172.21.15.7:6815/2396372699","state":["exists","up"]},{"osd":4,"uuid":"2d14e0c7-6370-4889-8f20-87458638b022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6809","nonce":576672935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6811","nonce":576672935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6815","nonce":576672935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":576672935},{"type":"v1","addr":"172.21.15.175:6813","nonce":576672935}]},"public_addr":"172.21.15.175:6809/576672935","cluster_addr":"172.21.15.175:6811/576672935","heartbeat_back_addr":"172.21.15.175:6815/576672935","heartbeat_front_addr":"172.21.15.175:6813/576672935","state":["exists","up"]},{"osd":5,"uuid":"6f01cff3-7564-4832-a4f8-0b2920c1b201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6809","nonce":1526104805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6811","nonce":1526104805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6815","nonce":1526104805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1526104805},{"type":"v1","addr":"172.21.15.189:6813","nonce":1526104805}]},"public_addr":"172.21.15.189:6809/1526104805","cluster_addr":"172.21.15.189:6811/1526104805","heartbeat_back_addr":"172.21.15.189:6815/1526104805","heartbeat_front_addr":"172.21.15.189:6813/1526104805","state":["exists","up"]},{"osd":6,"uuid":"785c5d21-30ac-46c0-9f6c-543778283cb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6819","nonce":3095237375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6821","nonce":3095237375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6825","nonce":3095237375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3095237375},{"type":"v1","addr":"172.21.15.7:6823","nonce":3095237375}]},"public_addr":"172.21.15.7:6819/3095237375","cluster_addr":"172.21.15.7:6821/3095237375","heartbeat_back_addr":"172.21.15.7:6825/3095237375","heartbeat_front_addr":"172.21.15.7:6823/3095237375","state":["exists","up"]},{"osd":7,"uuid":"41e3e868-f9cf-461e-84ce-4321ccdd9fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6817","nonce":935680699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6819","nonce":935680699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6823","nonce":935680699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":935680699},{"type":"v1","addr":"172.21.15.175:6821","nonce":935680699}]},"public_addr":"172.21.15.175:6817/935680699","cluster_addr":"172.21.15.175:6819/935680699","heartbeat_back_addr":"172.21.15.175:6823/935680699","heartbeat_front_addr":"172.21.15.175:6821/935680699","state":["exists","up"]},{"osd":8,"uuid":"f9407bcc-7a87-425c-a20b-dd361583d662","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6817","nonce":1210852331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6819","nonce":1210852331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6823","nonce":1210852331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1210852331},{"type":"v1","addr":"172.21.15.189:6821","nonce":1210852331}]},"public_addr":"172.21.15.189:6817/1210852331","cluster_addr":"172.21.15.189:6819/1210852331","heartbeat_back_addr":"172.21.15.189:6823/1210852331","heartbeat_front_addr":"172.21.15.189:6821/1210852331","state":["exists","up"]},{"osd":9,"uuid":"b78e7f03-fad5-401e-a013-def76183c002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6827","nonce":1288548158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6829","nonce":1288548158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6833","nonce":1288548158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1288548158},{"type":"v1","addr":"172.21.15.7:6831","nonce":1288548158}]},"public_addr":"172.21.15.7:6827/1288548158","cluster_addr":"172.21.15.7:6829/1288548158","heartbeat_back_addr":"172.21.15.7:6833/1288548158","heartbeat_front_addr":"172.21.15.7:6831/1288548158","state":["exists","up"]},{"osd":10,"uuid":"f61b6ecf-df12-4a81-93f4-b17aac68e6f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6825","nonce":2694135540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6827","nonce":2694135540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6831","nonce":2694135540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2694135540},{"type":"v1","addr":"172.21.15.175:6829","nonce":2694135540}]},"public_addr":"172.21.15.175:6825/2694135540","cluster_addr":"172.21.15.175:6827/2694135540","heartbeat_back_addr":"172.21.15.175:6831/2694135540","heartbeat_front_addr":"172.21.15.175:6829/2694135540","state":["exists","up"]},{"osd":11,"uuid":"2053f9b6-4b88-48c3-8665-7501ea3b4b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6825","nonce":1625559024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6827","nonce":1625559024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6831","nonce":1625559024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":1625559024},{"type":"v1","addr":"172.21.15.189:6829","nonce":1625559024}]},"public_addr":"172.21.15.189:6825/1625559024","cluster_addr":"172.21.15.189:6827/1625559024","heartbeat_back_addr":"172.21.15.189:6831/1625559024","heartbeat_front_addr":"172.21.15.189:6829/1625559024","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:43:51.750561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:16.675884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:44:40.262843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:13.392043+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:45:39.035286+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:03.936367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:46:38.868764+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:04.693012+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:47:30.435344+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:05.537767+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:32.543906+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-13T06:48:58.354053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/3381938819":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1735370722":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1576245213":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/239861889":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6801/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/3124714037":"2024-06-14T06:41:59.455768+0000","172.21.15.7:0/1109324950":"2024-06-14T06:41:26.246814+0000","172.21.15.7:6800/4091601586":"2024-06-14T06:41:26.246814+0000","172.21.15.7:0/1476683041":"2024-06-14T06:41:59.455768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 msgr2=0x7f8c980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 0x7f8c980a5300 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009be0 tx=0x7f8ca4009260 comp rx=0 tx=0).stop 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 msgr2=0x7f8cb419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000bae0 tx=0x7f8ca000bb10 comp rx=0 tx=0).stop 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 shutdown_connections 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cb410eaa0 0x7f8cb41a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cb4108aa0 0x7f8cb419f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8c980a2e40 0x7f8c980a5300 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 --2- 172.21.15.7:0/1000790113 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cb4109460 0x7f8cb419f780 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 >> 172.21.15.7:0/1000790113 conn(0x7f8cb4075c60 msgr2=0x7f8cb4076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.328+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 shutdown_connections 2024-06-13T07:12:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:28.329+0000 7f8cb9c00700 1 -- 172.21.15.7:0/1000790113 wait complete. 2024-06-13T07:12:28.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:28 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1000790113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:12:28.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:28 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1000790113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:12:28.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:28 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1000790113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-13T07:12:29.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs fail cephfs 2024-06-13T07:12:29.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:29 smithi175 ceph-mon[36886]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:29.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:29 smithi007 ceph-mon[32077]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:29.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:29 smithi189 ceph-mon[36616]: pgmap v1002: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-06-13T07:12:31.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:31 smithi007 ceph-mon[32077]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T07:12:31.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:31 smithi189 ceph-mon[36616]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T07:12:31.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:31 smithi175 ceph-mon[36886]: pgmap v1003: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-13T07:12:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.062+0000 7fb231a00700 1 -- 172.21.15.7:0/2849416571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 msgr2=0x7fb22c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.062+0000 7fb231a00700 1 --2- 172.21.15.7:0/2849416571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c108f40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb218009990 tx=0x7fb218009ca0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 -- 172.21.15.7:0/2849416571 shutdown_connections 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 --2- 172.21.15.7:0/2849416571 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c109480 0x7fb22c115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 --2- 172.21.15.7:0/2849416571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c10eac0 0x7fb22c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 --2- 172.21.15.7:0/2849416571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c108f40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 -- 172.21.15.7:0/2849416571 >> 172.21.15.7:0/2849416571 conn(0x7fb22c075c60 msgr2=0x7fb22c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 -- 172.21.15.7:0/2849416571 shutdown_connections 2024-06-13T07:12:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.063+0000 7fb231a00700 1 -- 172.21.15.7:0/2849416571 wait complete. 2024-06-13T07:12:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 Processor -- start 2024-06-13T07:12:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 -- start start 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 0x7fb22c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 0x7fb22c1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c118e60 con 0x7fb22c108ac0 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c118ce0 con 0x7fb22c10eac0 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.064+0000 7fb231a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c118fe0 con 0x7fb22c109480 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45892/0 (socket says 172.21.15.7:45892) 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 -- 172.21.15.7:0/1855945437 learned_addr learned my addr 172.21.15.7:0/1855945437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22a000700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 0x7fb22c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22b400700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 0x7fb22c1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 -- 172.21.15.7:0/1855945437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 msgr2=0x7fb22c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 0x7fb22c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 -- 172.21.15.7:0/1855945437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 msgr2=0x7fb22c1a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 0x7fb22c1a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22aa00700 1 -- 172.21.15.7:0/1855945437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c1a42a0 con 0x7fb22c108ac0 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22a000700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 0x7fb22c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.065+0000 7fb22b400700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 0x7fb22c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb22aa00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9c0 tx=0x7fb21c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cbd0 con 0x7fb22c108ac0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21c00cd70 con 0x7fb22c108ac0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c010430 con 0x7fb22c108ac0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c1a4590 con 0x7fb22c108ac0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.066+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c1abe10 con 0x7fb22c108ac0 2024-06-13T07:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.067+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c069180 con 0x7fb22c108ac0 2024-06-13T07:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.070+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb21c004750 con 0x7fb22c108ac0 2024-06-13T07:12:32.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.071+0000 7fb223400700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 0x7fb2100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:32.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.071+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9063+0+0 (secure 0 0 0) 0x7fb21c0cf840 con 0x7fb22c108ac0 2024-06-13T07:12:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.071+0000 7fb22a000700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 0x7fb2100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.072+0000 7fb22a000700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 0x7fb2100a50e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb22c077b50 tx=0x7fb21800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.074+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb21c053850 con 0x7fb22c108ac0 2024-06-13T07:12:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.354+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb22c02dca0 con 0x7fb22c108ac0 2024-06-13T07:12:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.360+0000 7fb223400700 1 -- 172.21.15.7:0/1855945437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7fb21c09c400 con 0x7fb22c108ac0 2024-06-13T07:12:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.363+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 msgr2=0x7fb2100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.363+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 0x7fb2100a50e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb22c077b50 tx=0x7fb21800b260 comp rx=0 tx=0).stop 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.363+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 msgr2=0x7fb22c19f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.363+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9c0 tx=0x7fb21c00ecd0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 shutdown_connections 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7fb2100a2c20 0x7fb2100a50e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb22c10eac0 0x7fb22c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb22c108ac0 0x7fb22c19f290 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 --2- 172.21.15.7:0/1855945437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb22c109480 0x7fb22c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 >> 172.21.15.7:0/1855945437 conn(0x7fb22c075c60 msgr2=0x7fb22c0761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 shutdown_connections 2024-06-13T07:12:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:32.364+0000 7fb231a00700 1 -- 172.21.15.7:0/1855945437 wait complete. 2024-06-13T07:12:32.368 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a[32073]: 2024-06-13T07:12:32.356+0000 7fc9b9200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T07:12:32.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mon[32077]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mds[70871]: -- 172.21.15.7:0/3387369659 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598d9d71a40 con 0x5598d9d44800 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: starting mds.g at 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.665 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.666 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-mds[71289]: -- 172.21.15.7:0/3985716004 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0ad28fa40 con 0x55e0ad263000 2024-06-13T07:12:32.666 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:12:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: starting mds.a at 2024-06-13T07:12:32.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mon[36616]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.837 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mds[67517]: -- 172.21.15.189:0/4160037728 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c773ffba40 con 0x55c773fcec00 2024-06-13T07:12:32.838 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: starting mds.i at 2024-06-13T07:12:32.838 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.838 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.838 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-mds[67201]: -- 172.21.15.189:0/2961056298 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586e3a65a40 con 0x5586e3a38c00 2024-06-13T07:12:32.838 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:12:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: starting mds.c at 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/1855945437' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: osdmap e99: 12 total, 12 up, 12 in 2024-06-13T07:12:32.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mon[36886]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-13T07:12:32.869 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.869 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.869 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mds[66979]: -- 172.21.15.175:0/37462408 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a15f699a40 con 0x55a15f66c800 2024-06-13T07:12:32.869 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: starting mds.b at 2024-06-13T07:12:32.870 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: ignoring --setuser ceph since I am not root 2024-06-13T07:12:32.870 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: ignoring --setgroup ceph since I am not root 2024-06-13T07:12:32.870 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-mds[66659]: -- 172.21.15.175:0/3872648873 <== mon.2 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d6e39fa40 con 0x559d6e373000 2024-06-13T07:12:32.870 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:12:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: starting mds.h at 2024-06-13T07:12:33.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-13T07:12:33.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:33 smithi007 ceph-mon[32077]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:33.711 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:33.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:33 smithi189 ceph-mon[36616]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:33.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:33 smithi175 ceph-mon[36886]: pgmap v1004: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6836/3337813195,v1:172.21.15.189:6838/3337813195] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.189:6837/1116040269,v1:172.21.15.189:6839/1116040269] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6836/2815326522,v1:172.21.15.175:6838/2815326522] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.175:6837/733821285,v1:172.21.15.175:6839/733821285] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6838/672283670,v1:172.21.15.7:6840/672283670] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: mds.? [v2:172.21.15.7:6839/2825141170,v1:172.21.15.7:6841/2825141170] up:boot 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:12:34.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T07:12:34.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:34 smithi007 ceph-mon[32077]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:12:34.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6836/3337813195,v1:172.21.15.189:6838/3337813195] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.189:6837/1116040269,v1:172.21.15.189:6839/1116040269] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6836/2815326522,v1:172.21.15.175:6838/2815326522] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.175:6837/733821285,v1:172.21.15.175:6839/733821285] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6838/672283670,v1:172.21.15.7:6840/672283670] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: mds.? [v2:172.21.15.7:6839/2825141170,v1:172.21.15.7:6841/2825141170] up:boot 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T07:12:34.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:34 smithi189 ceph-mon[36616]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:12:34.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6836/3337813195,v1:172.21.15.189:6838/3337813195] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.189:6837/1116040269,v1:172.21.15.189:6839/1116040269] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6836/2815326522,v1:172.21.15.175:6838/2815326522] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.175:6837/733821285,v1:172.21.15.175:6839/733821285] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6838/672283670,v1:172.21.15.7:6840/672283670] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: mds.? [v2:172.21.15.7:6839/2825141170,v1:172.21.15.7:6841/2825141170] up:boot 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-13T07:12:34.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:34 smithi175 ceph-mon[36886]: from='mgr.14152 172.21.15.7:0/1702448856' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-13T07:12:35.555 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:35 smithi007 ceph-mon[32077]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-06-13T07:12:35.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:35 smithi189 ceph-mon[36616]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-06-13T07:12:35.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:35 smithi175 ceph-mon[36886]: pgmap v1006: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-06-13T07:12:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 -- 172.21.15.7:0/675730016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 msgr2=0x7f34b410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 --2- 172.21.15.7:0/675730016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b410f330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b0a0 tx=0x7f34ac00b3b0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 -- 172.21.15.7:0/675730016 shutdown_connections 2024-06-13T07:12:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 --2- 172.21.15.7:0/675730016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b410f330 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 --2- 172.21.15.7:0/675730016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41084f0 0x7f34b41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 --2- 172.21.15.7:0/675730016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41024e0 0x7f34b4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 -- 172.21.15.7:0/675730016 >> 172.21.15.7:0/675730016 conn(0x7f34b4079530 msgr2=0x7f34b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.175+0000 7f34bb400700 1 -- 172.21.15.7:0/675730016 shutdown_connections 2024-06-13T07:12:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.176+0000 7f34bb400700 1 -- 172.21.15.7:0/675730016 wait complete. 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 Processor -- start 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 -- start start 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41024e0 0x7f34b419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41084f0 0x7f34b419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34b41127b0 con 0x7f34b4102e80 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4112630 con 0x7f34b41084f0 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.177+0000 7f34bb400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4112930 con 0x7f34b41024e0 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45950/0 (socket says 172.21.15.7:45950) 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 -- 172.21.15.7:0/3652718588 learned_addr learned my addr 172.21.15.7:0/3652718588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b8a00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41024e0 0x7f34b419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:36.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b9400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41084f0 0x7f34b419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 -- 172.21.15.7:0/3652718588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41024e0 msgr2=0x7f34b419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41024e0 0x7f34b419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 -- 172.21.15.7:0/3652718588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41084f0 msgr2=0x7f34b419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41084f0 0x7f34b419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.178+0000 7f34b3e00700 1 -- 172.21.15.7:0/3652718588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b419fed0 con 0x7f34b4102e80 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34b3e00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9a0 tx=0x7f34a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a000ca10 con 0x7f34b4102e80 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b41a01c0 con 0x7f34b4102e80 2024-06-13T07:12:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a000cbb0 con 0x7f34b4102e80 2024-06-13T07:12:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0018680 con 0x7f34b4102e80 2024-06-13T07:12:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.179+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41a7a40 con 0x7f34b4102e80 2024-06-13T07:12:36.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.180+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b4069180 con 0x7f34b4102e80 2024-06-13T07:12:36.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.182+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f34a0010430 con 0x7f34b4102e80 2024-06-13T07:12:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.184+0000 7f34b1600700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 0x7f349c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.184+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9321+0+0 (secure 0 0 0) 0x7f34a00cfb50 con 0x7f34b4102e80 2024-06-13T07:12:36.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.184+0000 7f34b8a00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 0x7f349c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.185+0000 7f34b8a00700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 0x7f349c0a5160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009be0 tx=0x7f34a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:36.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.187+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f34a009c610 con 0x7f34b4102e80 2024-06-13T07:12:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:36.498+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34b41a7c70 con 0x7f34b4102e80 2024-06-13T07:12:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.417+0000 7f34b1600700 1 -- 172.21.15.7:0/3652718588 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f34a009bf10 con 0x7f34b4102e80 2024-06-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 msgr2=0x7f349c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 0x7f349c0a5160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009be0 tx=0x7f34a8009260 comp rx=0 tx=0).stop 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 msgr2=0x7f34b419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9a0 tx=0x7f34a000ecb0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 shutdown_connections 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f349c0a2ca0 0x7f349c0a5160 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b41084f0 0x7f34b419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34b41024e0 0x7f34b419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 --2- 172.21.15.7:0/3652718588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b4102e80 0x7f34b419b400 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.421+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 >> 172.21.15.7:0/3652718588 conn(0x7f34b4079530 msgr2=0x7f34b4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.422+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 shutdown_connections 2024-06-13T07:12:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:37.422+0000 7f34bb400700 1 -- 172.21.15.7:0/3652718588 wait complete. 2024-06-13T07:12:37.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:37 smithi007 ceph-mon[32077]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:37.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:37 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:12:37.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:37 smithi189 ceph-mon[36616]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:37.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:37 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:12:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:37 smithi175 ceph-mon[36886]: pgmap v1007: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:37.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:37 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-13T07:12:38.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-13T07:12:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:38 smithi007 ceph-mon[32077]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:12:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:38 smithi007 ceph-mon[32077]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:12:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:38 smithi007 ceph-mon[32077]: Cluster is now healthy 2024-06-13T07:12:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:38 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:12:38.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:38 smithi007 ceph-mon[32077]: fsmap 12 up:standby 2024-06-13T07:12:38.696 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:38 smithi175 ceph-mon[36886]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:12:38.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:38 smithi175 ceph-mon[36886]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:12:38.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:38 smithi175 ceph-mon[36886]: Cluster is now healthy 2024-06-13T07:12:38.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:38 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:12:38.697 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:38 smithi175 ceph-mon[36886]: fsmap 12 up:standby 2024-06-13T07:12:38.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:38 smithi189 ceph-mon[36616]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-13T07:12:38.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:38 smithi189 ceph-mon[36616]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-13T07:12:38.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:38 smithi189 ceph-mon[36616]: Cluster is now healthy 2024-06-13T07:12:38.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:38 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3652718588' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-13T07:12:38.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:38 smithi189 ceph-mon[36616]: fsmap 12 up:standby 2024-06-13T07:12:38.857 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:39.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:39 smithi007 ceph-mon[32077]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:39.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:39 smithi189 ceph-mon[36616]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:39.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:39 smithi175 ceph-mon[36886]: pgmap v1008: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-06-13T07:12:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 -- 172.21.15.7:0/1412867239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8109480 msgr2=0x7f8ea8115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 --2- 172.21.15.7:0/1412867239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8109480 0x7f8ea8115990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000b0a0 tx=0x7f8ea000b3b0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 -- 172.21.15.7:0/1412867239 shutdown_connections 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 --2- 172.21.15.7:0/1412867239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8109480 0x7f8ea8115990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 --2- 172.21.15.7:0/1412867239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 0x7f8ea810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 --2- 172.21.15.7:0/1412867239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8108ac0 0x7f8ea8108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 -- 172.21.15.7:0/1412867239 >> 172.21.15.7:0/1412867239 conn(0x7f8ea8075c60 msgr2=0x7f8ea81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 -- 172.21.15.7:0/1412867239 shutdown_connections 2024-06-13T07:12:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.297+0000 7f8ead800700 1 -- 172.21.15.7:0/1412867239 wait complete. 2024-06-13T07:12:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.298+0000 7f8ead800700 1 Processor -- start 2024-06-13T07:12:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.299+0000 7f8ead800700 1 -- start start 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.299+0000 7f8ead800700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.299+0000 7f8ead800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 0x7f8ea819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ead800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 0x7f8ea81a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ead800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8118e10 con 0x7f8ea810eac0 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ead800700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8118c90 con 0x7f8ea8108ac0 2024-06-13T07:12:41.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ead800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8118f90 con 0x7f8ea8109480 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 0x7f8ea819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 0x7f8ea81a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42044/0 (socket says 172.21.15.7:42044) 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 0x7f8ea819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.7:37144/0 (socket says 172.21.15.7:37144) 2024-06-13T07:12:41.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 -- 172.21.15.7:0/3790152961 learned_addr learned my addr 172.21.15.7:0/3790152961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 -- 172.21.15.7:0/3790152961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 msgr2=0x7f8ea819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 0x7f8ea819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 -- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 msgr2=0x7f8ea81a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 0x7f8ea81a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.300+0000 7f8ea6a00700 1 -- 172.21.15.7:0/3790152961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea81a4270 con 0x7f8ea8108ac0 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8ea6a00700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800e9a0 tx=0x7f8e9800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800ca10 con 0x7f8ea8108ac0 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9800cbb0 con 0x7f8ea8108ac0 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea81a4560 con 0x7f8ea8108ac0 2024-06-13T07:12:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98018680 con 0x7f8ea8108ac0 2024-06-13T07:12:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.301+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81abde0 con 0x7f8ea8108ac0 2024-06-13T07:12:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.303+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea8069180 con 0x7f8ea8108ac0 2024-06-13T07:12:41.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.305+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8e98010430 con 0x7f8ea8108ac0 2024-06-13T07:12:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.306+0000 7f8e9f400700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 0x7f8e8c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.306+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9321+0+0 (secure 0 0 0) 0x7f8e980cf610 con 0x7f8ea8108ac0 2024-06-13T07:12:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.306+0000 7f8ea6000700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 0x7f8e8c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.307+0000 7f8ea6000700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 0x7f8e8c0a5160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8077a60 tx=0x7f8e94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.310+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e9809c0d0 con 0x7f8ea8108ac0 2024-06-13T07:12:41.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:41.541+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ea81abfb0 con 0x7f8ea8108ac0 2024-06-13T07:12:41.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:41 smithi007 ceph-mon[32077]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 614 B/s wr, 3 op/s 2024-06-13T07:12:41.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:41 smithi189 ceph-mon[36616]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 614 B/s wr, 3 op/s 2024-06-13T07:12:41.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:41 smithi175 ceph-mon[36886]: pgmap v1009: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 614 B/s wr, 3 op/s 2024-06-13T07:12:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.435+0000 7f8e9f400700 1 -- 172.21.15.7:0/3790152961 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f8e9809b9d0 con 0x7f8ea8108ac0 2024-06-13T07:12:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.439+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 msgr2=0x7f8e8c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 0x7f8e8c0a5160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8077a60 tx=0x7f8e94009260 comp rx=0 tx=0).stop 2024-06-13T07:12:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 msgr2=0x7f8ea819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800e9a0 tx=0x7f8e9800ecb0 comp rx=0 tx=0).stop 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 shutdown_connections 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea810eac0 0x7f8ea81a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f8e8c0a2ca0 0x7f8e8c0a5160 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ea8108ac0 0x7f8ea819f260 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 --2- 172.21.15.7:0/3790152961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8109480 0x7f8ea819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 >> 172.21.15.7:0/3790152961 conn(0x7f8ea8075c60 msgr2=0x7f8ea8076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.440+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 shutdown_connections 2024-06-13T07:12:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:42.441+0000 7f8ead800700 1 -- 172.21.15.7:0/3790152961 wait complete. 2024-06-13T07:12:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-06-13T07:12:42.831 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:42 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/3790152961' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:42.832 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:42 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:42.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:42 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/3790152961' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:42.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:42 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:42.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:42 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/3790152961' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:42.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:42 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:43.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-13T07:12:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:43 smithi007 ceph-mon[32077]: pgmap v1010: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-13T07:12:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:43 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:43.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:43 smithi007 ceph-mon[32077]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:12:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:43 smithi189 ceph-mon[36616]: pgmap v1010: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-13T07:12:43.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:43 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:43.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:43 smithi189 ceph-mon[36616]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:12:43.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:43 smithi175 ceph-mon[36886]: pgmap v1010: 129 pgs: 129 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-13T07:12:43.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:43 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:43.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:43 smithi175 ceph-mon[36886]: osdmap e100: 12 total, 12 up, 12 in 2024-06-13T07:12:45.637 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:45 smithi007 ceph-mon[32077]: pgmap v1012: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:45.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:45 smithi189 ceph-mon[36616]: pgmap v1012: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:45.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:45 smithi175 ceph-mon[36886]: pgmap v1012: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 -- 172.21.15.7:0/2948397743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c81045c0 msgr2=0x7f19c8104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2948397743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c81045c0 0x7f19c8104a40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f19b00099c0 tx=0x7f19b0009cd0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 -- 172.21.15.7:0/2948397743 shutdown_connections 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2948397743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c8104f80 0x7f19c8111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2948397743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 0x7f19c810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2948397743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c81045c0 0x7f19c8104a40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 -- 172.21.15.7:0/2948397743 >> 172.21.15.7:0/2948397743 conn(0x7f19c8100070 msgr2=0x7f19c8102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 -- 172.21.15.7:0/2948397743 shutdown_connections 2024-06-13T07:12:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.928+0000 7f19c7e00700 1 -- 172.21.15.7:0/2948397743 wait complete. 2024-06-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 Processor -- start 2024-06-13T07:12:45.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 -- start start 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81045c0 0x7f19c81a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 0x7f19c819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81148e0 con 0x7f19c8104f80 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8114760 con 0x7f19c810a670 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19c7e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8114a60 con 0x7f19c81045c0 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.930+0000 7f19bfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50918/0 (socket says 172.21.15.7:50918) 2024-06-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 -- 172.21.15.7:0/2629688092 learned_addr learned my addr 172.21.15.7:0/2629688092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19c6a00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81045c0 0x7f19c81a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19c7400700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 0x7f19c819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 -- 172.21.15.7:0/2629688092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81045c0 msgr2=0x7f19c81a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81045c0 0x7f19c81a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 -- 172.21.15.7:0/2629688092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 msgr2=0x7f19c819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 0x7f19c819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 -- 172.21.15.7:0/2629688092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c819b240 con 0x7f19c8104f80 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.931+0000 7f19bfe00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f19b0006010 tx=0x7f19b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.932+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0003680 con 0x7f19c8104f80 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.932+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b00050d0 con 0x7f19c8104f80 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.932+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0003db0 con 0x7f19c8104f80 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.932+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c819b4d0 con 0x7f19c8104f80 2024-06-13T07:12:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.932+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a77a0 con 0x7f19c8104f80 2024-06-13T07:12:45.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.933+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c8105ce0 con 0x7f19c8104f80 2024-06-13T07:12:45.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.935+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f19b0005280 con 0x7f19c8104f80 2024-06-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.938+0000 7f19bf400700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 0x7f19ac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.938+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8825+0+0 (secure 0 0 0) 0x7f19b00d00f0 con 0x7f19c8104f80 2024-06-13T07:12:45.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.938+0000 7f19c6a00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 0x7f19ac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:45.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.939+0000 7f19c6a00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 0x7f19ac0a5300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f19c8109680 tx=0x7f19b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:45.941+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f19b005d850 con 0x7f19c8104f80 2024-06-13T07:12:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.176+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19c819bfc0 con 0x7f19c8104f80 2024-06-13T07:12:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.455+0000 7f19bf400700 1 -- 172.21.15.7:0/2629688092 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f19b0013070 con 0x7f19c8104f80 2024-06-13T07:12:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 msgr2=0x7f19ac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 0x7f19ac0a5300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f19c8109680 tx=0x7f19b8009260 comp rx=0 tx=0).stop 2024-06-13T07:12:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 msgr2=0x7f19c81a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f19b0006010 tx=0x7f19b000f690 comp rx=0 tx=0).stop 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 shutdown_connections 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7f19ac0a2e40 0x7f19ac0a5300 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19c810a670 0x7f19c819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c81045c0 0x7f19c81a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 --2- 172.21.15.7:0/2629688092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19c8104f80 0x7f19c81a0e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.460+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 >> 172.21.15.7:0/2629688092 conn(0x7f19c8100070 msgr2=0x7f19c8101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.463+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 shutdown_connections 2024-06-13T07:12:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:46.463+0000 7f19c7e00700 1 -- 172.21.15.7:0/2629688092 wait complete. 2024-06-13T07:12:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-06-13T07:12:46.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:46 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:46.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:46 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:46.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:46 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:47.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:47 smithi007 ceph-mon[32077]: pgmap v1013: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:47.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:47 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:47.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:47 smithi007 ceph-mon[32077]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:12:47.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b9295806-294f-11ef-bc9c-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-13T07:12:47.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:47 smithi189 ceph-mon[36616]: pgmap v1013: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:47.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:47 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:47.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:47 smithi189 ceph-mon[36616]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:12:47.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:47 smithi175 ceph-mon[36886]: pgmap v1013: 97 pgs: 97 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:47 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2629688092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:47.868 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:47 smithi175 ceph-mon[36886]: osdmap e101: 12 total, 12 up, 12 in 2024-06-13T07:12:47.941 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/mon.a/config 2024-06-13T07:12:49.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:49 smithi007 ceph-mon[32077]: pgmap v1015: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:49.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:49 smithi189 ceph-mon[36616]: pgmap v1015: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:49.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:49 smithi175 ceph-mon[36886]: pgmap v1015: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 -- 172.21.15.7:0/2918129832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 msgr2=0x7ffa98104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2918129832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa98104b30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009990 tx=0x7ffa84009ca0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 -- 172.21.15.7:0/2918129832 shutdown_connections 2024-06-13T07:12:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2918129832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa98105070 0x7ffa98111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2918129832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa9810a6b0 0x7ffa9810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2918129832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa98104b30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.246+0000 7ffa9f600700 1 -- 172.21.15.7:0/2918129832 >> 172.21.15.7:0/2918129832 conn(0x7ffa981001a0 msgr2=0x7ffa981025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.247+0000 7ffa9f600700 1 -- 172.21.15.7:0/2918129832 shutdown_connections 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.247+0000 7ffa9f600700 1 -- 172.21.15.7:0/2918129832 wait complete. 2024-06-13T07:12:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.248+0000 7ffa9f600700 1 Processor -- start 2024-06-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.248+0000 7ffa9f600700 1 -- start start 2024-06-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 0x7ffa9819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa9810a6b0 0x7ffa9819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98114a40 con 0x7ffa98105070 2024-06-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa981148c0 con 0x7ffa981046b0 2024-06-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9f600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98114bc0 con 0x7ffa9810a6b0 2024-06-13T07:12:50.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9cc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9cc00700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.7:42080/0 (socket says 172.21.15.7:42080) 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa97e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 0x7ffa9819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9cc00700 1 -- 172.21.15.7:0/2967105680 learned_addr learned my addr 172.21.15.7:0/2967105680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.249+0000 7ffa9d600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa9810a6b0 0x7ffa9819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 -- 172.21.15.7:0/2967105680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa9810a6b0 msgr2=0x7ffa9819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa9810a6b0 0x7ffa9819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 -- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 msgr2=0x7ffa9819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 0x7ffa9819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 -- 172.21.15.7:0/2967105680 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9819fee0 con 0x7ffa981046b0 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa97e00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 0x7ffa9819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-13T07:12:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa9cc00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9a0 tx=0x7ffa8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.250+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00ca10 con 0x7ffa981046b0 2024-06-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.251+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa981a01d0 con 0x7ffa981046b0 2024-06-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.251+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa981a7a50 con 0x7ffa981046b0 2024-06-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.251+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c00cbb0 con 0x7ffa981046b0 2024-06-13T07:12:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.251+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c018680 con 0x7ffa981046b0 2024-06-13T07:12:50.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.252+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78005330 con 0x7ffa981046b0 2024-06-13T07:12:50.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.254+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ffa8c018820 con 0x7ffa981046b0 2024-06-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.255+0000 7ffa95600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 0x7ffa800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.256+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8324+0+0 (secure 0 0 0) 0x7ffa8c0cf1c0 con 0x7ffa981046b0 2024-06-13T07:12:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.256+0000 7ffa97e00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 0x7ffa800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-13T07:12:50.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.257+0000 7ffa97e00700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 0x7ffa800a5160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffa98109cb0 tx=0x7ffa84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-13T07:12:50.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.258+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa8c09c060 con 0x7ffa981046b0 2024-06-13T07:12:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:50.489+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffa78005620 con 0x7ffa981046b0 2024-06-13T07:12:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.487+0000 7ffa95600700 1 -- 172.21.15.7:0/2967105680 <== mon.2 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v102) ==== 178+0+0 (secure 0 0 0) 0x7ffa8c09b960 con 0x7ffa981046b0 2024-06-13T07:12:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.490+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 msgr2=0x7ffa800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 0x7ffa800a5160 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffa98109cb0 tx=0x7ffa84005fd0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 msgr2=0x7ffa9819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9a0 tx=0x7ffa8c00ecb0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 shutdown_connections 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa9810a6b0 0x7ffa9819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:6800/2901233239,v1:172.21.15.7:6801/2901233239] conn(0x7ffa800a2ca0 0x7ffa800a5160 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa981046b0 0x7ffa9819aed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 --2- 172.21.15.7:0/2967105680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa98105070 0x7ffa9819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 >> 172.21.15.7:0/2967105680 conn(0x7ffa981001a0 msgr2=0x7ffa98100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 shutdown_connections 2024-06-13T07:12:51.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-06-13T07:12:51.491+0000 7ffa9f600700 1 -- 172.21.15.7:0/2967105680 wait complete. 2024-06-13T07:12:51.492 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-06-13T07:12:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:51 smithi189 ceph-mon[36616]: pgmap v1016: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:51.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:51 smithi189 ceph-mon[36616]: from='client.? 172.21.15.7:0/2967105680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:51.837 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:51 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:51.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:51 smithi007 ceph-mon[32077]: pgmap v1016: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:51.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:51 smithi007 ceph-mon[32077]: from='client.? 172.21.15.7:0/2967105680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:51.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:51 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:51 smithi175 ceph-mon[36886]: pgmap v1016: 33 pgs: 33 active+clean; 1.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-13T07:12:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:51 smithi175 ceph-mon[36886]: from='client.? 172.21.15.7:0/2967105680' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:51.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:51 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-13T07:12:52.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:12:52.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:12:52.426 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:12:52.455 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-13T07:12:52.455 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:12:52.520 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:12:52.541 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-13T07:12:52.559 INFO:tasks.cephadm:Stopping all daemons... 2024-06-13T07:12:52.559 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-13T07:12:52.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a 2024-06-13T07:12:52.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:52 smithi189 ceph-mon[36616]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:52.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:52 smithi189 ceph-mon[36616]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:12:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:52 smithi175 ceph-mon[36886]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:52.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:52 smithi175 ceph-mon[36886]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:12:52.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:52 smithi007 ceph-mon[32077]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-13T07:12:52.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:52 smithi007 ceph-mon[32077]: osdmap e102: 12 total, 12 up, 12 in 2024-06-13T07:12:52.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:52 smithi007 systemd[1]: Stopping Ceph mon.a for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:12:53.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a[32073]: 2024-06-13T07:12:53.278+0000 7fc9c0000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:12:53.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a[32073]: 2024-06-13T07:12:53.278+0000 7fc9c0000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-13T07:12:53.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:53 smithi007 podman[114179]: 2024-06-13 07:12:53.381476112 +0000 UTC m=+0.263071784 container died 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-13T07:12:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:53 smithi007 podman[114179]: 2024-06-13 07:12:53.822219066 +0000 UTC m=+0.703814724 container cleanup 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-13T07:12:54.071 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:53 smithi007 bash[114179]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a 2024-06-13T07:12:54.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Jun 13 07:12:54 smithi007 podman[114190]: 2024-06-13 07:12:54.071826308 +0000 UTC m=+0.677192281 container remove 1f9fe77f8603fd7940139d2c1718fe1fd08f8aa98c037b38aafe3d528626f420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-13T07:12:54.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.a.service' 2024-06-13T07:12:54.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:12:54.559 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-13T07:12:54.559 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-13T07:12:54.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.b 2024-06-13T07:12:54.867 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:54 smithi175 systemd[1]: Stopping Ceph mon.b for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:12:55.617 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-b[36882]: 2024-06-13T07:12:55.274+0000 7f72e839a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:12:55.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-b[36882]: 2024-06-13T07:12:55.274+0000 7f72e839a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-13T07:12:55.618 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:55 smithi175 podman[73004]: 2024-06-13 07:12:55.323642753 +0000 UTC m=+0.181486426 container died c3844d35f6f48a36b3dae4fc65b61fc758b53a08f541b2dd5f3aa0d22c9fdbaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-13T07:12:56.302 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:56 smithi175 podman[73004]: 2024-06-13 07:12:56.026270606 +0000 UTC m=+0.884114275 container cleanup c3844d35f6f48a36b3dae4fc65b61fc758b53a08f541b2dd5f3aa0d22c9fdbaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-13T07:12:56.302 INFO:journalctl@ceph.mon.b.smithi175.stdout:Jun 13 07:12:56 smithi175 bash[73004]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-b 2024-06-13T07:12:56.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.b.service' 2024-06-13T07:12:56.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:12:56.484 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-13T07:12:56.484 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-13T07:12:56.484 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.c 2024-06-13T07:12:56.836 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:56 smithi189 systemd[1]: Stopping Ceph mon.c for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:12:57.281 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c[36602]: 2024-06-13T07:12:56.955+0000 7f9f404c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:12:57.281 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c[36602]: 2024-06-13T07:12:56.955+0000 7f9f404c1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-13T07:12:57.281 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:56 smithi189 podman[74014]: 2024-06-13 07:12:56.976763902 +0000 UTC m=+0.155948416 container died 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-13T07:12:57.586 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:57 smithi189 podman[74014]: 2024-06-13 07:12:57.38176385 +0000 UTC m=+0.560948362 container cleanup 462d97e6aa1467940bdb5f99aa57059fe42d6ad7c713d0ef88cb85e879a9c257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:12:57.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Jun 13 07:12:57 smithi189 bash[74014]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mon-c 2024-06-13T07:12:57.841 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mon.c.service' 2024-06-13T07:12:57.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:12:57.997 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-13T07:12:57.997 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-13T07:12:57.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x 2024-06-13T07:12:58.413 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:58 smithi007 systemd[1]: Stopping Ceph mgr.x for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:12:58.914 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T07:12:58.575+0000 7f7fe9600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:12:58.914 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x[32304]: 2024-06-13T07:12:58.575+0000 7f7fe9600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-13T07:12:58.914 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:58 smithi007 podman[114377]: 2024-06-13 07:12:58.668963013 +0000 UTC m=+0.234679480 container died 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-13T07:12:59.346 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:59 smithi007 podman[114377]: 2024-06-13 07:12:59.136733638 +0000 UTC m=+0.702450117 container cleanup 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:12:59.346 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:59 smithi007 bash[114377]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x 2024-06-13T07:12:59.663 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Jun 13 07:12:59 smithi007 podman[114391]: 2024-06-13 07:12:59.346488625 +0000 UTC m=+0.655678251 container remove 0f285255b691be725b4ba2f32d03748f072ae4308e4188ec778f2176deada95e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-x, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-06-13T07:12:59.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.x.service' 2024-06-13T07:12:59.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:12:59.955 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-13T07:12:59.956 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-13T07:12:59.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.y 2024-06-13T07:13:00.257 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:00 smithi175 systemd[1]: Stopping Ceph mgr.y for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:00.617 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:00 smithi175 podman[73208]: 2024-06-13 07:13:00.318607753 +0000 UTC m=+0.148432931 container died fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-13T07:13:01.117 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:00 smithi175 podman[73208]: 2024-06-13 07:13:00.973603263 +0000 UTC m=+0.803428442 container cleanup fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:13:01.117 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:00 smithi175 bash[73208]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y 2024-06-13T07:13:01.490 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:01 smithi175 podman[73221]: 2024-06-13 07:13:01.433845193 +0000 UTC m=+1.111088022 container remove fc9ef54016b98ca6bad6d7633652aa79ceb5f0c42692d1fecc9a982679518ad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-13T07:13:01.490 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Jun 13 07:13:01 smithi175 systemd[1]: ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-13T07:13:01.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.y.service' 2024-06-13T07:13:01.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:01.628 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-13T07:13:01.628 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-13T07:13:01.628 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.z 2024-06-13T07:13:01.966 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:01 smithi189 systemd[1]: Stopping Ceph mgr.z for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:02.311 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:01 smithi189 podman[74137]: 2024-06-13 07:13:01.96641164 +0000 UTC m=+0.137819263 container died 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:13:02.586 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:02 smithi189 podman[74137]: 2024-06-13 07:13:02.312428817 +0000 UTC m=+0.483836392 container cleanup 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-13T07:13:02.586 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:02 smithi189 bash[74137]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z 2024-06-13T07:13:02.587 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:02 smithi189 podman[74152]: 2024-06-13 07:13:02.522288887 +0000 UTC m=+0.544215633 container remove 94ce2ec073c9c854e83ca801d05773993a48b486aa075eebe8cbac676e2ed7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-13T07:13:02.587 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Jun 13 07:13:02 smithi189 systemd[1]: ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-06-13T07:13:02.741 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mgr.z.service' 2024-06-13T07:13:02.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:02.775 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-13T07:13:02.775 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-13T07:13:02.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.0 2024-06-13T07:13:03.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:02 smithi007 systemd[1]: Stopping Ceph osd.0 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:03.664 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0[44384]: 2024-06-13T07:13:03.397+0000 7fd1baa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:03.664 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0[44384]: 2024-06-13T07:13:03.397+0000 7fd1baa00700 -1 osd.0 102 *** Got signal Terminated *** 2024-06-13T07:13:03.664 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0[44384]: 2024-06-13T07:13:03.397+0000 7fd1baa00700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:09.487 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:09 smithi007 podman[114497]: 2024-06-13 07:13:09.176025598 +0000 UTC m=+5.925057929 container died 733eaa5a3d07d293e7318b66df98cbba0b762721c5a2ad0ebf3d5a1422c2de2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-13T07:13:09.827 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:09 smithi007 podman[114497]: 2024-06-13 07:13:09.617378303 +0000 UTC m=+6.366410671 container cleanup 733eaa5a3d07d293e7318b66df98cbba0b762721c5a2ad0ebf3d5a1422c2de2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T07:13:09.827 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:09 smithi007 bash[114497]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0 2024-06-13T07:13:10.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:09 smithi007 podman[114515]: 2024-06-13 07:13:09.826894309 +0000 UTC m=+0.628218980 container remove 733eaa5a3d07d293e7318b66df98cbba0b762721c5a2ad0ebf3d5a1422c2de2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:13:11.014 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:10 smithi007 podman[114604]: 2024-06-13 07:13:10.622739679 +0000 UTC m=+0.036091944 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:11.014 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:10 smithi007 podman[114604]: 2024-06-13 07:13:10.736097369 +0000 UTC m=+0.149449672 container create bfe9eb06a729f3a5f5098c2d0c95823fd361443ef8f9268ad1d3a2468e4eacb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T07:13:11.413 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:11 smithi007 podman[114604]: 2024-06-13 07:13:11.014738434 +0000 UTC m=+0.428090740 container init bfe9eb06a729f3a5f5098c2d0c95823fd361443ef8f9268ad1d3a2468e4eacb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-13T07:13:11.413 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:11 smithi007 podman[114604]: 2024-06-13 07:13:11.026745207 +0000 UTC m=+0.440097514 container start bfe9eb06a729f3a5f5098c2d0c95823fd361443ef8f9268ad1d3a2468e4eacb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-13T07:13:11.413 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:11 smithi007 podman[114604]: 2024-06-13 07:13:11.113667124 +0000 UTC m=+0.527019471 container attach bfe9eb06a729f3a5f5098c2d0c95823fd361443ef8f9268ad1d3a2468e4eacb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-13T07:13:12.163 INFO:journalctl@ceph.osd.0.smithi007.stdout:Jun 13 07:13:11 smithi007 podman[114604]: 2024-06-13 07:13:11.807915108 +0000 UTC m=+1.221267405 container died bfe9eb06a729f3a5f5098c2d0c95823fd361443ef8f9268ad1d3a2468e4eacb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True) 2024-06-13T07:13:12.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.0.service' 2024-06-13T07:13:12.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:12.648 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-13T07:13:12.648 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-13T07:13:12.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.1 2024-06-13T07:13:12.969 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:12 smithi175 systemd[1]: Stopping Ceph osd.1 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:13.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1[41444]: 2024-06-13T07:13:12.967+0000 7fb0b210c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:13.368 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1[41444]: 2024-06-13T07:13:12.967+0000 7fb0b210c700 -1 osd.1 102 *** Got signal Terminated *** 2024-06-13T07:13:13.368 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1[41444]: 2024-06-13T07:13:12.967+0000 7fb0b210c700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:18.735 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:18 smithi175 podman[73345]: 2024-06-13 07:13:18.460867035 +0000 UTC m=+5.612701598 container died 72daf7692a12c10c57daf3adb996eaca7dde16bb36e324072e21d43666bda10f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-13T07:13:19.103 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:18 smithi175 podman[73345]: 2024-06-13 07:13:18.852869124 +0000 UTC m=+6.004703684 container cleanup 72daf7692a12c10c57daf3adb996eaca7dde16bb36e324072e21d43666bda10f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-13T07:13:19.103 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:18 smithi175 bash[73345]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1 2024-06-13T07:13:19.367 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73362]: 2024-06-13 07:13:19.104538989 +0000 UTC m=+0.641976227 container remove 72daf7692a12c10c57daf3adb996eaca7dde16bb36e324072e21d43666bda10f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-06-13T07:13:19.723 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73446]: 2024-06-13 07:13:19.364961167 +0000 UTC m=+0.017004862 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:19.723 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73446]: 2024-06-13 07:13:19.480277937 +0000 UTC m=+0.132321629 container create 772136ddc93429642276d0c5116c4a908641b3f997305c72eff1d29c719f42b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-13T07:13:19.723 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73446]: 2024-06-13 07:13:19.724346642 +0000 UTC m=+0.376390340 container init 772136ddc93429642276d0c5116c4a908641b3f997305c72eff1d29c719f42b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-13T07:13:20.118 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73446]: 2024-06-13 07:13:19.727828 +0000 UTC m=+0.379871693 container start 772136ddc93429642276d0c5116c4a908641b3f997305c72eff1d29c719f42b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:20.118 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:19 smithi175 podman[73446]: 2024-06-13 07:13:19.849533163 +0000 UTC m=+0.501576862 container attach 772136ddc93429642276d0c5116c4a908641b3f997305c72eff1d29c719f42b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-13T07:13:20.481 INFO:journalctl@ceph.osd.1.smithi175.stdout:Jun 13 07:13:20 smithi175 podman[73446]: 2024-06-13 07:13:20.211612844 +0000 UTC m=+0.863656545 container died 772136ddc93429642276d0c5116c4a908641b3f997305c72eff1d29c719f42b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-13T07:13:20.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.1.service' 2024-06-13T07:13:20.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:20.867 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-13T07:13:20.867 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-13T07:13:20.867 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.2 2024-06-13T07:13:21.166 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:20 smithi189 systemd[1]: Stopping Ceph osd.2 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:21.481 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2[41683]: 2024-06-13T07:13:21.165+0000 7f71c8782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:21.481 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2[41683]: 2024-06-13T07:13:21.165+0000 7f71c8782700 -1 osd.2 102 *** Got signal Terminated *** 2024-06-13T07:13:21.481 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2[41683]: 2024-06-13T07:13:21.165+0000 7f71c8782700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:26.983 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:26 smithi189 podman[74259]: 2024-06-13 07:13:26.693291619 +0000 UTC m=+5.634048246 container died 51f7ec104a1c307133c1548c0d508963489367578e0f6219316580b386ea563d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-13T07:13:27.336 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74259]: 2024-06-13 07:13:27.083225812 +0000 UTC m=+6.023982427 container cleanup 51f7ec104a1c307133c1548c0d508963489367578e0f6219316580b386ea563d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-13T07:13:27.337 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 bash[74259]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2 2024-06-13T07:13:27.706 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74286]: 2024-06-13 07:13:27.368306812 +0000 UTC m=+0.663510252 container remove 51f7ec104a1c307133c1548c0d508963489367578e0f6219316580b386ea563d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:28.063 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74377]: 2024-06-13 07:13:27.610421124 +0000 UTC m=+0.017675765 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:28.063 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74377]: 2024-06-13 07:13:27.727604063 +0000 UTC m=+0.134858695 container create 864ea69778ce7ff005b5a61653b3f2288c1304783c3b76878af4bc7010fda970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:28.063 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74377]: 2024-06-13 07:13:27.947208606 +0000 UTC m=+0.354463252 container init 864ea69778ce7ff005b5a61653b3f2288c1304783c3b76878af4bc7010fda970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-13T07:13:28.063 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:27 smithi189 podman[74377]: 2024-06-13 07:13:27.950280795 +0000 UTC m=+0.357535426 container start 864ea69778ce7ff005b5a61653b3f2288c1304783c3b76878af4bc7010fda970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-13T07:13:28.063 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:28 smithi189 podman[74377]: 2024-06-13 07:13:28.063763919 +0000 UTC m=+0.471018564 container attach 864ea69778ce7ff005b5a61653b3f2288c1304783c3b76878af4bc7010fda970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-13T07:13:28.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Jun 13 07:13:28 smithi189 podman[74377]: 2024-06-13 07:13:28.426472062 +0000 UTC m=+0.833726695 container died 864ea69778ce7ff005b5a61653b3f2288c1304783c3b76878af4bc7010fda970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-06-13T07:13:29.048 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.2.service' 2024-06-13T07:13:29.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:29.088 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-13T07:13:29.088 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-13T07:13:29.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.3 2024-06-13T07:13:29.663 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:29 smithi007 systemd[1]: Stopping Ceph osd.3 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:29.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:29 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:29.408+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:30.163 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:29 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T07:13:29.693+0000 7fcab4a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:30.163 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:29 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T07:13:29.693+0000 7fcab4a00700 -1 osd.3 102 *** Got signal Terminated *** 2024-06-13T07:13:30.163 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:29 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T07:13:29.693+0000 7fcab4a00700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:30.695 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:30 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:30.437+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:31.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:30.948+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:31.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:31 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:31.481+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:32.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:31.969+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:32.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:32 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:32.515+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:33.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:33.008+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:33.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:33 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:32.999+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:33.913 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:33 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3[51275]: 2024-06-13T07:13:33.722+0000 7fcab0200700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:07.095066+0000 front 2024-06-13T07:13:07.094367+0000 (oldest deadline 2024-06-13T07:13:32.994525+0000) 2024-06-13T07:13:33.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:33 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:33.522+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:34.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:33.745+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:33.818+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:34.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:34.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:34.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:34 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:34.033+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:34.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:34 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:34.536+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:35.086 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:34.738+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:34.833+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:35.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:34.978+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:35.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:35 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:35.052+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:35.913 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:35 smithi007 podman[114708]: 2024-06-13 07:13:35.612344723 +0000 UTC m=+6.063940269 container died ecb354582b200285e9b9b623384f4f6dd3c241295a579d5136929c0bbb0b5af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-13T07:13:35.914 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:35 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:35.553+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:36.086 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:35.739+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:35.880+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:36.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:35.931+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:36.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:36 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:36.093+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:36.413 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:36 smithi007 podman[114708]: 2024-06-13 07:13:36.119900753 +0000 UTC m=+6.571496259 container cleanup ecb354582b200285e9b9b623384f4f6dd3c241295a579d5136929c0bbb0b5af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-13T07:13:36.413 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:36 smithi007 bash[114708]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3 2024-06-13T07:13:36.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:36 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:36.599+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:36.663 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:36 smithi007 podman[114741]: 2024-06-13 07:13:36.413095007 +0000 UTC m=+0.782103920 container remove ecb354582b200285e9b9b623384f4f6dd3c241295a579d5136929c0bbb0b5af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:13:37.086 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:36.772+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:36.867+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:37.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:36.980+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:37.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:37 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:37.078+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:37.413 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:37 smithi007 podman[114828]: 2024-06-13 07:13:37.178174654 +0000 UTC m=+0.042280186 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:37.413 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:37 smithi007 podman[114828]: 2024-06-13 07:13:37.289109952 +0000 UTC m=+0.153215510 container create df16e2423d2fafa91b3d50fdde69c3b14e06fa92633f8a9e9fa427d43c7d7100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True) 2024-06-13T07:13:37.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:37 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:37.627+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:37.913 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:37 smithi007 podman[114828]: 2024-06-13 07:13:37.58474975 +0000 UTC m=+0.448855315 container init df16e2423d2fafa91b3d50fdde69c3b14e06fa92633f8a9e9fa427d43c7d7100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:37.913 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:37 smithi007 podman[114828]: 2024-06-13 07:13:37.597623064 +0000 UTC m=+0.461728598 container start df16e2423d2fafa91b3d50fdde69c3b14e06fa92633f8a9e9fa427d43c7d7100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:13:37.914 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:37 smithi007 podman[114828]: 2024-06-13 07:13:37.683389039 +0000 UTC m=+0.547494677 container attach df16e2423d2fafa91b3d50fdde69c3b14e06fa92633f8a9e9fa427d43c7d7100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:38.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:37.829+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:38.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:37.736+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:38.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:38 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:38.019+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:38.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:38 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:38.112+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:38.413 INFO:journalctl@ceph.osd.3.smithi007.stdout:Jun 13 07:13:38 smithi007 podman[114828]: 2024-06-13 07:13:38.063399105 +0000 UTC m=+0.927504670 container died df16e2423d2fafa91b3d50fdde69c3b14e06fa92633f8a9e9fa427d43c7d7100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-13T07:13:38.904 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:38 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:38.585+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:38.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.3.service' 2024-06-13T07:13:39.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:39.075 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-13T07:13:39.075 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-13T07:13:39.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.4 2024-06-13T07:13:39.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:38.809+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:39.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:38.780+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:39.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:39 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:39.117+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:39.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:39 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:39.117+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:39.367 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:39 smithi175 systemd[1]: Stopping Ceph osd.4 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:39.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:39 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:38.999+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:39.867 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:39 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:39.404+0000 7f8928d70700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:39.867 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:39 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:39.404+0000 7f8928d70700 -1 osd.4 102 *** Got signal Terminated *** 2024-06-13T07:13:39.868 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:39 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:39.404+0000 7f8928d70700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:39.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:39 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:39.555+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:40.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:39.821+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:40.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:39.811+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:40.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:39 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:39.965+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:40.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:40 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:40.083+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:40.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:40 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:40.083+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:40.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:40 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:40.529+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:41.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:40.810+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:41.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:40.861+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:41.117 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:40 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:40.751+0000 7f892135c700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:16.035744+0000 front 2024-06-13T07:13:16.035786+0000 (oldest deadline 2024-06-13T07:13:40.135966+0000) 2024-06-13T07:13:41.117 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:40 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:40.975+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:41.118 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:40 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:40.975+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:41.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:41 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:41.124+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:41.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:41 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:41.124+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:41.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:41 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:41.503+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:41.968 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:41 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:41.775+0000 7f892135c700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:16.035744+0000 front 2024-06-13T07:13:16.035786+0000 (oldest deadline 2024-06-13T07:13:40.135966+0000) 2024-06-13T07:13:42.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:41.831+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:42.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:41.830+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:42.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:41.860+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:42.224 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:41 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:41.968+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:42.224 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:41 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:41.968+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:42.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:42 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:42.151+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:42.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:42 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:42.151+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:42.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:42 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:42.492+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:43.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:42.858+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:43.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:42.906+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:43.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:42.835+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:43.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:42.835+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:43.117 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:42 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:42.782+0000 7f892135c700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:16.035744+0000 front 2024-06-13T07:13:16.035786+0000 (oldest deadline 2024-06-13T07:13:40.135966+0000) 2024-06-13T07:13:43.117 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:42 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:42.974+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:43.118 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:42 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:42.974+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:43.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:43 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:43.103+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:43.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:43 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:43.103+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:43.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:43.287+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:43.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:43 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:43.459+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:43.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:43 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:43.459+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:44.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:43.819+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:44.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:43.881+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:44.087 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:43.881+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:44.088 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:43.885+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:44.117 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4[47941]: 2024-06-13T07:13:43.749+0000 7f892135c700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:16.035744+0000 front 2024-06-13T07:13:16.035786+0000 (oldest deadline 2024-06-13T07:13:40.135966+0000) 2024-06-13T07:13:44.118 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:43.939+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:44.118 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:43 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:43.939+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:44.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:44 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:44.141+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:44.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:44 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:44.141+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:44.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:44 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:44.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:44.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:44 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:44.416+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:44.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:44 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:44.416+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:45.335 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:45 smithi175 podman[73567]: 2024-06-13 07:13:45.097744191 +0000 UTC m=+5.824145148 container died 298f66c6cc65ea1ec4dc044f84c1ea8861c36705dc75c46abc91157fb1eacbed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:45.336 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:44 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:44.890+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:45.336 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:44 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:44.890+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:45.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:44.882+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:44.857+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:45.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:44.884+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:45.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:44.884+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:45.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:45 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:45.376+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:45.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:45 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:45.376+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:45.414 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:45 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:45.155+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:45.414 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:45 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:45.155+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:45.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:45 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:45.336+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:45.618 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:45 smithi175 podman[73567]: 2024-06-13 07:13:45.480340949 +0000 UTC m=+6.206741907 container cleanup 298f66c6cc65ea1ec4dc044f84c1ea8861c36705dc75c46abc91157fb1eacbed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-13T07:13:45.618 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:45 smithi175 bash[73567]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4 2024-06-13T07:13:46.061 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:45 smithi175 podman[73587]: 2024-06-13 07:13:45.74880942 +0000 UTC m=+0.653635858 container remove 298f66c6cc65ea1ec4dc044f84c1ea8861c36705dc75c46abc91157fb1eacbed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True) 2024-06-13T07:13:46.061 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:45 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:45.914+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:46.062 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:45 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:45.914+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:46.335 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:46 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:46.303+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:46.335 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73675]: 2024-06-13 07:13:45.964909905 +0000 UTC m=+0.016803951 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:46.335 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73675]: 2024-06-13 07:13:46.099774065 +0000 UTC m=+0.151668094 container create 025988e99f67153c03ba43b5e987efa4c3ceba53d6b1fb8e7090f3ff5f191144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-13T07:13:46.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:45 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:45.867+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:46.336 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:45 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:45.875+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:46.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:45 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:45.906+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:46.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:45 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:45.906+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:46.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:46 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:46.365+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:46.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:46 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:46.365+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:46.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:46 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:46.127+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:46.413 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:46 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:46.127+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:46.618 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73675]: 2024-06-13 07:13:46.336168445 +0000 UTC m=+0.388062491 container init 025988e99f67153c03ba43b5e987efa4c3ceba53d6b1fb8e7090f3ff5f191144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-13T07:13:46.618 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73675]: 2024-06-13 07:13:46.339395219 +0000 UTC m=+0.391289250 container start 025988e99f67153c03ba43b5e987efa4c3ceba53d6b1fb8e7090f3ff5f191144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-13T07:13:46.618 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73675]: 2024-06-13 07:13:46.43588638 +0000 UTC m=+0.487780412 container attach 025988e99f67153c03ba43b5e987efa4c3ceba53d6b1fb8e7090f3ff5f191144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-13T07:13:46.900 INFO:journalctl@ceph.osd.4.smithi175.stdout:Jun 13 07:13:46 smithi175 podman[73701]: 2024-06-13 07:13:46.659712088 +0000 UTC m=+0.018133426 container died 025988e99f67153c03ba43b5e987efa4c3ceba53d6b1fb8e7090f3ff5f191144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:47.213 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:46 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:46.900+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:47.213 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:46 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:46.900+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:47.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.4.service' 2024-06-13T07:13:47.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:47.279 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-13T07:13:47.279 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-13T07:13:47.279 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.5 2024-06-13T07:13:47.306 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:46 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:46.853+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:47.306 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:46 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:46.887+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:47.307 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:46 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:46.941+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:47.307 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:46 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:46.941+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:47.371 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:47 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:47.094+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:47.371 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:47 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:47.094+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:47.586 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 systemd[1]: Stopping Ceph osd.5 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:47.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:47 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:47.345+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:47.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:47 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:47.371+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:47.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:47 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:47.371+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:47.836 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:47.586+0000 7f01c956b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:47.836 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:47.586+0000 7f01c956b700 -1 osd.5 102 *** Got signal Terminated *** 2024-06-13T07:13:47.836 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:47.586+0000 7f01c956b700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:48.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:47 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:47.924+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:48.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:47 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:47.924+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:48.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:47 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:47.924+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:48.336 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:47.981+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:48.336 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:47.981+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:48.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:47.887+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:48.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:47 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:47.899+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:48.352 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:48.083+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:48.353 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:48.083+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:48.353 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:48.083+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:48.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:48 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:48.297+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:48.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:48.352+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:48.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:48.352+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:49.273 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:48 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:48.972+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:49.273 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:48 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:48.972+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:49.273 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:48 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:48.972+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:49.338 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:48 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:48.910+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:49.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:48 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:48.925+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:49.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:48 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:48.968+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:49.339 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:48 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:48.968+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:49.375 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:49 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:49.101+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:49.375 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:49 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:49.101+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:49.375 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:49 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:49.101+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:49.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:49 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:49.273+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:49.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:49 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:49.273+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:49.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:49 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:49.374+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:49.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:49 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:49.374+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:50.276 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:49 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:49.981+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:50.276 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:49 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:49.981+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:50.276 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:49 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:49.981+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:50.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:49 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:49.888+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:50.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:49 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:49.951+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:50.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:49 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:49.969+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:50.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:49 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:49.969+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:50.338 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:49 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:49.969+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.995658+0000 front 2024-06-13T07:13:23.995846+0000 (oldest deadline 2024-06-13T07:13:49.895877+0000) 2024-06-13T07:13:50.391 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:50.056+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:50.391 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:50.056+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:50.391 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:50.056+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:50.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:50 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:50.276+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:50.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:50 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:50.276+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:50.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:50.391+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:50.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:50 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:50.391+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:51.238 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:50 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:50.897+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:51.238 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:50 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:50.928+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:51.238 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:50 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:50.966+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:51.238 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:50 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:50.966+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:51.238 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:50 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:50.966+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.995658+0000 front 2024-06-13T07:13:23.995846+0000 (oldest deadline 2024-06-13T07:13:49.895877+0000) 2024-06-13T07:13:51.291 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:51 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:51.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:51.291 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:51 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:51.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:51.291 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:51 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:51.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:51.377 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.021+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:51.377 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.021+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:51.377 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.021+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:51.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:51 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:51.291+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:51.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:51 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:51.291+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:51.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:51.377+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:51.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:51.377+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:51.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:51.377+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:52.319 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:52 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:52.025+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:52.319 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:52 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:52.025+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:52.319 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:52 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:52.025+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:52.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:51.879+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:52.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:51.879+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:52.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:51.954+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.292621+0000 front 2024-06-13T07:13:08.292704+0000 (oldest deadline 2024-06-13T07:13:32.893256+0000) 2024-06-13T07:13:52.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:51.955+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.594447+0000 front 2024-06-13T07:13:17.594545+0000 (oldest deadline 2024-06-13T07:13:42.294525+0000) 2024-06-13T07:13:52.337 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5[48171]: 2024-06-13T07:13:51.955+0000 7f01c1b57700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.995658+0000 front 2024-06-13T07:13:23.995846+0000 (oldest deadline 2024-06-13T07:13:49.895877+0000) 2024-06-13T07:13:52.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:51.884+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:52.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:51 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:51.884+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:52.356 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.990+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:52.356 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.990+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:52.356 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:51 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:51.990+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:52.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:52 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:52.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:52.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:52 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:52.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:52.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:52.356+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:52.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:52.356+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:52.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:52.356+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:53.218 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:52 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:52.893+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:53.218 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:52 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:52.893+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:53.218 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:52 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:52.866+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:53.218 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:52 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:52.866+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:53.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:53 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:53.281+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:53.368 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:53 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:53.281+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:53.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:53 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:53.064+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:53.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:53 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:53.064+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:53.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:53 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:53.064+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:53.406 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:52.979+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:53.406 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:52.979+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:53.406 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:52 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:52.979+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:53.536 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:53 smithi189 podman[74484]: 2024-06-13 07:13:53.219258678 +0000 UTC m=+5.740111086 container died c823c0ed8510532354ef4c0d6c457161e7acc3ec37ff9f08289ab3f652a42622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-13T07:13:53.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:53.406+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:53.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:53.406+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:53.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:53.406+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:53.836 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:53 smithi189 podman[74484]: 2024-06-13 07:13:53.628467021 +0000 UTC m=+6.149319429 container cleanup c823c0ed8510532354ef4c0d6c457161e7acc3ec37ff9f08289ab3f652a42622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:13:53.837 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:53 smithi189 bash[74484]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5 2024-06-13T07:13:54.247 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:53 smithi189 podman[74499]: 2024-06-13 07:13:53.83984237 +0000 UTC m=+0.624790818 container remove c823c0ed8510532354ef4c0d6c457161e7acc3ec37ff9f08289ab3f652a42622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-13T07:13:54.248 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:53 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:53.898+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:54.248 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:53 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:53.898+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:54.248 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:53 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:53.882+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:54.248 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:53 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:53.882+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:54.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:54 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:54.255+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:54.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:54 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:54.255+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:54.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:54 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:54.058+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:54.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:54 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:54.058+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:54.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:54 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:54.058+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:54.389 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:53.948+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:54.389 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:53.948+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:54.389 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:53 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:53.948+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:54.543 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.151212069 +0000 UTC m=+0.017216509 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:13:54.543 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.305643007 +0000 UTC m=+0.171647481 container create 03f4e179350cb7fdfd1555d26121c22174474d32f61edb999b5d877174e73a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-13T07:13:54.665 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:54.389+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:54.666 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:54.389+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:54.666 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:54.389+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:54.836 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.544112599 +0000 UTC m=+0.410117043 container init 03f4e179350cb7fdfd1555d26121c22174474d32f61edb999b5d877174e73a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-13T07:13:54.837 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.547597416 +0000 UTC m=+0.413601850 container start 03f4e179350cb7fdfd1555d26121c22174474d32f61edb999b5d877174e73a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-13T07:13:54.837 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.626970899 +0000 UTC m=+0.492975348 container attach 03f4e179350cb7fdfd1555d26121c22174474d32f61edb999b5d877174e73a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:13:55.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:54.907+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:55.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:54.907+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:55.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:54.907+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:55.179 INFO:journalctl@ceph.osd.5.smithi189.stdout:Jun 13 07:13:54 smithi189 podman[74591]: 2024-06-13 07:13:54.839098642 +0000 UTC m=+0.705103074 container died 03f4e179350cb7fdfd1555d26121c22174474d32f61edb999b5d877174e73a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-13T07:13:55.179 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:54 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:54.890+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:55.179 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:54 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:54.890+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:55.179 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:54 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:54.904+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:55.179 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:54 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:54.904+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:55.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:55.287+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:55.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:55.287+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:55.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:55.038+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:55.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:55.038+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:55.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:55 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:55.038+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:55.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.5.service' 2024-06-13T07:13:55.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:13:55.534 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-13T07:13:55.534 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-13T07:13:55.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.6 2024-06-13T07:13:55.662 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:55.405+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:55.662 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:55.405+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:55.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:55.405+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:56.163 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:55 smithi007 systemd[1]: Stopping Ceph osd.6 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:13:56.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:55.890+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:56.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:55.890+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:56.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:55 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:55.890+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:56.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:55 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:55.886+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:56.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:55 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:55.886+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:56.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:55 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:55.856+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:56.338 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:55 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:55.856+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:56.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:56 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:56.312+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:56.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:56 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:56.312+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:56.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:56 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:56.074+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:56.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:56 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:56.074+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:56.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:56 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:56.074+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:56.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.239+0000 7fef8ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:13:56.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.239+0000 7fef8ac00700 -1 osd.6 102 *** Got signal Terminated *** 2024-06-13T07:13:56.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.239+0000 7fef8ac00700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:13:56.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.434+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:56.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.434+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:56.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:56.434+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:57.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:56.861+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:57.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:56.861+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:57.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:56.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:57.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:56.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:57.088 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:56 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:56.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:13:57.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:56.865+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:57.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:56.865+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:57.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:56.865+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:57.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:56 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:56.865+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:13:57.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:57.314+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:57.314+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:57.314+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:57.114+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:57.114+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:57.114+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:57.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:57 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:57.114+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:13:57.891 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:57.460+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:57.891 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:57.460+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:57.891 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:57.460+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:58.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:57.830+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:58.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:57.830+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:58.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:57.830+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:13:58.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:57.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:58.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:57.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:58.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:57 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:57.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:13:58.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:57.891+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:58.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:57.891+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:58.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:57.891+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:58.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:57 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:57.891+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:13:58.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:58.323+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:58.323+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:58.323+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:58.087+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:58.087+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:58.087+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:58.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:58 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:58.087+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:13:58.862 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:58.471+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:58.863 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:58.471+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:58.863 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:58.471+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:59.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:58.862+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:13:59.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:58.862+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:13:59.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:58.862+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:13:59.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:58 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:58.862+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:58.868+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:58.868+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:58.868+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:58.883+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:58.883+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:13:59.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:58 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:58.883+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:13:59.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:59.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:13:59.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:59.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:13:59.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:13:59.319+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:13:59.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:59.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:13:59.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:59.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:13:59.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:59.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:13:59.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:13:59 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:13:59.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:13:59.848 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:59.438+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:13:59.848 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:59.438+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:13:59.848 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:59.438+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:13:59.848 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:13:59.438+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:34.205492+0000 front 2024-06-13T07:13:34.205556+0000 (oldest deadline 2024-06-13T07:13:58.906068+0000) 2024-06-13T07:14:00.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:59.859+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:00.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:59.859+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:00.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:13:59.859+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:00.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:59.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:00.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:59.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:00.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:13:59 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:13:59.834+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:00.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:59.847+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:00.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:59.847+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:00.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:59.847+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:00.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:13:59 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:13:59.847+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:00.617 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:00.342+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:00.342+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:00.342+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:00.135+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:00.135+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:00.135+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:00.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:00 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:00.135+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:00.663 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:14:00.390+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:05.793618+0000 front 2024-06-13T07:13:05.793768+0000 (oldest deadline 2024-06-13T07:13:29.293837+0000) 2024-06-13T07:14:00.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:14:00.390+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:15.096926+0000 front 2024-06-13T07:13:15.096991+0000 (oldest deadline 2024-06-13T07:13:43.297702+0000) 2024-06-13T07:14:00.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:14:00.390+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.600393+0000 front 2024-06-13T07:13:25.600273+0000 (oldest deadline 2024-06-13T07:13:50.900435+0000) 2024-06-13T07:14:00.664 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6[57804]: 2024-06-13T07:14:00.390+0000 7fef86400700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:34.205492+0000 front 2024-06-13T07:13:34.205556+0000 (oldest deadline 2024-06-13T07:13:58.906068+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:00.846+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:00.846+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:00.846+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:00.811+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:00.811+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:01.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:00 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:00.811+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:01.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:00.837+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:01.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:00.837+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:01.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:00.837+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:01.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:00.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:01.519 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:01.353+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:01.519 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:01.353+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:01.519 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:01.353+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:01.520 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:01.134+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:01.520 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:01.134+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:01.520 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:01.134+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:01.520 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:01 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:01.134+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:02.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:01.866+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:02.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:01.866+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:02.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:01.866+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:02.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:01.833+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:02.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:01.833+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:02.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:01 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:01.833+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:02.131 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:01 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:01.824+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:02.131 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:01 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:01.824+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:02.131 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:01 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:01.824+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:02.131 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:01 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:01.824+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:02.337 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:02.086+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:02.338 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:02.086+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:02.338 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:02.086+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:02.338 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:02.086+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:02.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:02 smithi007 podman[114934]: 2024-06-13 07:14:02.131788765 +0000 UTC m=+6.048237441 container died cc31d7521250cb5bf0c075354ec511b95dc428c79c0da393e0391315df2a51bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-13T07:14:02.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:02.338+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:02.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:02.338+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:02.618 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:02 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:02.338+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:02.679 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:02 smithi007 podman[114934]: 2024-06-13 07:14:02.655972151 +0000 UTC m=+6.572420800 container cleanup cc31d7521250cb5bf0c075354ec511b95dc428c79c0da393e0391315df2a51bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:02.679 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:02 smithi007 bash[114934]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6 2024-06-13T07:14:03.164 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:02 smithi007 podman[114954]: 2024-06-13 07:14:02.957982532 +0000 UTC m=+0.817759474 container remove cc31d7521250cb5bf0c075354ec511b95dc428c79c0da393e0391315df2a51bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:03.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:02 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:02.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:03.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:02 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:02.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:03.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:02 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:02.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:03.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:02 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:02.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:03.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:02.903+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:03.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:02.903+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:03.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:02.903+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:03.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:02.859+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:03.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:02.859+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:03.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:02 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:02.859+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:03.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:03.352+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:03.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:03.352+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:03.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:03.352+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:03.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:03.103+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:03.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:03.103+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:03.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:03.103+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:03.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:03 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:03.103+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:04.145 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:03.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:03.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:03.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:03 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:03.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:03 smithi007 podman[115039]: 2024-06-13 07:14:03.720504983 +0000 UTC m=+0.034274710 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:03 smithi007 podman[115039]: 2024-06-13 07:14:03.850439435 +0000 UTC m=+0.164209190 container create 49994ad158869374ba9939dbe457b982196a7f20ac3b855f16c948bb55c50b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-13T07:14:04.146 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:04 smithi007 podman[115039]: 2024-06-13 07:14:04.145863684 +0000 UTC m=+0.459633445 container init 49994ad158869374ba9939dbe457b982196a7f20ac3b855f16c948bb55c50b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-13T07:14:04.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:03.910+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:04.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:03.910+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:04.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:03.910+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:04.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:03.848+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:04.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:03.848+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:04.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:03 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:03.848+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:04.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:04.078+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:04.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:04.078+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:04.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:04.078+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:04.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:04.078+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:04.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:04 smithi007 podman[115039]: 2024-06-13 07:14:04.158675819 +0000 UTC m=+0.472445547 container start 49994ad158869374ba9939dbe457b982196a7f20ac3b855f16c948bb55c50b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:04.413 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:04 smithi007 podman[115039]: 2024-06-13 07:14:04.244472399 +0000 UTC m=+0.558242190 container attach 49994ad158869374ba9939dbe457b982196a7f20ac3b855f16c948bb55c50b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-13T07:14:04.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:04.395+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:04.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:04.395+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:04.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:04 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:04.395+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:04.913 INFO:journalctl@ceph.osd.6.smithi007.stdout:Jun 13 07:14:04 smithi007 podman[115039]: 2024-06-13 07:14:04.624110883 +0000 UTC m=+0.937880645 container died 49994ad158869374ba9939dbe457b982196a7f20ac3b855f16c948bb55c50b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-13T07:14:04.913 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:04 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:04.819+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:04.914 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:04 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:04.819+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:04.914 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:04 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:04.819+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:04.914 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:04 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:04.819+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:04.960+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:04.960+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:04.960+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:04.887+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:04.887+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:05.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:04 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:04.887+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:05.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.6.service' 2024-06-13T07:14:05.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:05.060+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:05.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:05.060+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:05.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:05.060+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:05.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:05.060+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:05.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:05.458 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-13T07:14:05.458 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-13T07:14:05.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.7 2024-06-13T07:14:05.765 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.409+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:05.765 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.409+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:05.765 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.409+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:05.765 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 systemd[1]: Stopping Ceph osd.7 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:05.766 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.765+0000 7fa2e7564700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:05.766 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.765+0000 7fa2e7564700 -1 osd.7 102 *** Got signal Terminated *** 2024-06-13T07:14:05.766 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:05 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:05.765+0000 7fa2e7564700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:14:06.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:05.863+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:06.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:05.863+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:06.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:05.863+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:06.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:05.863+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:05.942+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:05.942+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:05.942+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:05.852+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:05.852+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:06.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:05 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:05.852+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:06.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:06.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:06.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:06.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:06.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:06.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:06.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:06.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:06.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:06.424+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:06.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:06.424+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:06.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:06 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:06.424+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:06.831+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:06.831+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:06.831+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:06.937+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:06.937+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:06.937+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:07.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:06 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:06.937+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:07.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:06.834+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:07.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:06.834+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:07.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:06.834+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:07.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:06 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:06.834+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:07.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:07.061+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:07.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:07.061+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:07.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:07.061+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:07.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:07.061+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:07.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:07.433+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:07.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:07.433+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:07.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:07 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:07.433+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:08.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:07.951+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:07.951+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:07.951+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:07.951+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:07.799+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:07.799+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:07.799+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:08.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:07 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:07.799+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:08.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:07.853+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:08.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:07.853+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:08.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:07.853+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:08.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:07.853+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:08.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:07 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:07.853+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:08.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:08.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:08.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:08.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:08.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:08.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:08.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:08.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:08.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:08.453+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:08.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:08.453+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:08.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:08 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:08.453+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:09.086 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:08.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:09.151 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:08.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:09.151 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:08.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:09.151 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:08.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:09.152 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:08.757+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:09.152 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:08.757+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:09.152 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:08.757+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:09.152 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:08 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:08.757+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:09.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:08.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:09.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:08.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:09.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:08.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:09.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:08.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:09.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:08 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:08.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:09.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:09.023+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:09.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:09.023+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:09.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:09.023+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:09.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:09.023+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:09.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:09.468+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:09.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:09.468+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:09.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:09 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:09.468+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:10.011 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:09 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:09.711+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:10.011 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:09 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:09.711+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:10.011 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:09 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:09.711+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:10.011 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:09 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:09.711+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:10.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:09.877+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:10.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:09.877+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:10.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:09.877+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:10.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:09.877+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:10.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:09 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:09.877+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:10.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.011+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:10.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.011+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:10.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.011+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:10.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.011+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:10.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:10.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:10.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:10.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:10.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:10.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:10.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:10.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:10.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:10.014+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:10.867 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:10.425+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.307659+0000 front 2024-06-13T07:13:17.307631+0000 (oldest deadline 2024-06-13T07:13:43.207779+0000) 2024-06-13T07:14:10.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:10.425+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:23.208219+0000 front 2024-06-13T07:13:23.208267+0000 (oldest deadline 2024-06-13T07:13:48.508283+0000) 2024-06-13T07:14:10.868 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7[54740]: 2024-06-13T07:14:10.425+0000 7fa2dfb50700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.409641+0000 front 2024-06-13T07:13:31.409487+0000 (oldest deadline 2024-06-13T07:13:57.309316+0000) 2024-06-13T07:14:10.962 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:10.679+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:10.962 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:10.679+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:10.963 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:10.679+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:10.963 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:10.679+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:11.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:10.872+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:11.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:10.872+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:11.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:10.872+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:11.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:10.872+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:11.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:10 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:10.872+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:11.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.963+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:11.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.963+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:11.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.963+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:11.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:10 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:10.963+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:11.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:11 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:11.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:11.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:11 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:11.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:11.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:11 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:11.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:11.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:11 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:11.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:11.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:11 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:11.016+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:11.731 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:11 smithi175 podman[73791]: 2024-06-13 07:14:11.47292784 +0000 UTC m=+5.811639148 container died 14a1204d1ae967d5261c859502ea311b5d443f1fc22920fa58a0f05e2ac2e037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-13T07:14:11.990 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:11.714+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:11.991 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:11.714+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:11.991 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:11.714+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:11.991 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:11.714+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:12.021 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:11 smithi175 podman[73791]: 2024-06-13 07:14:11.8315397 +0000 UTC m=+6.170250995 container cleanup 14a1204d1ae967d5261c859502ea311b5d443f1fc22920fa58a0f05e2ac2e037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:14:12.021 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:11 smithi175 bash[73791]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7 2024-06-13T07:14:12.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:11.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:12.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:11.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:12.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:11.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:12.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:11.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:12.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:11 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:11.839+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:12.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:11.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:12.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:11.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:12.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:11.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:12.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:11 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:11.991+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:12.367 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73808]: 2024-06-13 07:14:12.099583322 +0000 UTC m=+0.626647607 container remove 14a1204d1ae967d5261c859502ea311b5d443f1fc22920fa58a0f05e2ac2e037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-13T07:14:12.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.021+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:12.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.021+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:12.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.021+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:12.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.021+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:12.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.021+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:12.694 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.343050918 +0000 UTC m=+0.017008560 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:12.694 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.467229584 +0000 UTC m=+0.141187245 container create 2b25f4aa907db438773c35912c8e8e2f08eebe5361fd8fcefc3a4352960375fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:12.694 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.695095157 +0000 UTC m=+0.369052796 container init 2b25f4aa907db438773c35912c8e8e2f08eebe5361fd8fcefc3a4352960375fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:14:12.991 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.69900724 +0000 UTC m=+0.372964860 container start 2b25f4aa907db438773c35912c8e8e2f08eebe5361fd8fcefc3a4352960375fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-13T07:14:12.991 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.786377976 +0000 UTC m=+0.460335602 container attach 2b25f4aa907db438773c35912c8e8e2f08eebe5361fd8fcefc3a4352960375fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-13T07:14:13.018 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:12 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:12.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:13.018 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:12 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:12.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:13.018 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:12 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:12.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:13.018 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:12 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:12.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:13.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:12 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:12.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:13.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:12 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:12.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:13.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:12 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:12.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:13.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:12 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:12.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:13.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:12 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:12.832+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:13.296 INFO:journalctl@ceph.osd.7.smithi175.stdout:Jun 13 07:14:12 smithi175 podman[73895]: 2024-06-13 07:14:12.992591851 +0000 UTC m=+0.666549468 container died 2b25f4aa907db438773c35912c8e8e2f08eebe5361fd8fcefc3a4352960375fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-13T07:14:13.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.998+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:13.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.998+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:13.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.998+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:13.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.998+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:13.297 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:12 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:12.998+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:13.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.018+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:13.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.018+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:13.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.018+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:13.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.018+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:13.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.7.service' 2024-06-13T07:14:13.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:13.667 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-13T07:14:13.667 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-13T07:14:13.667 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.8 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 systemd[1]: Stopping Ceph osd.8 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:13.744+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:13.744+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:13.744+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:13.744+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:13.997 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:13.744+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:14.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:13 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:13.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:14.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:13 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:13.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:14.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:13 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:13.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:14.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:13 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:13.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:14.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:13 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:13.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:14.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.997+0000 7fd3de8ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:14.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.997+0000 7fd3de8ba700 -1 osd.8 102 *** Got signal Terminated *** 2024-06-13T07:14:14.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:13 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:13.997+0000 7fd3de8ba700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:14:14.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:14.010+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:14.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:14.010+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:14.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:14.010+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:14.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:14.010+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:14.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:14 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:14.042+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:14.369 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:14 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:14.042+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:14.369 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:14 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:14.042+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:14.369 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:14 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:14.042+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:14.369 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:14 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:14.042+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:15.052 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:14.742+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:15.053 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:14.742+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:15.053 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:14.742+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:15.053 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:14.742+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:15.053 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:14 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:14.742+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:15.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:15.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:15.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:15.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:15.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:15.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:14 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:14.864+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:15.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:15.052+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:15.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:15.052+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:15.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:15.052+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:15.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:15.052+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:15.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:15.052+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:48.858418+0000 front 2024-06-13T07:13:48.858477+0000 (oldest deadline 2024-06-13T07:14:14.158547+0000) 2024-06-13T07:14:15.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:15.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:15.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:15.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:15.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:15.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:15 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:15.039+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:16.029 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:15.725+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:16.030 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:15.725+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:16.030 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:15.725+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:16.030 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:15.725+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:16.030 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:15 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:15.725+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:16.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:16.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:16.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:16.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:16.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:16.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:15 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:15.829+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:16.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:16.030+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:16.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:16.030+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:16.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:16.030+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:16.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:16.030+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:16.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:16.030+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:48.858418+0000 front 2024-06-13T07:13:48.858477+0000 (oldest deadline 2024-06-13T07:14:14.158547+0000) 2024-06-13T07:14:16.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:16.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:16.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:16.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:16.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:16.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:16 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:16.080+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:17.049 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:16.717+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:17.049 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:16.717+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:17.049 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:16.717+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:17.049 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:16.717+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:17.049 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:16 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:16.717+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:17.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:17.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:17.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:17.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:17.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:17.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:16 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:16.816+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:17.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:17.049+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:17.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:17.049+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:17.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:17.049+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:17.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:17.049+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:17.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:17.049+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:48.858418+0000 front 2024-06-13T07:13:48.858477+0000 (oldest deadline 2024-06-13T07:14:14.158547+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:17.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:17 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:17.045+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:18.019 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:17.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:18.019 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:17.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:18.019 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:17.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:18.019 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:17.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:18.019 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:17 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:17.758+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:18.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:18.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:18.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:18.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:18.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:18.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:17 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:17.838+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:18.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.019+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:18.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.019+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:18.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.019+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:18.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.019+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:18.337 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.019+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:48.858418+0000 front 2024-06-13T07:13:48.858477+0000 (oldest deadline 2024-06-13T07:14:14.158547+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:18.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:18.075+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.985+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.551076+0000 front 2024-06-13T07:13:17.551219+0000 (oldest deadline 2024-06-13T07:13:40.451281+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.985+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.553584+0000 front 2024-06-13T07:13:25.553708+0000 (oldest deadline 2024-06-13T07:13:51.453819+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.985+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.454697+0000 front 2024-06-13T07:13:31.454572+0000 (oldest deadline 2024-06-13T07:13:57.354593+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.985+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:40.657557+0000 front 2024-06-13T07:13:40.657527+0000 (oldest deadline 2024-06-13T07:14:06.557509+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8[54720]: 2024-06-13T07:14:18.985+0000 7fd3daeae700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:48.858418+0000 front 2024-06-13T07:13:48.858477+0000 (oldest deadline 2024-06-13T07:14:14.158547+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:18.772+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:18.772+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:18.772+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:19.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:18.772+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:19.088 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:18 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:18.772+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:19.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:19.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:19.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:19.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:19.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:19.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:18 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:18.840+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:19.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:19.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:19.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:19.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:19.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:19.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:19 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:19.044+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:19.797 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:19 smithi189 podman[74695]: 2024-06-13 07:14:19.512463428 +0000 UTC m=+5.647029958 container died 18dcbe4c70ebc1c26dd4a7562da146325e004a578fe8a53397a1cc60271a4e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:19 smithi189 podman[74695]: 2024-06-13 07:14:19.822372208 +0000 UTC m=+5.956938786 container cleanup 18dcbe4c70ebc1c26dd4a7562da146325e004a578fe8a53397a1cc60271a4e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:19 smithi189 bash[74695]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:19.797+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:19.797+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:19.797+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:20.081 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:19.797+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:20.082 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:19 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:19.797+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:20.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:20.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:20.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:20.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:20.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:20.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:19 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:19.806+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:20.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74714]: 2024-06-13 07:14:20.082152159 +0000 UTC m=+0.573648664 container remove 18dcbe4c70ebc1c26dd4a7562da146325e004a578fe8a53397a1cc60271a4e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:14:20.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:20.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:20.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:20.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:20.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:20.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:20 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:20.062+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:20.761 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74799]: 2024-06-13 07:14:20.408463341 +0000 UTC m=+0.017572381 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:20.761 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74799]: 2024-06-13 07:14:20.524761039 +0000 UTC m=+0.133870052 container create 1f2bdc73fe74c7dacf6b366f07b02f7351b400f776d0cb897f99a51031247a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-13T07:14:20.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:20.761+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:20.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:20.761+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:20.761+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:20.761+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:20 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:20.761+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74799]: 2024-06-13 07:14:20.768684759 +0000 UTC m=+0.377793778 container init 1f2bdc73fe74c7dacf6b366f07b02f7351b400f776d0cb897f99a51031247a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74799]: 2024-06-13 07:14:20.77189218 +0000 UTC m=+0.381001198 container start 1f2bdc73fe74c7dacf6b366f07b02f7351b400f776d0cb897f99a51031247a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:21.061 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:20 smithi189 podman[74799]: 2024-06-13 07:14:20.877211403 +0000 UTC m=+0.486320418 container attach 1f2bdc73fe74c7dacf6b366f07b02f7351b400f776d0cb897f99a51031247a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-13T07:14:21.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:21.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:21.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:21.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:21.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:21.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:20 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:20.833+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:21.336 INFO:journalctl@ceph.osd.8.smithi189.stdout:Jun 13 07:14:21 smithi189 podman[74799]: 2024-06-13 07:14:21.062410676 +0000 UTC m=+0.671519695 container died 1f2bdc73fe74c7dacf6b366f07b02f7351b400f776d0cb897f99a51031247a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:14:21.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:21.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:21.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:21.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:21.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:21.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:21.105+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:21.752 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.8.service' 2024-06-13T07:14:21.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:21.789 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-13T07:14:21.789 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-13T07:14:21.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.9 2024-06-13T07:14:22.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:21.747+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:22.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:21.747+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:22.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:21.747+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:22.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:21.747+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:22.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:21 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:21.747+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:21.860+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:22.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:21 smithi007 systemd[1]: Stopping Ceph osd.9 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:22.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:22.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:22.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:22.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:22.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:22.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:22 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:22.149+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:22.831 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.558+0000 7f9a09000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.558+0000 7f9a09000700 -1 osd.9 102 *** Got signal Terminated *** 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.558+0000 7f9a09000700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:22.832 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:23.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:22.768+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:23.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:22.768+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:23.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:22.768+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:23.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:22.768+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:23.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:22 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:22.768+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:23.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:23.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:22 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:22.831+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:23.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:23.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:23.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:23.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:23.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:23.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:23 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:23.136+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:24.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:23.795+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:24.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:23.795+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:24.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:23.795+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:24.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:23.795+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:24.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:23.795+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:24.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:24.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:24.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:24.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:24.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:24.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:23 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:23.811+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:24.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:24.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:24.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:24.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:24.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:24.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:24 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:24.147+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:25.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:24.782+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:25.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:24.782+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:25.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:24.782+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:25.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:24.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:25.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:24 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:24.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:25.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:25.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:25.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:25.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:25.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:25.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:24 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:24.795+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:25.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:25.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:25.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:25.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:25.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:25.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:25 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:25.160+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:26.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:25.756+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:26.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:25.756+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:26.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:25.756+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:26.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:25.756+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:26.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:25 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:25.756+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:26.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.751+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:26.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.751+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:26.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.751+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:26.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.751+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:26.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.752+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:26.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:25 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:25.752+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:26.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:26.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:26 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:26.192+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:27.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:27.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:27.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:27.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:27.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:27.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:26 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:26.712+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:27.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:03.263348+0000 front 2024-06-13T07:13:03.263278+0000 (oldest deadline 2024-06-13T07:13:32.665003+0000) 2024-06-13T07:14:27.163 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:17.269670+0000 front 2024-06-13T07:13:17.269771+0000 (oldest deadline 2024-06-13T07:13:38.969906+0000) 2024-06-13T07:14:27.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:24.273451+0000 front 2024-06-13T07:13:24.273268+0000 (oldest deadline 2024-06-13T07:13:47.773490+0000) 2024-06-13T07:14:27.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:31.177638+0000 front 2024-06-13T07:13:31.177706+0000 (oldest deadline 2024-06-13T07:13:56.478001+0000) 2024-06-13T07:14:27.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.382241+0000 front 2024-06-13T07:13:43.382136+0000 (oldest deadline 2024-06-13T07:14:07.482192+0000) 2024-06-13T07:14:27.164 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:26 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9[64387]: 2024-06-13T07:14:26.727+0000 7f99ff800700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:50.984762+0000 front 2024-06-13T07:13:50.984495+0000 (oldest deadline 2024-06-13T07:14:14.484229+0000) 2024-06-13T07:14:27.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:27.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:27.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:27.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:27.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:27.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:27.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:27 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:27.150+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:28.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:28.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:28.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:28.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:28.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:28.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:27 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:27.710+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:28.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:28 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:28.173+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:28.913 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:28 smithi007 podman[115144]: 2024-06-13 07:14:28.651821751 +0000 UTC m=+6.360698149 container died e06cad935ca191809df7fccc04b6946464a8b2e9d154b4165dcf63309d970d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-06-13T07:14:29.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:29.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:29.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:29.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:29.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:29.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:28 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:28.702+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:29.268 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:29 smithi007 podman[115144]: 2024-06-13 07:14:29.099456613 +0000 UTC m=+6.808333005 container cleanup e06cad935ca191809df7fccc04b6946464a8b2e9d154b4165dcf63309d970d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:14:29.268 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:29 smithi007 bash[115144]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9 2024-06-13T07:14:29.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:29.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:29 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:29.169+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:29.662 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:29 smithi007 podman[115164]: 2024-06-13 07:14:29.267060606 +0000 UTC m=+0.592767683 container remove e06cad935ca191809df7fccc04b6946464a8b2e9d154b4165dcf63309d970d1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-13T07:14:30.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:30.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:30.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:30.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:30.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:30.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:29.736+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:30.414 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.060463769 +0000 UTC m=+0.035260255 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:30.414 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.1929359 +0000 UTC m=+0.167732429 container create db44529657edd352ab75cc7b0e1ee32a4e0fa2b025a380c370a999ab279c90f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:14:30.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:30.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:30.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:30.617 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:30.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:30.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:30.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:30 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:30.141+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:30.905 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.421451056 +0000 UTC m=+0.396247622 container init db44529657edd352ab75cc7b0e1ee32a4e0fa2b025a380c370a999ab279c90f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-13T07:14:30.906 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.433184065 +0000 UTC m=+0.407980566 container start db44529657edd352ab75cc7b0e1ee32a4e0fa2b025a380c370a999ab279c90f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-13T07:14:30.906 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.511777528 +0000 UTC m=+0.486574114 container attach db44529657edd352ab75cc7b0e1ee32a4e0fa2b025a380c370a999ab279c90f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:30.906 INFO:journalctl@ceph.osd.9.smithi007.stdout:Jun 13 07:14:30 smithi007 podman[115253]: 2024-06-13 07:14:30.906014266 +0000 UTC m=+0.880810785 container died db44529657edd352ab75cc7b0e1ee32a4e0fa2b025a380c370a999ab279c90f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-13T07:14:31.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:31.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:31.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:31.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:31.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:31.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:30 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:30.763+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:31.519 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:31.158+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:31.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.9.service' 2024-06-13T07:14:31.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:31.768 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-13T07:14:31.769 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-13T07:14:31.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.10 2024-06-13T07:14:32.057 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:31 smithi175 systemd[1]: Stopping Ceph osd.10 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:32.057 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.057+0000 7f542c37d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:32.057 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.057+0000 7f542c37d700 -1 osd.10 102 *** Got signal Terminated *** 2024-06-13T07:14:32.057 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.057+0000 7f542c37d700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:14:32.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:32.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:32.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:32.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:32.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:32.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:31 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:31.730+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:32.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:32.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:32.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:32.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:32.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:32.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:32.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:32 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:32.131+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:33.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:33.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:32 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:32.734+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:33.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:33.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:33 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:33.117+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:34.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:34.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:33.728+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:34.618 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:34 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:34.120+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:35.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:35.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:34 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:34.720+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:35.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:35.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:35 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:35.079+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:36.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:36.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:35 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:35.753+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:36.367 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:04.888460+0000 front 2024-06-13T07:13:04.888895+0000 (oldest deadline 2024-06-13T07:13:30.188462+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6804 osd.1 since back 2024-06-13T07:13:14.790075+0000 front 2024-06-13T07:13:14.790101+0000 (oldest deadline 2024-06-13T07:13:40.090240+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.891967+0000 front 2024-06-13T07:13:25.891896+0000 (oldest deadline 2024-06-13T07:13:46.991958+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:30.993556+0000 front 2024-06-13T07:13:30.993818+0000 (oldest deadline 2024-06-13T07:13:56.193978+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:43.696187+0000 front 2024-06-13T07:13:43.696232+0000 (oldest deadline 2024-06-13T07:14:09.596336+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:49.596926+0000 front 2024-06-13T07:13:49.596843+0000 (oldest deadline 2024-06-13T07:14:14.296896+0000) 2024-06-13T07:14:36.368 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:36 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10[61626]: 2024-06-13T07:14:36.111+0000 7f5424969700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:01.199163+0000 front 2024-06-13T07:14:01.199203+0000 (oldest deadline 2024-06-13T07:14:25.298754+0000) 2024-06-13T07:14:37.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:37.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:37.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:36 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:36.796+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:37.857 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:37 smithi175 podman[74001]: 2024-06-13 07:14:37.598499102 +0000 UTC m=+5.637154865 container died 30f3703a1fe70d2cfb437e4ebb722562416febc6fb3d34c1a50ae267b8b46cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-13T07:14:38.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:38.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:37.754+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:38.117 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:37 smithi175 podman[74001]: 2024-06-13 07:14:37.957185846 +0000 UTC m=+5.995841616 container cleanup 30f3703a1fe70d2cfb437e4ebb722562416febc6fb3d34c1a50ae267b8b46cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-13T07:14:38.117 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:37 smithi175 bash[74001]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10 2024-06-13T07:14:38.549 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74020]: 2024-06-13 07:14:38.216701209 +0000 UTC m=+0.614195033 container remove 30f3703a1fe70d2cfb437e4ebb722562416febc6fb3d34c1a50ae267b8b46cad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:38.867 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74106]: 2024-06-13 07:14:38.550705529 +0000 UTC m=+0.094568974 container create 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-13T07:14:38.867 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74106]: 2024-06-13 07:14:38.47303733 +0000 UTC m=+0.016900748 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:38.868 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74106]: 2024-06-13 07:14:38.744598198 +0000 UTC m=+0.288461614 container init 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:14:38.868 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74106]: 2024-06-13 07:14:38.747544288 +0000 UTC m=+0.291407692 container start 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-13T07:14:39.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:39.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:38 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:38.793+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:39.321 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:38 smithi175 podman[74106]: 2024-06-13 07:14:38.886016606 +0000 UTC m=+0.429880043 container attach 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-13T07:14:39.321 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:39 smithi175 podman[74106]: 2024-06-13 07:14:39.039470257 +0000 UTC m=+0.583333663 container died 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-13T07:14:39.578 INFO:journalctl@ceph.osd.10.smithi175.stdout:Jun 13 07:14:39 smithi175 podman[74132]: 2024-06-13 07:14:39.564594693 +0000 UTC m=+0.520218618 container remove 11e8162b368f65b247f767e41b73d9536d434b9c6efd459d92df618500d5638e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-13T07:14:39.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.10.service' 2024-06-13T07:14:39.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:39.620 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-13T07:14:39.620 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-13T07:14:39.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.11 2024-06-13T07:14:39.924 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 systemd[1]: Stopping Ceph osd.11 for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:39.924 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.798+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.924+0000 7faa83297700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.924+0000 7faa83297700 -1 osd.11 102 *** Got signal Terminated *** 2024-06-13T07:14:39.925 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:39 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:39.924+0000 7faa83297700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:41.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:40 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:40.783+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:42.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:41 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:41.803+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-06-13T07:14:17.720420+0000 front 2024-06-13T07:14:17.720444+0000 (oldest deadline 2024-06-13T07:14:41.220419+0000) 2024-06-13T07:14:43.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:43.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:42 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:42.830+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-06-13T07:14:17.720420+0000 front 2024-06-13T07:14:17.720444+0000 (oldest deadline 2024-06-13T07:14:41.220419+0000) 2024-06-13T07:14:44.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:44.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:43 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:43.822+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-06-13T07:14:17.720420+0000 front 2024-06-13T07:14:17.720444+0000 (oldest deadline 2024-06-13T07:14:41.220419+0000) 2024-06-13T07:14:45.336 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-06-13T07:13:08.107419+0000 front 2024-06-13T07:13:08.107054+0000 (oldest deadline 2024-06-13T07:13:33.406920+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-06-13T07:13:25.609817+0000 front 2024-06-13T07:13:25.609793+0000 (oldest deadline 2024-06-13T07:13:51.510041+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-06-13T07:13:33.111892+0000 front 2024-06-13T07:13:33.112022+0000 (oldest deadline 2024-06-13T07:13:56.512718+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6812 osd.4 since back 2024-06-13T07:13:42.914589+0000 front 2024-06-13T07:13:42.914573+0000 (oldest deadline 2024-06-13T07:14:07.515242+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-06-13T07:13:47.515693+0000 front 2024-06-13T07:13:47.515623+0000 (oldest deadline 2024-06-13T07:14:13.316485+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-06-13T07:14:00.118774+0000 front 2024-06-13T07:14:00.118905+0000 (oldest deadline 2024-06-13T07:14:26.018548+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.175:6820 osd.7 since back 2024-06-13T07:14:09.519364+0000 front 2024-06-13T07:14:09.519437+0000 (oldest deadline 2024-06-13T07:14:31.819485+0000) 2024-06-13T07:14:45.337 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:44 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11[61773]: 2024-06-13T07:14:44.862+0000 7faa7b883700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-06-13T07:14:17.720420+0000 front 2024-06-13T07:14:17.720444+0000 (oldest deadline 2024-06-13T07:14:41.220419+0000) 2024-06-13T07:14:45.774 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:45 smithi189 podman[74917]: 2024-06-13 07:14:45.502973826 +0000 UTC m=+5.686162791 container died a336930347a22995a526fa4e542d1607fc0671dee1fb48f40cc6086b7a82ac4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-06-13T07:14:46.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:45 smithi189 podman[74917]: 2024-06-13 07:14:45.866067057 +0000 UTC m=+6.049255978 container cleanup a336930347a22995a526fa4e542d1607fc0671dee1fb48f40cc6086b7a82ac4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-13T07:14:46.086 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:45 smithi189 bash[74917]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11 2024-06-13T07:14:46.481 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[74934]: 2024-06-13 07:14:46.117213184 +0000 UTC m=+0.607695233 container remove a336930347a22995a526fa4e542d1607fc0671dee1fb48f40cc6086b7a82ac4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True) 2024-06-13T07:14:46.769 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[75020]: 2024-06-13 07:14:46.385435427 +0000 UTC m=+0.017346320 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-13T07:14:46.769 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[75020]: 2024-06-13 07:14:46.484534067 +0000 UTC m=+0.116444959 container create cef0d841dbee2062fdf151c7ddfa106176598d952795e1fae29299b3a2425af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:14:46.769 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[75020]: 2024-06-13 07:14:46.703677387 +0000 UTC m=+0.335588280 container init cef0d841dbee2062fdf151c7ddfa106176598d952795e1fae29299b3a2425af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:14:46.769 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[75020]: 2024-06-13 07:14:46.706918779 +0000 UTC m=+0.338829690 container start cef0d841dbee2062fdf151c7ddfa106176598d952795e1fae29299b3a2425af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-13T07:14:46.769 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:46 smithi189 podman[75020]: 2024-06-13 07:14:46.770290338 +0000 UTC m=+0.402201239 container attach cef0d841dbee2062fdf151c7ddfa106176598d952795e1fae29299b3a2425af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:14:47.087 INFO:journalctl@ceph.osd.11.smithi189.stdout:Jun 13 07:14:47 smithi189 podman[75020]: 2024-06-13 07:14:47.014106883 +0000 UTC m=+0.646017785 container died cef0d841dbee2062fdf151c7ddfa106176598d952795e1fae29299b3a2425af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-osd-11-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-13T07:14:47.668 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@osd.11.service' 2024-06-13T07:14:47.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:47.707 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-13T07:14:47.707 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-13T07:14:47.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.a 2024-06-13T07:14:48.163 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:47 smithi007 systemd[1]: Stopping Ceph mds.a for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:48.663 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: 2024-06-13T07:14:48.305+0000 7fcf7c600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:48.663 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:48 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a[71282]: 2024-06-13T07:14:48.305+0000 7fcf7c600700 -1 mds.a *** got signal Terminated *** 2024-06-13T07:14:52.663 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:52 smithi007 podman[115370]: 2024-06-13 07:14:52.3297595 +0000 UTC m=+4.150552227 container died 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:14:53.109 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:52 smithi007 podman[115370]: 2024-06-13 07:14:52.882064864 +0000 UTC m=+4.702857603 container cleanup 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-13T07:14:53.109 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:52 smithi007 bash[115370]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a 2024-06-13T07:14:53.413 INFO:journalctl@ceph.mds.a.smithi007.stdout:Jun 13 07:14:53 smithi007 podman[115382]: 2024-06-13 07:14:53.108709591 +0000 UTC m=+0.765865469 container remove 37bb32521bb1bb8bf620fd7df0f9188569618c27e19a35bbac945dfb38a61e1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-13T07:14:53.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.a.service' 2024-06-13T07:14:53.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:53.701 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-13T07:14:53.701 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-13T07:14:53.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.d 2024-06-13T07:14:54.163 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:53 smithi007 systemd[1]: Stopping Ceph mds.d for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:14:54.663 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d[70084]: 2024-06-13T07:14:54.358+0000 7fb38ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:14:54.663 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:54 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d[70084]: 2024-06-13T07:14:54.358+0000 7fb38ae00700 -1 mds.d *** got signal Terminated *** 2024-06-13T07:14:58.324 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:58 smithi007 podman[115491]: 2024-06-13 07:14:58.065804205 +0000 UTC m=+3.870607948 container died 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-13T07:14:58.663 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:58 smithi007 podman[115491]: 2024-06-13 07:14:58.51096556 +0000 UTC m=+4.315769316 container cleanup 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:14:58.663 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:58 smithi007 bash[115491]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d 2024-06-13T07:14:59.163 INFO:journalctl@ceph.mds.d.smithi007.stdout:Jun 13 07:14:58 smithi007 podman[115504]: 2024-06-13 07:14:58.69556257 +0000 UTC m=+0.613092778 container remove 78b13c73d99766c742ed6f9cfb226c91025d6dc47a560ca4cd4ad02b1354136f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-d, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-13T07:14:59.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.d.service' 2024-06-13T07:14:59.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:14:59.383 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-13T07:14:59.383 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-13T07:14:59.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.g 2024-06-13T07:14:59.912 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:14:59 smithi007 systemd[1]: Stopping Ceph mds.g for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:00.413 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: 2024-06-13T07:15:00.003+0000 7f38d1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:00.413 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:00 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g[70867]: 2024-06-13T07:15:00.003+0000 7f38d1a00700 -1 mds.g *** got signal Terminated *** 2024-06-13T07:15:04.315 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:04 smithi007 podman[115612]: 2024-06-13 07:15:04.027943101 +0000 UTC m=+4.137272383 container died 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-13T07:15:04.663 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:04 smithi007 podman[115612]: 2024-06-13 07:15:04.514746455 +0000 UTC m=+4.624075730 container cleanup 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-13T07:15:04.663 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:04 smithi007 bash[115612]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g 2024-06-13T07:15:05.138 INFO:journalctl@ceph.mds.g.smithi007.stdout:Jun 13 07:15:04 smithi007 podman[115639]: 2024-06-13 07:15:04.816285863 +0000 UTC m=+0.772325848 container remove 5cef0ce4949f4406d79d57160b04a169473d2f91c9bd5c35c0b537e29d5b8d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-13T07:15:05.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.g.service' 2024-06-13T07:15:05.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:05.295 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-13T07:15:05.295 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-13T07:15:05.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.j 2024-06-13T07:15:05.910 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:05 smithi007 systemd[1]: Stopping Ceph mds.j for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:06.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j[69628]: 2024-06-13T07:15:05.910+0000 7f0049600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:06.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:05 smithi007 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j[69628]: 2024-06-13T07:15:05.910+0000 7f0049600700 -1 mds.j *** got signal Terminated *** 2024-06-13T07:15:09.248 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:08 smithi007 podman[115742]: 2024-06-13 07:15:08.981089575 +0000 UTC m=+3.197978148 container died 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-13T07:15:10.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:09 smithi007 podman[115742]: 2024-06-13 07:15:09.726920658 +0000 UTC m=+3.943809238 container cleanup 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-13T07:15:10.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:09 smithi007 bash[115742]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j 2024-06-13T07:15:10.163 INFO:journalctl@ceph.mds.j.smithi007.stdout:Jun 13 07:15:09 smithi007 podman[115754]: 2024-06-13 07:15:09.877974587 +0000 UTC m=+0.884668594 container remove 33bb3558c20dd5f1259bf563184279c00d94ed3a411b2622f6390f9d799428fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-13T07:15:10.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.j.service' 2024-06-13T07:15:10.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:10.519 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-13T07:15:10.519 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-13T07:15:10.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.b 2024-06-13T07:15:10.812 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:10 smithi175 systemd[1]: Stopping Ceph mds.b for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:11.072 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T07:15:10.812+0000 7f25defa2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:11.072 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:10 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b[66975]: 2024-06-13T07:15:10.812+0000 7f25defa2700 -1 mds.b *** got signal Terminated *** 2024-06-13T07:15:12.729 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:12 smithi175 podman[74211]: 2024-06-13 07:15:12.462079725 +0000 UTC m=+1.754370069 container died 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-13T07:15:13.104 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:12 smithi175 podman[74211]: 2024-06-13 07:15:12.829220658 +0000 UTC m=+2.121511009 container cleanup 539679fdd7916e3b8e50be3eb01091bc11ecf8f0edffca03025055ca7cb190ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-13T07:15:13.104 INFO:journalctl@ceph.mds.b.smithi175.stdout:Jun 13 07:15:12 smithi175 bash[74211]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-b 2024-06-13T07:15:13.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.b.service' 2024-06-13T07:15:13.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:13.284 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-13T07:15:13.284 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-13T07:15:13.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.e 2024-06-13T07:15:13.867 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jun 13 07:15:13 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-e[66353]: 2024-06-13T07:15:13.608+0000 7f3289f92700 -1 mds.e *** got signal Terminated *** 2024-06-13T07:15:17.224 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jun 13 07:15:16 smithi175 podman[74344]: 2024-06-13 07:15:16.957225637 +0000 UTC m=+3.445772894 container died 2a5dcef6be2b6778d0a8ce99ada6a606835e291fd80a93b9dcb3a50e336f9126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-13T07:15:17.591 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jun 13 07:15:17 smithi175 podman[74344]: 2024-06-13 07:15:17.324556083 +0000 UTC m=+3.813103339 container cleanup 2a5dcef6be2b6778d0a8ce99ada6a606835e291fd80a93b9dcb3a50e336f9126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-06-13T07:15:17.591 INFO:journalctl@ceph.mds.e.smithi175.stdout:Jun 13 07:15:17 smithi175 bash[74344]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-e 2024-06-13T07:15:17.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.e.service' 2024-06-13T07:15:17.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:17.763 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-13T07:15:17.763 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-13T07:15:17.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.h 2024-06-13T07:15:18.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:17 smithi175 systemd[1]: Stopping Ceph mds.h for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:18.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: 2024-06-13T07:15:18.094+0000 7fca3ecfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:18.117 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:18 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h[66655]: 2024-06-13T07:15:18.094+0000 7fca3ecfc700 -1 mds.h *** got signal Terminated *** 2024-06-13T07:15:20.711 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:20 smithi175 podman[74464]: 2024-06-13 07:15:20.461394286 +0000 UTC m=+2.472800668 container died 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-13T07:15:21.102 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:20 smithi175 podman[74464]: 2024-06-13 07:15:20.810932223 +0000 UTC m=+2.822338606 container cleanup 7e885081543e2591e0a1fef59b27c204b9822e49040b3191d73febbc4e290785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-13T07:15:21.102 INFO:journalctl@ceph.mds.h.smithi175.stdout:Jun 13 07:15:20 smithi175 bash[74464]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-h 2024-06-13T07:15:21.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.h.service' 2024-06-13T07:15:21.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:21.274 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-13T07:15:21.274 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-13T07:15:21.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.k 2024-06-13T07:15:21.605 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:21 smithi175 systemd[1]: Stopping Ceph mds.k for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:21.867 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k[66034]: 2024-06-13T07:15:21.606+0000 7fa5e3728700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:21.867 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:21 smithi175 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k[66034]: 2024-06-13T07:15:21.606+0000 7fa5e3728700 -1 mds.k *** got signal Terminated *** 2024-06-13T07:15:23.064 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:22 smithi175 podman[74586]: 2024-06-13 07:15:22.794336394 +0000 UTC m=+1.298998251 container died ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-13T07:15:23.367 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:23 smithi175 podman[74586]: 2024-06-13 07:15:23.148204292 +0000 UTC m=+1.652866149 container cleanup ee57faf5290150143e5a7f26e1e1a83034b8d440d8efc2570d0105287fc6ff7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-13T07:15:23.368 INFO:journalctl@ceph.mds.k.smithi175.stdout:Jun 13 07:15:23 smithi175 bash[74586]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-k 2024-06-13T07:15:23.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.k.service' 2024-06-13T07:15:23.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:23.602 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-13T07:15:23.603 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-13T07:15:23.603 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.c 2024-06-13T07:15:23.897 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:23 smithi189 systemd[1]: Stopping Ceph mds.c for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:24.161 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: 2024-06-13T07:15:23.897+0000 7faefc3d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:24.162 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:23 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c[67197]: 2024-06-13T07:15:23.897+0000 7faefc3d2700 -1 mds.c *** got signal Terminated *** 2024-06-13T07:15:28.206 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:27 smithi189 podman[75124]: 2024-06-13 07:15:27.911721084 +0000 UTC m=+4.101475831 container died 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-13T07:15:28.547 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:28 smithi189 podman[75124]: 2024-06-13 07:15:28.305853564 +0000 UTC m=+4.495608287 container cleanup 16209d14c56cfb38618bd00d3b8904cf9b7964e266fef1ec6c3f78089785f6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-13T07:15:28.547 INFO:journalctl@ceph.mds.c.smithi189.stdout:Jun 13 07:15:28 smithi189 bash[75124]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-c 2024-06-13T07:15:28.697 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.c.service' 2024-06-13T07:15:28.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:28.732 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-13T07:15:28.732 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-13T07:15:28.732 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.f 2024-06-13T07:15:29.058 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:28 smithi189 systemd[1]: Stopping Ceph mds.f for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:29.336 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f[66894]: 2024-06-13T07:15:29.058+0000 7fbad9cdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:29.337 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:29 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f[66894]: 2024-06-13T07:15:29.058+0000 7fbad9cdf700 -1 mds.f *** got signal Terminated *** 2024-06-13T07:15:32.225 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:31 smithi189 podman[75255]: 2024-06-13 07:15:31.916906739 +0000 UTC m=+2.965825079 container died bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-13T07:15:32.534 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:32 smithi189 podman[75255]: 2024-06-13 07:15:32.300830696 +0000 UTC m=+3.349749027 container cleanup bc1e457d6d8e00ab016a2b8505fd017b2c552860841b1183adf46d7c9289d4fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:15:32.534 INFO:journalctl@ceph.mds.f.smithi189.stdout:Jun 13 07:15:32 smithi189 bash[75255]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-f 2024-06-13T07:15:32.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.f.service' 2024-06-13T07:15:32.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:32.694 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-13T07:15:32.695 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-13T07:15:32.695 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.i 2024-06-13T07:15:32.836 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:32 smithi189 systemd[1]: Stopping Ceph mds.i for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:33.336 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: 2024-06-13T07:15:33.004+0000 7f730d59a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:33.336 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:33 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i[67513]: 2024-06-13T07:15:33.004+0000 7f730d59a700 -1 mds.i *** got signal Terminated *** 2024-06-13T07:15:36.753 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:36 smithi189 podman[75377]: 2024-06-13 07:15:36.451299899 +0000 UTC m=+3.539237709 container died cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-13T07:15:36.753 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:36 smithi189 podman[75377]: 2024-06-13 07:15:36.75460046 +0000 UTC m=+3.842538267 container cleanup cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-13T07:15:37.067 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:36 smithi189 bash[75377]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i 2024-06-13T07:15:37.067 INFO:journalctl@ceph.mds.i.smithi189.stdout:Jun 13 07:15:36 smithi189 podman[75390]: 2024-06-13 07:15:36.956035669 +0000 UTC m=+0.507611597 container remove cf002acecaacdbbde4a07b8a71f5e07de214ac29e92419ca74d8549de1c588a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-13T07:15:37.080 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.i.service' 2024-06-13T07:15:37.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:37.112 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-13T07:15:37.112 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-13T07:15:37.112 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.l 2024-06-13T07:15:37.336 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:37 smithi189 systemd[1]: Stopping Ceph mds.l for b9295806-294f-11ef-bc9c-c7b262605968... 2024-06-13T07:15:37.836 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l[66592]: 2024-06-13T07:15:37.415+0000 7fdb7c48a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-13T07:15:37.837 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:37 smithi189 ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l[66592]: 2024-06-13T07:15:37.415+0000 7fdb7c48a700 -1 mds.l *** got signal Terminated *** 2024-06-13T07:15:40.208 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:39 smithi189 podman[75498]: 2024-06-13 07:15:39.934063998 +0000 UTC m=+2.609094925 container died 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc) 2024-06-13T07:15:40.566 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:40 smithi189 podman[75498]: 2024-06-13 07:15:40.299882741 +0000 UTC m=+2.974913677 container cleanup 17d6ac0c5a2b1ced408dc54bbaa74c0292bc587877cfc5a60d39e01752176bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-06-13T07:15:40.566 INFO:journalctl@ceph.mds.l.smithi189.stdout:Jun 13 07:15:40 smithi189 bash[75498]: ceph-b9295806-294f-11ef-bc9c-c7b262605968-mds-l 2024-06-13T07:15:40.733 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9295806-294f-11ef-bc9c-c7b262605968@mds.l.service' 2024-06-13T07:15:40.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-13T07:15:40.764 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-13T07:15:40.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:15:41.101 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:15:46.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:15:46.463 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:15:49.008 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force --keep-logs 2024-06-13T07:15:49.197 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:15:51.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:15:51.755 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:15:51.794 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-13T07:15:51.821 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-13T07:15:51.823 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi007/crash 2024-06-13T07:15:51.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:15:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:15:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:15:51.928 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi175/crash 2024-06-13T07:15:51.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:15:51.991 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:15:51.991 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:15:51.993 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi189/crash 2024-06-13T07:15:51.994 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash -- . 2024-06-13T07:15:52.025 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/b9295806-294f-11ef-bc9c-c7b262605968/crash: Cannot open: No such file or directory 2024-06-13T07:15:52.025 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-06-13T07:15:52.026 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-13T07:15:52.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-13T07:15:52.204 INFO:teuthology.orchestra.run.smithi007.stdout:2024-06-13T06:57:57.319434+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-06-13T07:15:52.205 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-13T07:15:52.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-13T07:15:52.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-13T07:15:52.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-13T07:15:52.488 INFO:teuthology.orchestra.run.smithi007.stdout:2024-06-13T06:57:57.319434+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-06-13T07:15:52.489 INFO:tasks.cephadm:Compressing logs... 2024-06-13T07:15:52.490 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:15:52.493 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:15:52.496 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-13T07:15:52.531 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.2.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.f.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.5.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.c.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.c.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.z.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.i.log 2024-06-13T07:15:52.537 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:15:52.538 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.11.log 2024-06-13T07:15:52.538 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.8.log 2024-06-13T07:15:52.538 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:15:52.538 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.l.log 2024-06-13T07:15:52.541 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log: /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.l.log.gz 2024-06-13T07:15:52.547 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-06-13T07:15:52.547 INFO:teuthology.orchestra.run.smithi175.stderr: -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:15:52.547 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:15:52.547 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:15:52.547 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.7.log 2024-06-13T07:15:52.548 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log: --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.b.log 2024-06-13T07:15:52.549 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.4.log 2024-06-13T07:15:52.549 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.1.log 2024-06-13T07:15:52.549 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.y.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.h.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.10.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.e.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.b.log 2024-06-13T07:15:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.k.log 2024-06-13T07:15:52.554 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.b.log: /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.e.log.gz 2024-06-13T07:15:52.555 INFO:teuthology.orchestra.run.smithi175.stderr: 94.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.k.log.gz 2024-06-13T07:15:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-13T07:15:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-13T07:15:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-13T07:15:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log 2024-06-13T07:15:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log 2024-06-13T07:15:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.g.log 2024-06-13T07:15:52.560 INFO:teuthology.orchestra.run.smithi007.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-13T07:15:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.j.log 2024-06-13T07:15:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.3.log 2024-06-13T07:15:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.0.log 2024-06-13T07:15:52.563 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:15:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.x.log 2024-06-13T07:15:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.a.log 2024-06-13T07:15:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.6.log 2024-06-13T07:15:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log 2024-06-13T07:15:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.a.log 2024-06-13T07:15:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log: 94.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.j.log.gz 2024-06-13T07:15:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log 2024-06-13T07:15:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.d.log 2024-06-13T07:15:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.9.log 2024-06-13T07:15:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.d.log: /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.9.log: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:15:52.577 INFO:teuthology.orchestra.run.smithi007.stderr: 94.2% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.d.log.gz 2024-06-13T07:15:52.590 INFO:teuthology.orchestra.run.smithi007.stderr: 83.1% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:15:52.659 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:15:52.663 INFO:teuthology.orchestra.run.smithi175.stderr: 83.1% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:15:52.672 INFO:teuthology.orchestra.run.smithi189.stderr: 93.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.z.log.gz 2024-06-13T07:15:52.688 INFO:teuthology.orchestra.run.smithi175.stderr: 87.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:15:52.705 INFO:teuthology.orchestra.run.smithi189.stderr: 93.8% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.c.log.gz 2024-06-13T07:15:52.706 INFO:teuthology.orchestra.run.smithi175.stderr: 90.3% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:15:52.709 INFO:teuthology.orchestra.run.smithi189.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-13T07:15:52.711 INFO:teuthology.orchestra.run.smithi175.stderr: 91.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:15:52.716 INFO:teuthology.orchestra.run.smithi189.stderr: 94.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.f.log.gz 2024-06-13T07:15:52.731 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:15:52.731 INFO:teuthology.orchestra.run.smithi189.stderr: 87.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.log.gz 2024-06-13T07:15:52.738 INFO:teuthology.orchestra.run.smithi189.stderr: 90.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:15:52.749 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.audit.log.gz 2024-06-13T07:15:52.801 INFO:teuthology.orchestra.run.smithi189.stderr: 83.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph.cephadm.log.gz 2024-06-13T07:15:52.816 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-volume.log.gz 2024-06-13T07:15:52.823 INFO:teuthology.orchestra.run.smithi189.stderr: 94.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.i.log.gz 2024-06-13T07:15:53.103 INFO:teuthology.orchestra.run.smithi175.stderr: 93.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.y.log.gz 2024-06-13T07:15:53.581 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mgr.x.log.gz 2024-06-13T07:15:54.836 INFO:teuthology.orchestra.run.smithi175.stderr: 94.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.h.log.gz 2024-06-13T07:15:57.197 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.c.log.gz 2024-06-13T07:15:58.463 INFO:teuthology.orchestra.run.smithi175.stderr: 91.7% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.b.log.gz 2024-06-13T07:16:01.338 INFO:teuthology.orchestra.run.smithi007.stderr: 90.1% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mon.a.log.gz 2024-06-13T07:16:06.804 INFO:teuthology.orchestra.run.smithi175.stderr: 93.6% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.b.log.gz 2024-06-13T07:16:06.862 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.a.log.gz 2024-06-13T07:16:09.201 INFO:teuthology.orchestra.run.smithi007.stderr: 94.1% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-mds.g.log.gz 2024-06-13T07:17:43.599 INFO:teuthology.orchestra.run.smithi175.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.1.log.gz 2024-06-13T07:18:16.956 INFO:teuthology.orchestra.run.smithi175.stderr: 90.6% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.10.log.gz 2024-06-13T07:18:32.464 INFO:teuthology.orchestra.run.smithi175.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.7.log.gz 2024-06-13T07:19:11.675 INFO:teuthology.orchestra.run.smithi175.stderr: 90.7% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.4.log.gz 2024-06-13T07:19:11.677 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-06-13T07:19:11.677 INFO:teuthology.orchestra.run.smithi175.stderr:real 3m19.166s 2024-06-13T07:19:11.677 INFO:teuthology.orchestra.run.smithi175.stderr:user 4m45.557s 2024-06-13T07:19:11.677 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m9.734s 2024-06-13T07:21:13.005 INFO:teuthology.orchestra.run.smithi189.stderr: 90.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.8.log.gz 2024-06-13T07:22:26.386 INFO:teuthology.orchestra.run.smithi189.stderr: 90.8% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.2.log.gz 2024-06-13T07:23:15.264 INFO:teuthology.orchestra.run.smithi189.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.11.log.gz 2024-06-13T07:23:57.381 INFO:teuthology.orchestra.run.smithi189.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.5.log.gz 2024-06-13T07:23:57.383 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-06-13T07:23:57.383 INFO:teuthology.orchestra.run.smithi189.stderr:real 8m4.872s 2024-06-13T07:23:57.383 INFO:teuthology.orchestra.run.smithi189.stderr:user 5m41.758s 2024-06-13T07:23:57.383 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m13.026s 2024-06-13T07:24:51.649 INFO:teuthology.orchestra.run.smithi007.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.0.log.gz 2024-06-13T07:25:16.665 INFO:teuthology.orchestra.run.smithi007.stderr: 90.5% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.6.log.gz 2024-06-13T07:25:31.128 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.3.log.gz 2024-06-13T07:26:37.302 INFO:teuthology.orchestra.run.smithi007.stderr: 90.6% -- replaced with /var/log/ceph/b9295806-294f-11ef-bc9c-c7b262605968/ceph-osd.9.log.gz 2024-06-13T07:26:37.311 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-06-13T07:26:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:real 10m44.787s 2024-06-13T07:26:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:user 14m22.579s 2024-06-13T07:26:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:sys 1m38.719s 2024-06-13T07:26:37.313 INFO:tasks.cephadm:Archiving logs... 2024-06-13T07:26:37.314 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi007/log 2024-06-13T07:26:37.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:27:46.977 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi175/log 2024-06-13T07:27:46.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:28:37.152 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi189/log 2024-06-13T07:28:37.153 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-13T07:29:37.705 INFO:tasks.cephadm:Removing cluster... 2024-06-13T07:29:37.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force 2024-06-13T07:29:38.087 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:29:40.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force 2024-06-13T07:29:40.375 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:29:41.041 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9295806-294f-11ef-bc9c-c7b262605968 --force 2024-06-13T07:29:41.433 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: b9295806-294f-11ef-bc9c-c7b262605968 2024-06-13T07:29:42.168 INFO:tasks.cephadm:Removing cephadm ... 2024-06-13T07:29:42.168 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:29:42.241 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:29:42.260 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-13T07:29:42.275 INFO:tasks.cephadm:Teardown complete 2024-06-13T07:29:42.276 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-13T07:29:42.286 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-13T07:29:42.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:29:42.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:29:42.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-13T07:29:42.458 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-13T07:29:42.468 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-13T07:29:42.468 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:29:42.476 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:29:42.478 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-13T07:29:42.492 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:29:42.493 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:29:42.526 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-06-13T07:29:42.532 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:29:42.533 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-06-13T07:29:42.533 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1648us[+1667us] +/- 68ms 2024-06-13T07:29:42.533 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -375us[ -375us] +/- 66ms 2024-06-13T07:29:42.533 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -1121us[-1121us] +/- 71ms 2024-06-13T07:29:42.533 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:29:42.643 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:29:42.643 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-06-13T07:29:42.643 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1576us[+1578us] +/- 68ms 2024-06-13T07:29:42.644 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -455us[ -454us] +/- 66ms 2024-06-13T07:29:42.644 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -1212us[-1211us] +/- 71ms 2024-06-13T07:29:42.644 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:29:42.808 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-13T07:29:42.808 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-06-13T07:29:42.808 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1636us[+1649us] +/- 68ms 2024-06-13T07:29:42.809 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -432us[ -432us] +/- 66ms 2024-06-13T07:29:42.809 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -1181us[-1169us] +/- 71ms 2024-06-13T07:29:42.809 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-13T07:29:42.810 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-13T07:29:42.820 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-13T07:29:42.821 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-13T07:29:42.851 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:29:43.162 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:29:43.305 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-13T07:29:43.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:29:43.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:43.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:29:43.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:43.560 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-13T07:29:43.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:43.590 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-13T07:29:43.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-13T07:29:43.635 INFO:teuthology.task.internal:Duration was 3500.502658 seconds 2024-06-13T07:29:43.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-13T07:29:43.684 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-13T07:29:43.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:29:43.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:29:43.696 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-13T07:29:43.748 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:29:43.759 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:29:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-13T07:29:44.290 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-13T07:29:44.290 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-06-13T07:29:44.290 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:29:44.362 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-06-13T07:29:44.363 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:29:44.399 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-06-13T07:29:44.400 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-13T07:29:44.430 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-13T07:29:44.431 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:29:44.438 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:29:44.442 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-13T07:29:44.526 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-13T07:29:44.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:29:44.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:29:44.534 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-13T07:29:45.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-13T07:29:45.737 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-13T07:29:45.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:29:45.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:29:45.879 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-13T07:29:45.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-13T07:29:45.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:45.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:45.931 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:45.964 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-06-13T07:29:45.990 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-06-13T07:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-06-13T07:29:46.090 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:46.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:46.141 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:46.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:46.158 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-13T07:29:46.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-13T07:29:46.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-13T07:29:46.185 INFO:teuthology.task.internal:Transferring archived files... 2024-06-13T07:29:46.187 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi007 2024-06-13T07:29:46.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:29:46.308 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi175 2024-06-13T07:29:46.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:29:46.349 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-12_21:24:03-fs-squid-distro-default-smithi/7752742/remote/smithi189 2024-06-13T07:29:46.349 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-13T07:29:46.389 INFO:teuthology.task.internal:Removing archive directory... 2024-06-13T07:29:46.389 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:29:46.397 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:29:46.400 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-13T07:29:46.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-13T07:29:46.462 INFO:teuthology.task.internal:Not uploading archives. 2024-06-13T07:29:46.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-13T07:29:46.499 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-13T07:29:46.499 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:29:46.507 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:29:46.509 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-13T07:29:46.524 INFO:teuthology.orchestra.run.smithi175.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:29 /home/ubuntu/cephtest 2024-06-13T07:29:46.526 INFO:teuthology.orchestra.run.smithi189.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:29 /home/ubuntu/cephtest 2024-06-13T07:29:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 13 07:29 /home/ubuntu/cephtest 2024-06-13T07:29:46.563 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-13T07:29:46.576 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-13T07:29:46.657 INFO:teuthology.run:Summary data: client.0-kernel-sha1: ff88c41504f3e4c4268de18904a6f347f1b3f2d6 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 3500.50265789032 failure_reason: '"2024-06-13T06:57:57.319434+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/ea6338a6-2091-4fef-8ddd-3ceebf01a0d5/postgres/data/base/13494) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-13T07:29:46.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-13T07:29:46.757 INFO:teuthology.run:FAIL